jump to navigation

Cluster Resource Control Filters May 3, 2011

Posted by tamnau in 11g, RAC.
Tags: , , , , , , , , , ,
1 comment so far

In 11.2 RAC, to avoid listing all cluster resources, you can use the filter switch option “-w”.  This can be useful if you only want to look at a specific status of a given resource.

A. List for All Databases Managed by Cluster

In the example, shows the listing of status for all databases managed by the cluster.

[oracle@lnx01] export ORACLE_SID=GRID; . oraenv
[oracle@lnx01] crsctl stat res -t -w "TYPE = ora.database.type"

The output for the above example is as follows:
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.salt.db
      1        ONLINE  ONLINE       lnx01                 Open                
      2        ONLINE  ONLINE       lnx02                 Open                
ora.pepper.db
      1        ONLINE  ONLINE       lnx01                 Open                
      2        ONLINE  ONLINE       lnx02                 Open                

A list below are some type of resources that the filter can set for:

NOTE: The filter from the example can be change from “ora.database.type” to “ora.service.type”. For more options see listing below.

[oracle@lnx01] crsctl stat res |grep "TYPE=" |uniq

Sample output:
TYPE=ora.diskgroup.type
TYPE=ora.listener.type
TYPE=ora.scan_listener.type
TYPE=ora.diskgroup.type
TYPE=application
TYPE=ora.asm.type
TYPE=ora.cluster_vip_net1.type
TYPE=ora.cvu.type
TYPE=ora.database.type
TYPE=ora.service.type
TYPE=ora.database.type
TYPE=ora.service.type
TYPE=ora.gsd.type
TYPE=ora.network.type
TYPE=ora.oc4j.type
TYPE=ora.ons.type
TYPE=ora.scan_vip.type
TYPE=ora.database.type

B. Check Cluster Resources that are offline

Example:
[oracle@lnx01] crsctl stat res -t -w "STATE = OFFLINE"

NOTE: If not using GSD, then it is ok for the following resource to be offline.

Sample output
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.gsd
OFFLINE OFFLINE      lnx01
OFFLINE OFFLINE      lxn02

C. Verifying local resources that are online

Example:
[oracle@lnx01] crsctl stat res -init -t -w "STATE = ONLINE"

NOTE: You can use “-init” to check Local Resources

Sample Output
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
1        ONLINE  ONLINE       lnx01                 Started
ora.cluster_interconnect.haip
1        ONLINE  ONLINE       lnx01
ora.crf
1        ONLINE  ONLINE       lnx01
ora.crsd
1        ONLINE  ONLINE       lnx01
ora.cssd
1        ONLINE  ONLINE       lnx01
ora.cssdmonitor
1        ONLINE  ONLINE       lnx01
ora.ctssd
1        ONLINE  ONLINE       lnx01                 OBSERVER
ora.diskmon
1        ONLINE  ONLINE       lnx01
ora.drivers.acfs
1        ONLINE  ONLINE       lnx01
ora.evmd
1        ONLINE  ONLINE       lnx01
ora.gipcd
1        ONLINE  ONLINE       lnx01
ora.gpnpd
1        ONLINE  ONLINE       lnx01
ora.mdnsd
1        ONLINE  ONLINE       lnx01

D. To check for resources that are Not Online

When I run the “crsctl stat res -t” and it print  lines and lines that go on forever of all the resources for your cluster, I find that at times you can easily oversee resources that are in a pending or bad status that need special attention.  Don’t get me wrong, it is nice to know about all your resources. However, there is a better way that you can do a status check, and report only the pending or bad resources.

Note that the “crsctl stat res -t -w “((TARGET = OFFLINE) or (STATE = OFFLINE)” will only report resources with the OFFLINE target or state.  It will not report other target or state information.  Hence I will show you below in an example on how to capture resources with pending (starting, intermediate) or bad (offline, unknown etc…)  status.  Therefore I want to report all resources with the target or state that is not ONLINE. In addition, I want to also exclude that annoying  gsd (ora.gsd) resource from the check, as we are not hosting any oracle 9i RAC database on this cluster, therefore the target and state for this resource will always show as being offline.

Example: [oracle@lnx01] crsctl stat res -t -w "((TARGET != ONLINE) or (STATE != ONLINE) and TYPE != ora.gsd))"

NOTE: No output is shown if cluster is up and running.  Otherwise impacted resources will be displayed.

SCAN Reconfiguration July 30, 2010

Posted by tamnau in 11g, General, RAC.
Tags: , , , , ,
11 comments

As you may know, SCAN (Single Client Access Name) is a new feature that has been introduced in 11.2 Oracle RAC. To put it simply, the SCAN is actually a single name alias which can be configured for clients to connect to the cluster database. In addition it will also handle the load balancing and failover for client connections. When the nodes in the cluster changes (eg. added or removed), the benefits of SCAN can be realised, as there is no requirement for the client to be reconfigured.

A good example on configuring SCAN for clients, Pas from Oracle has written an article which explains how this can be achieved. Just visit his blog entry Using SCAN – Single Client Access Name to Connect to 11g R2 RAC from JDeveloper 11g.

Configuring your clients to used SCAN in 11.2 or later is optional, however when you install the Grid Infrastructure software for RAC, the SCAN is still required to be configured for setup of the cluster.

It is recommended that the SCAN is resolvable to 3 IP addresses. However a minimum of 1 IP Address may be configured. It should also be registered in DNS for round robin resolution.

Reconfiguration of SCAN

There may be situations where reconfiguration of the SCAN for the cluster is required.
Some examples are:

  • Allocation of IP addresses has changed for SCAN
  • Adding additional IP addresses for SCAN
  • DNS Changes associated with SCAN addressing
  • SCAN name change^

NOTE: ^ SCAN name change may be required if there is conflicting name for another cluster on same network or in DNS, however this would mean that client reference to old SCAN is required be updated. Hence caution should be taken to ensure that SCAN name is unique in DNS and network.

In the below scenario, the following steps will show how SCAN addressing can be reconfigured for a cluster.

At the time of the installation, the DNS registration had not been setup. As a workaround, one of the IP addresses allocated for the SCAN was referenced in the /etc/hosts file. Once the installation was completed and DNS registration updated, the SCAN was then reconfigured to be resolvable through DNS for all 3 IP addresses.

Below is the summary of the current and new configuration for SCAN.

Current New
SCAN Name myscan myscan
SCAN IP Address 172.43.22.89 172.43.22.89
172.43.22.90
172.43.22.91
SCAN entry in /etc/hosts file Yes* No
Registered in DNS and configured for round robin No Yes

 

NOTE: SCAN entry in the /etc/hosts file is not recommended, hence the reconfiguration to the NEW is required.

1. Verify Current Configuration for SCAN VIP

oracle@lnx01[GRID]:/app/oracle> srvctl config scan
SCAN name: myscan, Network: 1/172.43.22.0/255.255.255.0/bge0:nxge0
SCAN VIP name: scan1, IP: /myscan.earth.com/172.43.22.89

2. Verify Status for SCAN VIP

oracle@lnx01[GRID]:/app/oracle> srvctl status scan
SCAN VIP scan1 is enabled
SCAN VIP scan1 is running on node lnx02

3. Verify Current Configuration for SCAN Listener

oracle@lnx01[GRID]:/app/oracle> srvctl config scan_listener
SCAN Listener LISTENER_SCAN1 exists. Port: TCP:1521

4. Verify Status for SCAN Listener

oracle@lnx01[GRID]:/app/oracle> srvctl status scan_listener
SCAN Listener LISTENER_SCAN1 is enabled
SCAN listener LISTENER_SCAN1 is running on node lnx02

5. Stop the SCAN Listener

oracle@lnx01[GRID]:/app/oracle> srvctl stop scan_listener

6. Stop the SCAN VIPs

oracle@lnx01[GRID]:/app/oracle> srvctl stop scan

7. Verify Status for SCAN Listener has stopped

oracle@lnx01[GRID]:/app/oracle> srvctl status scan_listener
SCAN Listener LISTENER_SCAN1 is enabled
SCAN listener LISTENER_SCAN1 is not running

8. Verify Status for SCAN VIP has stopped

oracle@lnx01[GRID]:/app/oracle> srvctl status scan
SCAN VIP scan1 is enabled
SCAN VIP scan1 is not running

9. Update SCAN in DNS and associated IPs as required

10. Verify that resolv.conf is configured to search DNS domain that SCAN was registered for

oracle@lnx01[GRID]:/app/oracle> cat /etc/resolv.conf
search        earth.com
nameserver      172.43.7.1
nameserver      172.44.7.1

11. Verify that in DNS that that the SCAN is resolvable against all three IP Addresses

oracle@lnx01[GRID]:/app/oracle> nslookup myscan
Server:         172.43.7.1
Address:        172.43.7.1#53
Name:   myscan.earth.com
Address: 172.43.22.89
Name:   myscan.earth.com
Address: 172.43.22.90
Name:   myscan.earth.com
Address: 172.43.22.91

12. Verify that the scan IP and address name has been removed or commented out from the /etc/hosts file:

oracle@lnx01[GRID]:/app/oracle> grep myscan /etc/hosts

NOTE: In this example no rows are returned as IP and address name has been removed for scan

13. Modify the SCAN resource so that it update all SCAN VIPs resolvable by DNS

oracle@lnx01[GRID]:/app/oracle> sudo srvctl modify scan -n myscan

14. If the SCAN name was changed as well, run the following to update in Cluster Resource Control

oracle@lnx01[GRID]:/app/oracle> sudo crsctl modify type ora.scan_vip.type -attr "ATTRIBUTE=SCAN_NAME,DEFAULT_VALUE=myscan"

15. Verify that all SCAN VIPs are now resolvable and registered with the SCAN resource

oracle@lnx01[GRID]:/app/oracle> srvctl config scan
SCAN name: myscan, Network: 1/172.43.22.0/255.255.255.0/bge0:nxge0
SCAN VIP name: scan1, IP: /myscan.earth.com/172.43.22.90
SCAN VIP name: scan2, IP: /myscan.earth.com/172.43.22.91
SCAN VIP name: scan3, IP: /myscan.earth.com/172.43.22.89

16. Update the SCAN Listener with new SCAN VIPs identified from DNS

oracle@lnx01[GRID]:/app/oracle> srvctl modify scan_listener -u

17. Verify new SCAN Listeners registered for new SCAN VIPs

oracle@lnx01[GRID]:/app/oracle> srvctl config scan_listener
SCAN Listener LISTENER_SCAN1 exists. Port: TCP:1521
SCAN Listener LISTENER_SCAN2 exists. Port: TCP:1521
SCAN Listener LISTENER_SCAN3 exists. Port: TCP:1521

18. Start SCAN Listener

NOTE: This will also start the SCAN VIPs

oracle@lnx01[GRID]:/app/oracle> srvctl start scan_listener

19. Verify Status of all three SCAN VIPs have been enabled and are running.

oracle@lnx01[GRID]:/app/oracle> srvctl status scan
SCAN VIP scan1 is enabled
SCAN VIP scan1 is running on node lnx02
SCAN VIP scan2 is enabled
SCAN VIP scan2 is running on node lnx01
SCAN VIP scan3 is enabled
SCAN VIP scan3 is running on node lnx01

20. Verify Status of all three SCAN Listeners have been enabled and are running.

oracle@lnx01[GRID]:/app/oracle> srvctl status scan_listener
SCAN Listener LISTENER_SCAN1 is enabled
SCAN listener LISTENER_SCAN1 is running on node lnx02
SCAN Listener LISTENER_SCAN2 is enabled
SCAN listener LISTENER_SCAN2 is running on node lnx01
SCAN Listener LISTENER_SCAN3 is enabled
SCAN listener LISTENER_SCAN3 is running on node lnx01