RAC - Add grid 3rd node and oracle instance in 12.2 grid cluster
Operating System OEL 7.8
2 node Grid RAC - rac1,rac2 instance
Version : 12.2.0
Adding rac3 instance
hostname : srv1.example.com, srv2.example.com, srv3.example.com.
Grid Level user
- Add new public, private and vip IP address in all hosts.
- Add required package and pre requestors on new grid and oracle instance .
- Check scan IP nslookup at new node
- Check oracleasm storage configuration (oracleasm listdisks)
- Make passwordless ssh connection to oracle and grid user to new host.
- Run cluster verify utility of new node . (if all passed proceed further to add node)
- Run add node silently or using GUI
- Run root.sh
- Add node sh file on oracle and run root.sh
- Add instance using dbca from master node or existing instance not from new instance
- Add passwordless shh connection for oracle user
[root@srv1 sshsetup]# ./sshUserSetup.sh -user oracle -hosts "srv1 srv2 srv3" -advanced -exverify –noPromptPassphrase
- Add passwordless shh connection for grid user
[root@srv1 sshsetup]# ./sshUserSetup.sh -user grid -hosts "srv1 srv2 srv3" -advanced -exverify –noPromptPassphrase
- Check cluvfy utility and if any issue found fix prequestories
[grid@srv1 bin]$ cluvfy stage -pre nodeadd -n srv3 -fixup -fixupnoexec
- Existing information of clusterware resource
[grid@srv1 bin]$ crsctl status resource -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
ONLINE ONLINE srv1 STABLE
ONLINE ONLINE srv2 STABLE
ora.CRS.dg
ONLINE ONLINE srv1 STABLE
ONLINE ONLINE srv2 STABLE
ora.DATA.dg
ONLINE ONLINE srv1 STABLE
ONLINE ONLINE srv2 STABLE
ora.FRA.dg
ONLINE ONLINE srv1 STABLE
ONLINE ONLINE srv2 STABLE
ora.LISTENER.lsnr
ONLINE ONLINE srv1 STABLE
ONLINE ONLINE srv2 STABLE
ora.chad
ONLINE ONLINE srv1 STABLE
ONLINE ONLINE srv2 STABLE
ora.net1.network
ONLINE ONLINE srv1 STABLE
ONLINE ONLINE srv2 STABLE
ora.ons
ONLINE ONLINE srv1 STABLE
ONLINE ONLINE srv2 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE srv1 STABLE
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE srv2 STABLE
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE srv2 STABLE
ora.MGMTLSNR
1 ONLINE ONLINE srv2 169.254.238.221 192.
168.10.2,STABLE
ora.asm
1 ONLINE ONLINE srv1 Started,STABLE
2 ONLINE ONLINE srv2 Started,STABLE
3 OFFLINE OFFLINE STABLE
ora.cvu
1 ONLINE ONLINE srv2 STABLE
ora.mgmtdb
1 ONLINE ONLINE srv2 Open,STABLE
ora.qosmserver
1 ONLINE ONLINE srv2 STABLE
ora.rac.db
1 ONLINE ONLINE srv1 Open,HOME=/u01/app/o
racle/product/12.2.0
/db_1,STABLE
2 ONLINE ONLINE srv2 Open,HOME=/u01/app/o
racle/product/12.2.0
/db_1,STABLE
ora.scan1.vip
1 ONLINE ONLINE srv1 STABLE
ora.scan2.vip
1 ONLINE ONLINE srv2 STABLE
ora.scan3.vip
1 ONLINE ONLINE srv2 STABLE
ora.srv1.vip
1 ONLINE ONLINE srv1 STABLE
ora.srv2.vip
1 ONLINE ONLINE srv2 STABLE
--------------------------------------------------------------------------------
[grid@srv1 bin]$
[grid@srv1 bin]$ cat /etc/hosts
127.0.0.1 localhost.localdomain localhost
# Public
192.168.56.71 srv1.example.com srv1
192.168.56.72 srv2.example.com srv2
192.168.56.73 srv3.example.com srv3
192.168.56.74 srv4.example.com srv4
# Private
192.168.10.1 srv1-priv.example.com srv1-priv
192.168.10.2 srv2-priv.example.com srv2-priv
192.168.10.3 srv3-priv.example.com srv3-priv
192.168.10.4 srv4-priv.example.com srv4-priv
# Virtual
192.168.56.81 srv1-vip.example.com srv1-vip
192.168.56.82 srv2-vip.example.com srv2-vip
192.168.56.83 srv3-vip.example.com srv3-vip
192.168.56.84 srv4-vip.example.com srv4-vip
# SCAN (in production this should be configured in DNS)
192.168.56.91 srv-scan.example.com srv-scan
192.168.56.92 srv-scan.example.com srv-scan
192.168.56.93 srv-scan.example.com srv-scan
- Add node using commands prompt using slient installation
- Using GUI adding node
- Run root.sh scripts ( root user)
[root@srv3 ~]# /u01/app/12.2.0/grid/root.shPerforming root user operation.The following environment variables are set as:ORACLE_OWNER= gridORACLE_HOME= /u01/app/12.2.0/gridEnter the full pathname of the local bin directory: [/usr/local/bin]:Copying dbhome to /usr/local/bin ...Copying oraenv to /usr/local/bin ...Copying coraenv to /usr/local/bin ...Entries will be added to the /etc/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root script.Now product-specific root actions will be performed.Relinking oracle with rac_on optionUsing configuration parameter file: /u01/app/12.2.0/grid/crs/install/crsconfig_paramsThe log of current session can be found at:/u01/app/grid/crsdata/srv3/crsconfig/rootcrs_srv3_2021-10-24_10-25-34PM.log2021/10/24 22:26:41 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.2021/10/24 22:26:41 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.2021/10/24 22:27:11 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.2021/10/24 22:27:11 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.2021/10/24 22:27:13 CLSRSC-363: User ignored prerequisites during installation2021/10/24 22:27:13 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.2021/10/24 22:27:14 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.2021/10/24 22:27:17 CLSRSC-594: Executing installation step 5 of 19: 'SaveParamFile'.2021/10/24 22:27:20 CLSRSC-594: Executing installation step 6 of 19: 'SetupOSD'.2021/10/24 22:27:21 CLSRSC-594: Executing installation step 7 of 19: 'CheckCRSConfig'.2021/10/24 22:27:22 CLSRSC-594: Executing installation step 8 of 19: 'SetupLocalGPNP'.2021/10/24 22:27:23 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.2021/10/24 22:27:37 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.2021/10/24 22:27:37 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.2021/10/24 22:27:38 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.2021/10/24 22:27:54 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'2021/10/24 22:28:43 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.2021/10/24 22:28:44 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'srv3'CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'srv3' has completedCRS-4133: Oracle High Availability Services has been stopped.CRS-4123: Oracle High Availability Services has been started.2021/10/24 22:29:11 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.2021/10/24 22:29:12 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'srv3'CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'srv3' has completedCRS-4133: Oracle High Availability Services has been stopped.CRS-4123: Oracle High Availability Services has been started.CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'srv3'CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'srv3' has completedCRS-4133: Oracle High Availability Services has been stopped.2021/10/24 22:29:46 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.CRS-4123: Starting Oracle High Availability Services-managed resourcesCRS-2672: Attempting to start 'ora.evmd' on 'srv3'CRS-2672: Attempting to start 'ora.mdnsd' on 'srv3'CRS-2676: Start of 'ora.mdnsd' on 'srv3' succeededCRS-2676: Start of 'ora.evmd' on 'srv3' succeededCRS-2672: Attempting to start 'ora.gpnpd' on 'srv3'CRS-2676: Start of 'ora.gpnpd' on 'srv3' succeededCRS-2672: Attempting to start 'ora.gipcd' on 'srv3'CRS-2676: Start of 'ora.gipcd' on 'srv3' succeededCRS-2672: Attempting to start 'ora.cssdmonitor' on 'srv3'CRS-2676: Start of 'ora.cssdmonitor' on 'srv3' succeededCRS-2672: Attempting to start 'ora.cssd' on 'srv3'CRS-2672: Attempting to start 'ora.diskmon' on 'srv3'CRS-2676: Start of 'ora.diskmon' on 'srv3' succeededCRS-2676: Start of 'ora.cssd' on 'srv3' succeededCRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'srv3'CRS-2672: Attempting to start 'ora.ctssd' on 'srv3'CRS-2676: Start of 'ora.ctssd' on 'srv3' succeededCRS-2676: Start of 'ora.cluster_interconnect.haip' on 'srv3' succeededCRS-2672: Attempting to start 'ora.asm' on 'srv3'CRS-2676: Start of 'ora.asm' on 'srv3' succeededCRS-2672: Attempting to start 'ora.storage' on 'srv3'CRS-2676: Start of 'ora.storage' on 'srv3' succeededCRS-2672: Attempting to start 'ora.crf' on 'srv3'CRS-2676: Start of 'ora.crf' on 'srv3' succeededCRS-2672: Attempting to start 'ora.crsd' on 'srv3'CRS-2676: Start of 'ora.crsd' on 'srv3' succeededCRS-6017: Processing resource auto-start for servers: srv3CRS-2673: Attempting to stop 'ora.LISTENER_SCAN2.lsnr' on 'srv2'CRS-2672: Attempting to start 'ora.net1.network' on 'srv3'CRS-2672: Attempting to start 'ora.chad' on 'srv3'CRS-2672: Attempting to start 'ora.ASMNET1LSNR_ASM.lsnr' on 'srv3'CRS-2677: Stop of 'ora.LISTENER_SCAN2.lsnr' on 'srv2' succeededCRS-2676: Start of 'ora.net1.network' on 'srv3' succeededCRS-2673: Attempting to stop 'ora.scan2.vip' on 'srv2'CRS-2672: Attempting to start 'ora.ons' on 'srv3'CRS-2676: Start of 'ora.chad' on 'srv3' succeededCRS-2677: Stop of 'ora.scan2.vip' on 'srv2' succeededCRS-2672: Attempting to start 'ora.scan2.vip' on 'srv3'CRS-2676: Start of 'ora.scan2.vip' on 'srv3' succeededCRS-2672: Attempting to start 'ora.LISTENER_SCAN2.lsnr' on 'srv3'CRS-2676: Start of 'ora.ons' on 'srv3' succeededCRS-2676: Start of 'ora.ASMNET1LSNR_ASM.lsnr' on 'srv3' succeededCRS-2672: Attempting to start 'ora.asm' on 'srv3'CRS-2676: Start of 'ora.LISTENER_SCAN2.lsnr' on 'srv3' succeededCRS-2676: Start of 'ora.asm' on 'srv3' succeededCRS-2672: Attempting to start 'ora.FRA.dg' on 'srv3'CRS-2676: Start of 'ora.FRA.dg' on 'srv3' succeededCRS-6016: Resource auto-start has completed for server srv3CRS-6024: Completed start of Oracle Cluster Ready Services-managed resourcesCRS-4123: Oracle High Availability Services has been started.2021/10/24 22:32:51 CLSRSC-343: Successfully started Oracle Clusterware stack2021/10/24 22:32:51 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.clscfg: EXISTING configuration version 5 detected.clscfg: version 5 is 12c Release 2.Successfully accumulated necessary OCR keys.Creating OCR keys for user 'root', privgrp 'root'..Operation successful.2021/10/24 22:33:10 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.2021/10/24 22:33:23 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded[root@srv3 ~]#
- Clusterware status
- Add instance using oracle user
- Run root.sh using root user
- Set oracle environment and run ./dbca for adding instance
- Check database instance status