OraVR: RAC Installation on 19c

Install 19c 3 node RAC
1. Install Esxi Host
3. Configure RHEL 7.8
2. DNS Server
3. Install Storage Openfiler
4. Install Redhat 7.8
5. Configure the nodes
6. Grid Setup
7. RDBMS setup
8. DBCA
9. Patch the database
10. validations

1. Network Interfaces :

. Private Network

. Public Netwok

. Optional for access internet (depend on requirement)

--/etc/hosts : Scan ip must be resolve in DNS

#----------------------------------woravr-----------------------------#

#Public

192.168.45.101 woravr01.vishalbh.com woravr01

192.168.45.102 woravr02.vishalbh.com woravr02

192.168.45.103 woravr03.vishalbh.com woravr03

#Private

192.168.46.101 woravr01-priv.vishalbh.com woravr01-priv

192.168.46.102 woravr02-priv.vishalbh.com woravr02-priv

192.168.46.103 woravr03-priv.vishalbh.com woravr03-priv

#vip

192.168.45.104 woravr01-vip.vishalbh.com woravr01-vip

192.168.45.105 woravr02-vip.vishalbh.com woravr02-vip

192.168.45.106 woravr03-vip.vishalbh.com woravr03-vip

#scan

192.168.45.107 woravrscan.vishalbh.com woravrscan

192.168.45.108 woravrscan.vishalbh.com woravrscan

192.168.45.109 woravrscan.vishalbh.com woravrscan

#----------------------------------woravr-----------------------------#

--/etc/resolve.conf

# Generated by NetworkManager

search vishalbh.com #--- DNS server

nameserver 192.168.45.2 #--- your domain name server

More DNS configuration details :

[root@woravr02 ~]# nslookup woravrscan.vishalbh.com

Server: 192.168.45.2

Address: 192.168.45.2#53

Name: woravrscan.vishalbh.com

Address: 192.168.45.107

Name: woravrscan.vishalbh.com

Address: 192.168.45.108

Name: woravrscan.vishalbh.com

Address: 192.168.45.109

2. Install required RPM's '

yum update -y

yum install xorg-x11-server-Xorg xorg-x11-xauth xorg-x11-apps -y

yum install -y oracle-database-preinstall-19c.x86_64

yum install oracleasm-support

yum install bind* -y

sysctl -p

3. Stop the firewall

sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

systemctl stop firewalld

systemctl disable firewalld

systemctl status firewalld

4. Start the NTP sever or time server

systemctl enable chronyd.service

systemctl restart chronyd.service

chronyc -a 'burst 4/4'

chronyc -a makestep

5. User managment

groupadd -g 54341 oinstall

groupadd -g 54342 dba

groupadd -g 54343 oper

groupadd -g 54344 backupdba

groupadd -g 54345 dgdba

groupadd -g 54346 kmdba

groupadd -g 54347 asmdba

groupadd -g 54348 asmoper

groupadd -g 54349 asmadmin

groupadd -g 54350 racdba

usermod -g oinstall -G dba,oper,asmdba,asmadmin,asmoper,backupdba,dgdba,kmdba,racdba oracle

useradd -m -u 54342 -g oinstall -G dba,asmadmin,asmdba,asmoper -d /home/grid -s /bin/bash grid

echo "grid" | passwd --stdin grid

useradd -m -u 54341 -g oinstall -G dba,oper,backupdba,dgdba,kmdba,asmdba,asmadmin,racdba -d /home/oracle -s /bin/bash oracle

echo "oracle" | passwd --stdin oracle

6. Directory Stucture

mkdir -p /u01/app/oracle/19c/grid

mkdir -p /u01/app/oracle/oracle

mkdir -p /u01/app/oracle/19c/db

chmod -R 775 /u01

chown -R oracle:oinstall /u02/app/oracle

chown -R grid:oinstall /u02/app/oracle/19c/grid

7. Shared Disks for Oracleasm disks

1. Udev method (mostly Own by system team)

2. Oracleasm (multipath)

# rpm -qa | grep oracleasm

oracleasm-support-2.1.11-2.el7.x86_64

make Disk partion using fdisk comands (output like below):

[root@woravr02 ~]# fdisk -l | egrep 'Disk /dev/sd|dev'

Disk /dev/sdb: 209.7 GB, 209715200000 bytes, 409600000 sectors

/dev/sdb1 2048 409599999 204798976 83 Linux

Disk /dev/sdc: 157.3 GB, 157303177216 bytes, 307232768 sectors

/dev/sdc1 2048 307232767 153615360 83 Linux

Disk /dev/sdd: 104.9 GB, 104857600000 bytes, 204800000 sectors

/dev/sdd1 2048 204799999 102398976 83 Linux

Disk /dev/sde: 419.4 GB, 419430400000 bytes, 819200000 sectors

/dev/sde1 2048 819199999 409598976 83 Linux

[root@woravr02 ~]#

#oracleasm createdisk FRA /dev/sdb1

Writing disk header: done

Instantiating disk: done

#oracleasm createdisk ACFS /dev/sdc1

Writing disk header: done

Instantiating disk: done

#oracleasm createdisk OCR /dev/sdd1

Writing disk header: done

Instantiating disk: done

#oracleasm createdisk DATA /dev/sde1

Writing disk header: done

Instantiating disk: done

#/usr/sbin/oracleasm scandisks #--FROM other nodes

Reloading disk partitions: done

Cleaning any stale ASM disks...

Scanning system for ASM disks...

Instantiating disk "OCR"

Instantiating disk "DATA"

Instantiating disk "ACFS"

Instantiating disk "FRA"

#oracleasm listdisks

ACFS

DATA

FRA

OCR

8. unzip the Software

unzip LINUX.X64_193000_grid_home_2.zip -d /u01/app/oracle/19c/grid

unzip LINUX.X64_193000_db_home.zip -d /u01/app/oracle/19c/grid

9. User Equivalence

./sshUserSetup.sh -user grid -hosts "woravr01 woravr02 woravr03" -noPromptPassphrase -confirm -advanced

10. runcluvfy

./runcluvfy.sh stage -pre crsinst -n woravr01,woravr02,woravr03 -orainv oinstall -osdba asmdba -verbose

11. ./gridSetup.sh

More Details

12 . RunFixup if required .

13. orainstRoot

/u01/app/oraInventory/orainstRoot.sh ( each node )

14. root.sh

/u01/app/oracle/19c/grid/root.sh (each node take care exicution sequence)

15 . Check the Cluster status after Each root.sh script

./crsctl check cluster -all

**************************************************************

woravr02:

CRS-4537: Cluster Ready Services is online

CRS-4529: Cluster Synchronization Services is online

CRS-4533: Event Manager is online

**************************************************************

woravr02:

CRS-4537: Cluster Ready Services is online

CRS-4529: Cluster Synchronization Services is online

CRS-4533: Event Manager is online

**************************************************************

woravr03:

CRS-4537: Cluster Ready Services is online

CRS-4529: Cluster Synchronization Services is online

CRS-4533: Event Manager is online

**************************************************************

[root@woravr02 grid_1]# crsstat

Cluster woravrdc

-------------------------------------------------------------------------------------------

cvu | cvu | Online | - | - |

vip | scan1 | - | - | Online |

vip | scan2 | Online | - | - |

vip | scan3 | - | Online | - |

vip | woravr01 | Online | - | - |

vip | woravr02 | - | Online | - |

vip | woravr03 | - | - | Online |

-------------------------------------------------------------------------------------------

x : Resource is disabled

: Has been restarted less than 24 hours ago

----------------------------------------------------------------------------------------------------------

LISTENER_SCAN1 | TCP:1521 | - | - | Online | SCAN |

LISTENER_SCAN2 | TCP:1521 | Online | - | - | SCAN |

LISTENER_SCAN3 | TCP:1521 | - | Online | - | SCAN |

-----------------------------------------------------------------------------------------------------------

: Has been restarted less than 24 hours ago

16. Invoke ASMCA to create diskgroup

/u01/app/oracle/19c/grid/bin/./asmca

17. Intsall RDBMS Software

./runInstaller.sh

18. root.sh

/u01/app/oracle/19c/db/root.sh

19. Run DBCA

Create the database

20. Create service for database : optional

srvctl modify service -db woravr -service woravrsvc -i woravr01,woravr02 -t woravr03 -tafpolicy BASIC -failback YES

21. crsstat

Cluster woravrdc

-------------------------------------------------------------------------------------------

cvu | cvu | Online | - | - |

vip | scan1 | - | - | Online |

vip | scan2 | Online | - | - |

vip | scan3 | - | Online | - |

vip | woravr01 | Online | - | - |

vip | woravr02 | - | Online | - |

vip | woravr03 | - | - | Online |

-------------------------------------------------------------------------------------------

x : Resource is disabled

: Has been restarted less than 24 hours ago

----------------------------------------------------------------------------------------------------------

LISTENER_SCAN1 | TCP:1521 | - | - | Online | SCAN |

LISTENER_SCAN2 | TCP:1521 | Online | - | - | SCAN |

LISTENER_SCAN3 | TCP:1521 | - | Online | - | SCAN |

-----------------------------------------------------------------------------------------------------------

: Has been restarted less than 24 hours ago

-----------------------------------------------------------------------------------------

-----------------------------------------------------------------------------------------

: Has been restarted less than 24 hours ago

--------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------

ORACLE_HOME references listed in the Version column

1 : /u01/app/oracle/19c/db_1 oracle oinstall

: Has been restarted less than 24 hours ago

22. Add Entry on /etc/oratab

vi /etc/oratab

23. Create Env Profiles .

[oracle@woravr02 ~]$ cat .bash_profile

# .bash_profile

# Get the aliases and functions

if [ -f ~/.bashrc ]; then

. ~/.bashrc

# User specific environment and startup programs

PATH=$PATH:$HOME/.local/bin:$HOME/bin

export PATH

export TMP=/tmp

export TMPDIR=$TMP

export ORACLE_BASE=/u01/app/oracle_base

export GRID_HOME=/u01/app/oracle/19c/grid_1

export DB_HOME=/u01/app/oracle/19c/db_1

export ORACLE_HOME=$DB_HOME

export ORACLE_SID=woravr2

export ORACLE_TERM=xterm

export BASE_PATH=/usr/sbin:$PATH

export PATH=$ORACLE_HOME/bin:$GRID_HOME/bin:$BASE_PATH

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib

export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

alias grid='. /home/oracle/grid.env'

alias db='. /home/oracle/db.env'

alias l='ls -altrsh'

#-------------------------------------------------------

[oracle@woravr02 ~]$ cat db.env

export ORACLE_SID=woravr2

export ORACLE_HOME=$DB_HOME

export PATH=$ORACLE_HOME/bin:$BASE_PATH

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib

export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

[oracle@woravr02 ~]$ cat grid.env

export ORACLE_SID=+ASM2

export ORACLE_HOME=$GRID_HOME

export PATH=$ORACLE_HOME/bin:$BASE_PATH

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib

export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

[oracle@woravr02 ~]$

24 . Patch the database

below command from root user :

1. Export Grid Env

$ORACLE_HOME/OPatch/./opatchauto apply /tmp/31326369/31305339 -oh $ORACLE_HOME

2. Export the RDBMS Env

$ORACLE_HOME/OPatch/./opatchauto apply /tmp/31326369/31305339 -oh $ORACLE_HOME

repeat same steps for all nodes

open all pluggable databases and exicute the below command after patch

$ORACLE_HOME/OPatch/./datapatch -verbose

[oracle@woravr02 ~]$ db

[oracle@woravr02 ~]$ $ORACLE_HOME/OPatch/./opatch lsinventory | grep applied

Patch 31305087 : applied on Sun Aug 09 20:07:12 IST 2020

Patch 31281355 : applied on Sun Aug 09 20:02:59 IST 2020

[oracle@woravr02 ~]$ grid

[oracle@woravr02 ~]$ $ORACLE_HOME/OPatch/./opatch lsinventory | grep applied

Patch 31335188 : applied on Sun Aug 09 19:47:39 IST 2020

Patch 31305087 : applied on Sun Aug 09 19:46:48 IST 2020

Patch 31304218 : applied on Sun Aug 09 19:45:41 IST 2020

Patch 31281355 : applied on Sun Aug 09 19:42:14 IST 2020

[oracle@woravr02 ~]$