Centos 7.7下Oracle 11g RAC
Centos 7.7下Oracle 11g RAC部署
- 操作系统环境
1.1linux操作系统配置
两台主机IP规划信息如下：
|
rac1 |
rac2 |
|
Ip：192.168.198.180 |
Ip：192.168.198.181 |
|
priv:10.10.10.81 |
priv: 10.10.10.82 |
|
vip: 192.168.198.182 |
vip: 192.168.198.183 |
|
scan-ip:192.168.198.184 |
|
|
Linux系统版本均为7.7 |
|
|
Oracle版本为11.2.0.4.0 |
|
1.2hosts文件配置
[[email protected] ~]# vi /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
#public
192.168.198.180 rac1
192.168.198.181 rac2
#vip
192.168.198.182 rac1-vip
192.168.198.183 rac2-vip
#priv
10.10.10.81 rac1-priv
10.10.10.82 rac2-priv
#scan
192.168.198.184 rac-cluster-scan
两个节点的hosts文件内容要相同
1.3配置内核参数
[[email protected] ~]# vi /etc/sysctl.conf
# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 8389934592 # kernel.shmmax要以自己实际内存大小而定
kernel.shmall = 268435456
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586
1.4关闭防火墙
[[email protected] ~]# systemctl stop firewalld
[[email protected] ~]# systemctl disable firewalld
[[email protected] ~]# systemctl status firewalld
1.5 关闭SELINUX
[[email protected] ~]# vi /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
1.6安全限制调整
[[email protected] ~]# vi /etc/security/limits.conf
# /etc/security/limits.conf
#
#This file sets the resource limits for the users logged in via PAM.
#It does not affect resource limits of the system services.
#
#Also note that configuration files in /etc/security/limits.d directory,
#which are read in alphabetical order, override the settings in this
#file in case the domain is the same or more specific.
#That means for example that setting a limit for wildcard domain here
#can be overriden with a wildcard setting in a config file in the
#subdirectory, but a user specific setting here can be overriden only
#with a user specific setting in the subdirectory.
#
#Each line describes a limit for a user in the form:
#
#<domain> <type> <item> <value>
#
#Where:
#<domain> can be:
# - a user name
# - a group name, with @group syntax
# - the wildcard *, for default entry
# - the wildcard %, can be also used with %group syntax,
# for maxlogin limit
#
#<type> can have the two values:
# - "soft" for enforcing the soft limits
# - "hard" for enforcing hard limits
#
#<item> can be one of the following:
# - core - limits the core file size (KB)
# - data - max data size (KB)
# - fsize - maximum filesize (KB)
# - memlock - max locked-in-memory address space (KB)
# - nofile - max number of open file descriptors
# - rss - max resident set size (KB)
# - stack - max stack size (KB)
# - cpu - max CPU time (MIN)
# - nproc - max number of processes
# - as - address space limit (KB)
# - maxlogins - max number of logins for this user
# - maxsyslogins - max number of logins on the system
# - priority - the priority to run user process with
# - locks - max number of file locks the user can hold
# - sigpending - max number of pending signals
# - msgqueue - max memory used by POSIX message queues (bytes)
# - nice - max nice priority allowed to raise to values: [-20, 19]
# - rtprio - max realtime priority
#
#<domain> <type> <item> <value>
#
#* soft core 0
#* hard rss 10000
#@student hard nproc 20
#@faculty soft nproc 20
#@faculty hard nproc 50
#ftp hard nproc 0
#@student - maxlogins 4
# End of file
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
oracle soft stack 10240
grid soft nproc 2047
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536
grid soft stack 10240
1.7安装rpm包
[[email protected] ~]# yum -y install binutils compat-libstdc++-33 gcc gcc-c++ glibc glibc-common glibc-devel ksh libaio libaio-devel libgcc libstdc++ libstdc++-devel make sysstat openssh-clients compat-libcap1 xorg-x11-utils xorg-x11-xauth elfutils unixODBC unixODBC-devel libXp elfutils-libelf elfutils-libelf-devel smartmontools glibc-headers
2.用户配置
2.1添加Oracle/grid用户
[[email protected] ~]# groupadd -g 54321 oinstall
[[email protected] ~]# groupadd -g 54322 dba
[[email protected] ~]# groupadd -g 54323 oper
[[email protected] ~]# groupadd -g 54324 backupdba
[[email protected] ~]# groupadd -g 54325 dgdba
[[email protected] ~]# groupadd -g 54326 kmdba
[[email protected] ~]# groupadd -g 54327 asmdba
[[email protected] ~]# groupadd -g 54328 asmoper
[[email protected] ~]#groupadd -g 54329 asmadmin
[[email protected] ~]# groupadd -g 54330 racdba
[[email protected] ~]# useradd -u 54321 -g oinstall -G dba,asmdba,backupdba,dgdba,kmdba,racdba,oper oracle
[[email protected] ~]# useradd -u 54322 -g oinstall -G asmadmin,asmdba,asmoper,dba grid
[[email protected] ~]# passwd oracle
[[email protected] ~]# passwd grid
2.2创建目录
[[email protected] ~]# mkdir -p /oracle/app/11.2.0/grid
[[email protected] ~]# mkdir -p /oracle/app/grid
[[email protected] ~]# mkdir -p /oracle/app
[[email protected] ~]# mkdir -p /oracle/app/oracle/product/11.2.0/dbhome_1
[[email protected] ~]# chown -R oracle:oinstall /oracle
[[email protected] ~]# chown -R grid:oinstall /oracle/app/11.2.0/grid
[[email protected] ~]# chown -R grid:oinstall /oracle/app/grid
[[email protected] ~]# chown -R oracle:oinstall /oracle/app/oracle
[[email protected] ~]# chmod 771 /oracle/
[[email protected] ~]# chmod 771 /oracle/app
2.3设置Oracle用户环境变量
[[email protected] ~]# su – oracle
[[email protected] ~]$ vi .bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/.local/bin:$HOME/bin
export PATH
export ORACLE_BASE=/oracle/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/dbhome_1
export ORACLE_SID=zzw1 #节点俩边实例名分别为zzw1和zzw2
export LANG=en_US.UTF-8
export NLS_LANG=american_america.ZHS16GBK
export NLS_DATE_FORMAT="yyyy-mm-dd hh24:mi:ss"
export PATH=.:${PATH}:$HOME/bin:$ORACLE_HOME/bin:$ORACLE_HOME/OPatch
export PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin
export PATH=${PATH}:$ORACLE_BASE/common/oracle/bin:/home/oracle/run
export ORACLE_TERM=xterm
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib
export CLASSPATH=$ORACLE_HOME/JRE
export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib
export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib
export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib
export THREADS_FLAG=native
export TEMP=/tmp
export TMPDIR=/tmp
export GI_HOME=/oracle/app/11.2.0/grid
export PATH=${PATH}:$GI_HOME/bin
export ORA_NLS10=$GI_HOME/nls/data
umask 022
export TMOUT=0
2.4设置grid用户环境变量
[[email protected] ~]$ vi .bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/.local/bin:$HOME/bin
export PATH
export ORACLE_BASE=/oracle/app/grid
export ORACLE_HOME=/oracle/app/11.2.0/grid
export ORACLE_SID=+ASM1 #两边分别为+ASM1和+ASM2
export NLS_LANG=american_america.ZHS16GBK
export NLS_DATE_FORMAT="yyyy-mm-dd hh24:mi:ss"
export PATH=.:${PATH}:$HOME/bin:$ORACLE_HOME/bin:$ORACLE_HOME/OPatch
export PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin
export PATH=${PATH}:$ORACLE_BASE/common/oracle/bin
export ORACLE_TERM=xterm
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib
export CLASSPATH=$ORACLE_HOME/JRE
export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib
export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib
export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib
export THREADS_FLAG=native
export TEMP=/tmp
export TMPDIR=/tmp
umask 022
export TMOUT=0
~
3. UDEV绑盘
3.1创建4块新的虚拟磁盘
先关闭两边虚机
以此分别创建3个1G，1个10G的磁盘
以自己建的磁盘顺序依次绑定节点
第二台虚机添加硬盘时选择使用现有磁盘
找到刚刚创建的4块磁盘并绑定相同节点
3.2修改虚机参数
找到虚机.vmx参数文件
编辑该文件在文件末尾加上
diskLib.dataCacheMaxSize = "0"
diskLib.dataCacheMaxReadAheadSize = "0"
diskLib.dataCacheMinReadAheadSize = "0"
diskLib.dataCachePageSize = "4096"
diskLib.maxUnsyncedWrites = "0"
disk.locking = "FALSE"
disk.enableUUID = "TRUE"
两边都要添加之后保存文件再开机
3.3绑定磁盘组
[[email protected] ~]# fdisk -l
Disk /dev/sda: 42.9 GB, 42949672960 bytes, 83886080 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000b3588
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 2099199 1048576 83 Linux
/dev/sda2 2099200 83886079 40893440 8e Linux LVM
Disk /dev/sdb: 1073 MB, 1073741824 bytes, 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/sdc: 1073 MB, 1073741824 bytes, 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/sdd: 1073 MB, 1073741824 bytes, 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/sde: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/centos-root: 33.4 GB, 33386659840 bytes, 65208320 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/centos-swap: 8485 MB, 8485076992 bytes, 16572416 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
先查看两边磁盘是否挂接成功
查看自己两边磁盘UUID是否相同
[[email protected] ~]# /usr/lib/udev/scsi_id -g -u -d /dev/sdb
36000c299fee8a318f3d12deab7b4991a
[[email protected] ~]# /usr/lib/udev/scsi_id -g -u -d /dev/sdc
36000c2933c41c168f10bd3fee9abe77c
[[email protected] ~]# /usr/lib/udev/scsi_id -g -u -d /dev/sdd
36000c29d9d88b533e4f69484ae8a58e9
[[email protected] ~]# /usr/lib/udev/scsi_id -g -u -d /dev/sde
36000c29e3a08e62f7c157aa0012e84d1
两边添加绑定文件
[[email protected] ~]# vi /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="sdb", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/sdb", RESULT=="36000c299fee8a318f3d12deab7b4991a", SYMLINK+="asm-crs1", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sdc", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/sdc", RESULT=="36000c2933c41c168f10bd3fee9abe77c", SYMLINK+="asm-crs2", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sdd", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/sdd", RESULT=="36000c29d9d88b533e4f69484ae8a58e9", SYMLINK+="asm-crs3", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sde", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/sde", RESULT=="36000c29e3a08e62f7c157aa0012e84d1", SYMLINK+="asm-data", OWNER="grid", GROUP="asmadmin", MODE="0660"
两边编辑绑定文件后，依次执行以下命令
[[email protected] ~]# systemctl status systemd-udevd.service
[[email protected] ~]# systemctl enable systemd-udevd.service
[[email protected] ~]# /usr/sbin/udevadm control --reload-rules
[[email protected] ~]# /usr/sbin/udevadm trigger --type=devices
[[email protected] ~]# ls -l /dev/sd*
brw-rw---- 1 root disk 8, 0 Nov 14 11:23 /dev/sda
brw-rw---- 1 root disk 8, 1 Nov 14 11:23 /dev/sda1
brw-rw---- 1 root disk 8, 2 Nov 14 11:23 /dev/sda2
brw-rw---- 1 grid asmadmin 8, 16 Nov 14 11:52 /dev/sdb
brw-rw---- 1 grid asmadmin 8, 32 Nov 14 11:53 /dev/sdc
brw-rw---- 1 grid asmadmin 8, 49 Nov 14 11:56 /dev/sdd
brw-rw---- 1 grid asmadmin 8, 65 Nov 14 11:56 /dev/sde
[[email protected] ~]# ls -l /dev/asm*
lrwxrwxrwx 1 root root 4 Nov 14 11:57 /dev/asm-crs1 -> sdb
lrwxrwxrwx 1 root root 4 Nov 14 11:56 /dev/asm-crs2 -> sdc
lrwxrwxrwx 1 root root 4 Nov 14 11:56 /dev/asm-crs3 -> sdd
lrwxrwxrwx 1 root root 4 Nov 14 11:56 /dev/asm-data -> sde
4安装集群软件
4.1先上传grid安装包到grid家目录下
[[email protected] ~]$ ll
-rwxrwxr-x 1 grid oinstall 1205251894 2月 27 2015 p13390677_112040_Linux-x86-64_3of7.zip
上传过来后要修改grid安装包权限
4.2解压安装包
[[email protected] ~]$ unzip p13390677_112040_Linux-x86-64_3of7.zip
[[email protected] ~]$ ll
drwxr-xr-x 7 grid oinstall 156 8月 27 2013 grid
drwxr-xr-x 3 grid oinstall 18 7月 5 21:43 oradiag_grid
-rwxrwxr-x 1 grid oinstall 1205251894 2月 27 2015 p13390677_112040_Linux-x86-64_3of7.zip
[[email protected] ~]$ cd grid/
[[email protected] grid]$ ./runInstaller
两边执行自动修复脚本
依次再在两个节点用root用户执行两个脚本，先执行脚本1按节点一二顺序，再执行脚本2按节点一二顺序
在执行第二个脚本时会出现版本bug
要在新开窗口执行
安装完成后显示的报错可以忽略
4.3检查grid集群是否安装成功
[[email protected] ~]$ crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
[[email protected] ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.CRS.dg
ONLINE ONLINE rac1
ONLINE ONLINE rac2
ora.DATA.dg
ONLINE ONLINE rac1
ONLINE ONLINE rac2
ora.LISTENER.lsnr
ONLINE ONLINE rac1
ONLINE ONLINE rac2
ora.asm
ONLINE ONLINE rac1 Started
ONLINE ONLINE rac2 Started
ora.gsd
OFFLINE OFFLINE rac1
OFFLINE OFFLINE rac2
ora.net1.network
ONLINE ONLINE rac1
ONLINE ONLINE rac2
ora.ons
ONLINE ONLINE rac1
ONLINE ONLINE rac2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE rac2
ora.cvu
1 ONLINE ONLINE rac1
ora.oc4j
1 ONLINE ONLINE rac2
ora.rac1.vip
1 ONLINE ONLINE rac1
ora.rac2.vip
1 ONLINE ONLINE rac2
ora.scan1.vip
1 ONLINE ONLINE rac2
ora.zzw.db
1 ONLINE ONLINE rac1 Open
2 ONLINE ONLINE rac2 Open
5.安装数据库软件
上传数据库压缩包到oracle家目录下并赋权
[[email protected] ~]$ ll
-rwxrwxr-x 1 oracle oinstall 1212620273 Nov 7 2019 p13390677_112040_Linux-x86-64_1of7.zip
-rwxrwxr-x 1 oracle oinstall 113112960 Nov 7 2019 p13390677_112040_Linux-x86-64_2of7.zip
开始安装
[[email protected] ~]$ cd database/
[[email protected] database]$ ./runInstaller
执行以上脚本
使用grid用户执行asmca命令
[[email protected] ~]$ asmca
[[email protected] ~]$ dbca