10.2 RAC安装之clusterware

作者：八神 | 分类： 大话技术 | 标签： | 日期：2008-04-10

安装了下10.2的rac,在安装clusterware的过程中出现了一系列问题,零散的记录了下

 系统环境：
4台刀片主机(linux AS 4.4) + 牛叉存储
RAC + ASM

网络环境配置(每台主机双网卡，只需要配置public,private两个IP，VIP由vipca去虚拟，不需要配置)：
#RAC network configure
10.0.38.160       hack1
10.0.38.170       hack1-vip
192.168.0.1       hack1-priv
          
10.0.38.162       hack2
10.0.38.172       hack2-vip
192.168.0.2       hack2-priv
          
10.0.38.164       hack3
10.0.38.174       hack3-vip
192.168.0.3       hack3-priv
          
10.0.38.166       hack4
10.0.38.176       hack4-vip
192.168.0.4       hack4-priv
磁盘用途：
votedisk -> /dev/raw/raw1 -> /dev/sdb1
ocrdisk -> /dev/raw/raw2 -> /dev/sdc1
disk1 -> /dev/raw/raw3 -> /dev/sdd1
disk2 -> /dev/raw/raw4 -> /dev/sde1

/etc/sysctl.conf添加如下内容：
kernel.shmall = 2097152
kernel.shmmax = 8589934592

kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 65536
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default = 262144
net.core.rmem_max = 262144
net.core.wmem_default = 262144
net.core.wmem_max = 262144

系统包要求（更高）：
binutils-2.15.92.0.2-10.EL4
binutils-2.15.92.0.2-13.0.0.0.2.x86_64
compat-db-4.1.25-9
control-center-2.8.0-12
gcc-3.4.3-9.EL4
gcc-c++-3.4.3-9.EL4
glibc-2.3.4-2
glibc-common-2.3.4-2
gnome-libs-1.4.1.2.90-44.1
libstdc++-3.4.3-9.EL4
libstdc++-devel-3.4.3-9.EL4
make-3.80-5
pdksh-5.2.14-30
–部署目录
mkdir -p /opt/oracle/admin
mkdir -p /opt/oracle/products/10.2.0
mkdir -p /opt/oracle/products/crs
mkdir -p /opt/oracle/oraInventory
mkdir -p /opt/oracle/oradata/alidw
mkdir -p /home/oracle
–编辑/etc/oraInst.loc,指定oraInventory目录和group
inventory_loc=/opt/oracle/oraInventory
inst_group=oinstall

–创建oracle帐户
/usr/sbin/groupadd oinstall
/usr/sbin/groupadd dba
/usr/sbin/groupadd oper
/usr/sbin/useradd -u 200 -g oinstall -G dba,oper oracle

chown -Rf oracle.dba /home/oracle
chown -Rf oracle.dba /opt/oracle

–配置ssh
su - oracle
mkdir ~/.ssh
chmod 700 ~/.ssh
/usr/bin/ssh-keygen -t rsa
/usr/bin/ssh-keygen -t dsa
touch ~/.ssh/authorized_keys
cd ~/.ssh

ssh hack1 cat /home/oracle/.ssh/id_rsa.pub >> authorized_keys
ssh hack2 cat /home/oracle/.ssh/id_rsa.pub >> authorized_keys
ssh hack3 cat /home/oracle/.ssh/id_rsa.pub >> authorized_keys
ssh hack4 cat /home/oracle/.ssh/id_rsa.pub >> authorized_keys

ssh hack1 cat /home/oracle/.ssh/id_dsa.pub >> authorized_keys
ssh hack2 cat /home/oracle/.ssh/id_dsa.pub >> authorized_keys
ssh hack3 cat /home/oracle/.ssh/id_dsa.pub >> authorized_keys
ssh hack4 cat /home/oracle/.ssh/id_dsa.pub >> authorized_keys

scp authorized_keys oracle@hack2:/home/oracle/.ssh/
scp authorized_keys oracle@hack3:/home/oracle/.ssh/
scp authorized_keys oracle@hack4:/home/oracle/.ssh/

–配置oracle用户环境
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/bin
ORACLE_BASE=/opt/oracle
ORACLE_SID=asmrac4
ORACLE_HOME=$ORACLE_BASE/products/10.2.0
CRS_HOME=$ORACLE_BASE/products/crs
ORA_NLS33=$ORACLE_HOME/ocommon/nls/admin/data
PATH=$PATH:$ORACLE_HOME/bin:$ORACLE_HOME/oracm/bin
LD_LIBRARY_PATH=${LD_LIBRARY_PATH:+$LD_LIBRARY_PATH:}$ORACLE_HOME/lib:$ORACLE_HOME/ctx/lib:$ORACLE_HOME/oracm/lib
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib:$ORACLE_HOME/network/jlib

export ORACLE_BASE ORACLE_HOME ORA_NLS33 ORACLE_SID PATH LD_LIBRARY_PATH CLASSPATH TNS_ADMIN CRS_HOME

# ORACLE_TERM=xterm; export ORACLE_TERM
NLS_LANG=AMERICAN_AMERICA.US7ASCII; export NLS_LANG
alias sql=”sqlplus /nolog”
export SQLPATH=~/admin/sql:/$ORACLE_HOME/sqlplus/admin

export PATH
unset USERNAME

–部署供享磁盘，建立link
fdisk /dev/sdb /dev/sdc /dev/sdd /dev/sde

raw /dev/raw/raw1 /dev/sdb1
raw /dev/raw/raw2 /dev/sdc1
raw /dev/raw/raw3 /dev/sdd1
raw /dev/raw/raw4 /dev/sde1

chown -Rf oracle.dba /dev/sdb1
chown -Rf oracle.dba /dev/sdc1
chown -Rf oracle.dba /dev/sdd1
chown -Rf oracle.dba /dev/sde1
chown -Rf oracle.dba /dev/raw/raw1
chown -Rf oracle.dba /dev/raw/raw2
chown -Rf oracle.dba /dev/raw/raw3
chown -Rf oracle.dba /dev/raw/raw4

cd /opt/oracle/oradata/asmrac
ln -s /dev/raw/raw1 votedisk
ln -s /dev/raw/raw2 ocrdisk
ln -s /dev/raw/raw3 disk1
ln -s /dev/raw/raw4 disk2
–安装ASM工具和lib库
rpm -ivh oracleasm-support-2.0.3-1.x86_64.rpm
rpm -ivh oracleasm-2.6.9-42.ELsmp-2.0.3-1.x86_64.rpm
rpm -ivh oracleasmlib-2.0.2-1.x86_64.rpm

[root@hack3 oracle]# /etc/init.d/oracleasm configure
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets (’[]‘).  Hitting <ENTER> without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface []:
Default group to own the driver interface []:
Start Oracle ASM library driver on boot (y/n) [n]: y
Fix permissions of Oracle ASM disks on boot (y/n) [y]:
Writing Oracle ASM library driver configuration:           [  OK  ]
Creating /dev/oracleasm mount point:                       [  OK  ]
Loading module “oracleasm”:                                [  OK  ]
Mounting ASMlib driver filesystem:                         [  OK  ]
Scanning system for ASM disks:                             [  OK  ]
作好上面的准备工作后，就开始安装clusterware了，把软件上传到一台服务器解压，
他基本是在一台安装，编译好，再scp到其他接点的,安装完后，需要运行root.sh，这个步骤最容易出错
下面列了下运行情况和遇到的错误解决

[root@hack3 install]# /opt/oracle/products/crs/root.sh
WARNING: directory ‘/opt/oracle/products’ is not owned by root
WARNING: directory ‘/opt/oracle’ is not owned by root
Checking to see if Oracle CRS stack is already configured
/etc/oracle does not exist. Creating it now.

Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
WARNING: directory ‘/opt/oracle/products’ is not owned by root
WARNING: directory ‘/opt/oracle’ is not owned by root
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: hack3 hack3-priv hack3
node 2: hack4 hack4-priv hack4
Creating OCR keys for user ‘root’, privgrp ‘root’..
Operation successful.
Now formatting voting device: /opt/oracle/oradata/asmrac/votedisk
   Format of 1 voting devices complete.
Startup will be queued to init within 90 seconds.
 Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
  CSS is active on these nodes.
        hack3
CSS is inactive on these nodes.
        hack4
Local node checking complete.
Run root.sh on remaining nodes to start CRS daemons.

[root@hack4 lib32]#        /opt/oracle/products/crs/root.sh
WARNING: directory ‘/opt/oracle/products’ is not owned by root
WARNING: directory ‘/opt/oracle’ is not owned by root
Checking to see if Oracle CRS stack is already configured
/etc/oracle does not exist. Creating it now.

Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
WARNING: directory ‘/opt/oracle/products’ is not owned by root
WARNING: directory ‘/opt/oracle’ is not owned by root
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: hack3 hack3-priv hack3
node 2: hack4 hack4-priv hack4
clscfg: Arguments check out successfully.

NO KEYS WERE WRITTEN. Supply -force parameter to override.
-force is destructive and will destroy any previous cluster
configuration.
Oracle Cluster Registry for cluster has already been initialized
Startup will be queued to init within 90 seconds.
 Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
 CSS is active on these nodes.
        hack3
        hack4
CSS is active on all nodes.
Waiting for the Oracle CRSD and EVMD to start
Oracle CRS stack installed and running under init(1M)
Running vipca(silent) for configuring nodeapps
The given interface(s), “eth0″ is not public. Public interfaces should be used to configure virtual IPs.

–root.sh错误解决
1.直接出现Failed to upgrade Oracle Cluster Registry configuration，观察$CRS_HOME/logs/{node}/alert*日志，
发现有not set 0等字样
解决：不要直接使用共享lun，如/dev/sdb,一定要fdisk分区

2.rootconfig: line 815 Segmentation fault
解决：是gcc和g++有问题
1. cd /usr/bin 
2. mv gcc gcc.script
3. mv g++ g++.script
4. ln -s gcc32 gcc
5. ln -s g++32 g++

3.运行完root.sh后，不能启动vipca，总是出错
PRKH-1010 : Unable to communicate with CRS services.
[PRKH-1000 : Unable to load the SRVM HAS shared library
[PRKN-1008 : Unable to load the shared library “srvmhas10″
解决：参考metalink Note:342750.1
在64bit linux上缺少32位的glibc-devel包，导致有些库文件链接出错，从$CRS_HOME/install 下的make log里面可以看到
错误信息
我这边下了
glibc-common-2.3.4-2.39.i386.rpm
glibc-2.3.4-2.39.i386.rpm
glibc-headers-2.3.4-2.39.i386.rpm
glibc-devel-2.3.4-2.39.i386.rpm
安装的时候注意顺序
4.如何清理安装失败的CRS
$CRS_HOME/install/rootdeinstall.sh
$CRS_HOME/install/rootdelete.sh
ps -ef | grep crs 
kill <crs pid>
ps -ef | grep evm
kill <evm pid>
ps -ef | grep css 
kill <css pid>

dd if=/dev/zero of=/opt/oracle/oradata/asmrac/votedisk bs=8192 count=2560
dd if=/dev/zero of=/opt/oracle/oradata/asmrac/ocrdisk bs=8192 count=12800

干掉CRS_HOME目录下的所有东西
rm -rf /var/tmp/.oracle
rm -rf /tmp/.oracle
rm -rf /etc/oracle
rm -rf /opt/oracle/oraInventory/*
rm -rf /opt/oracle/products/crs/*
到此clusterware安装完毕，可以再直接安装oracle数据库软件，这个比较顺利，就不多说了

有2人发表了评论  ↓发表评论↓

  • (⊙ˍ⊙)ｂ（￣▽￣）ｄ＞﹏＜（─.─||）(^_-)

    江志伟 @ December 8, 2009 |

  • (⊙ˍ⊙)看不懂了………..

    保健品 @ August 12, 2010 |

表情：<(￣︶￣)> | (⊙ˍ⊙) | ＞﹏＜ | ｂ（￣▽￣）ｄ | （─.─||） | (^_-)

[ Ctrl+Enter提交 ]

DBA