11gR2 GI总结(一)

11gR2 GI变化还是蛮大的,出了问题找log,感觉都比较乱,有必要总结一下,对GI的software stack有一个理解。
一.11g clusterware resource
1.Networks
2.VIP addresses
3.Local listeners
4.SCAN listeners
5.ASM instances
6.Databases
7.Database instances
8.Services
9.User-defined resources
二.OCR包含资源
1.ASM disk groups, volumes, file systems, and instances
2.RAC databases and instances
3.SCAN listeners and local listeners
4.SCAN VIPs and local VIPs
5.Nodes and node applications
6.User-defined resources
7.Its own backups
三.Grid Infrastructure process overview：

1.Oracle agent include the following(不要求Root权限):
EVMD and EVMLOGGER
the gipc daemon
the gpnp daemon
The mDNS daemon
2.Oracle root Agent include(Root privige):
CRS daemon
CTSS daemon
Disk Monitoring daemon
ACFS drivers
3.CRS started,将创建另外的oracle agent and oracle root agent,这里如果GI是在grid账户下，grid agent include：
Starting and monitoring the local ASM instance
ONS and eONS daemons
The SCAN listener, where applicable
The Node listener
4,oracle root agent最终include：
GNS, if configured
GNS VIP if GNS enabled
ACFS Registry
Network
SCAN VIP, if applicable
Node VIP

一.11g rac oracle clusterware key service essential:
1,group service,manage cluster service
2,node monitor or the 'heartbeat' monitor the status of nodes in RAC
3,locking service performed by the LMS database process to manage concurrency activities for RAC
4,HA resource management for failover or load-balance

Oracle Clusterware consists of:
1,oracle cluster registry(包含db配置信息和crs信息，比如node list,cluster database instance mapping information,crs resource profiles.
相关命令：ocrcheck.

Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       3028
         Available space (kbytes) :     259092
         ID                       : 1857481567
         Device/File Name         :       +CRS
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Devi
crsctl query css votedisk.
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   a4a95bc3c2394facbf173138ffe76e4f (ORCL:CRSVOTEVOL1) [CRS]
Located 1 voting disk(s).

ce/File not configured Device/File not configured Device/File not configured Cluster registry integrity check succeeded

2,voting disk(所有节点读写心跳信息的重要机制,used to shutdown,fence,or reboot either nodes when network communication is lost.in order to prevet thesplit-brain,protect the db infomation.check configuration information:
crsctl query css votedisk.
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   a4a95bc3c2394facbf173138ffe76e4f (ORCL:CRSVOTEVOL1) [CRS]
Located 1 voting disk(s).
3,control files(scls_src) for clusterware(these control files enable and disable processes,such as enable or disable Oprocd and Oslsvmon process,

stop key daemons for crsd.bin and ocssd.bin,prevent the cluster startup when the reboots is occur.

For linux OS is located:

[grid@node1 scls_scr]$ pwd

/etc/oracle/scls_scr

[grid@node1 scls_scr]$ ls

node1
二.11G Cluster backgroud process
root      4757     1  0 13:30 ?        00:00:20 /u01/app/11.2.0.3/grid/bin/ohasd.bin reboot
grid      5081     1  0 13:31 ?        00:00:21 /u01/app/11.2.0.3/grid/bin/oraagent.bin
grid      5093     1  0 13:31 ?        00:00:00 /u01/app/11.2.0.3/grid/bin/mdnsd.bin
grid      5132     1  0 13:31 ?        00:00:04 /u01/app/11.2.0.3/grid/bin/gpnpd.bin
root      5142     1  0 13:31 ?        00:00:48 /u01/app/11.2.0.3/grid/bin/orarootagent.bin
grid      5145     1  0 13:31 ?        00:00:22 /u01/app/11.2.0.3/grid/bin/gipcd.bin
root      5158     1  0 13:31 ?        00:00:28 /u01/app/11.2.0.3/grid/bin/osysmond.bin
root      5166     1  0 13:31 ?        00:00:06 /u01/app/11.2.0.3/grid/bin/cssdmonitor
root      5181     1  0 13:31 ?        00:00:06 /u01/app/11.2.0.3/grid/bin/cssdagent
grid      5193     1  0 13:31 ?        00:00:52 /u01/app/11.2.0.3/grid/bin/ocssd.bin 
root      5302     1  0 13:31 ?        00:00:07 /u01/app/11.2.0.3/grid/bin/octssd.bin reboot
grid      5327     1  0 13:31 ?        00:00:07 /u01/app/11.2.0.3/grid/bin/evmd.bin
root      5496     1  0 13:32 ?        00:00:33 /u01/app/11.2.0.3/grid/bin/ologgerd -M -d /u01/app/11.2.0.3/grid/crf/db/node1
root      5650     1  0 13:36 ?        00:00:18 /u01/app/11.2.0.3/grid/bin/crsd.bin reboot
grid      5790  5327  0 13:37 ?        00:00:00 /u01/app/11.2.0.3/grid/bin/evmlogger.bin -o /u01/app/11.2.0.3/grid/evm/log/evmlogger.info -l /u01/app/11.2.0.3/grid/evm/log/evmlogger.log
grid      5831     1  0 13:37 ?        00:00:16 /u01/app/11.2.0.3/grid/bin/oraagent.bin
root      5834     1  0 13:37 ?        00:00:41 /u01/app/11.2.0.3/grid/bin/orarootagent.bin
grid      5839     1  0 13:37 ?        00:00:02 /u01/app/11.2.0.3/grid/bin/scriptagent.bin
grid      6130     1  0 13:38 ?        00:00:00 /u01/app/11.2.0.3/grid/bin/tnslsnr LISTENER_SCAN1 -inherit
grid      6134     1  0 13:38 ?        00:00:00 /u01/app/11.2.0.3/grid/bin/tnslsnr LISTENER2 -inherit
grid      6138     1  0 13:38 ?        00:00:00 /u01/app/11.2.0.3/grid/bin/tnslsnr LISTENER -inherit
oracle    6353     1  0 13:41 ?        00:00:23 /u01/app/11.2.0.3/grid/bin/oraagent.bin
grid     11416  5839  0 15:11 ?        00:00:00 /bin/sh /u01/app/11.2.0.3/grid/bin/oc4jctl check
grid     11428 11416  0 15:12 ?        00:00:00 /u01/app/11.2.0.3/grid/perl/bin/perl /u01/app/11.2.0.3/grid/bin/oc4jctl.pl check 8888
root     11453  5834  0 15:12 ?        00:00:00 /u01/app/11.2.0.3/grid/bin/orarootagent.bin
OHAS包含一下守护进程和服务 

1.The Grid Plug And Play (GPnP) daemon:

maintains the Grid Plug and Play profile and coordinates distribution of updates across all nodes in the cluster.

The Grid Plug and Playprofile essentially contains sufficient information to start Clusterware。

之前的版本，以上信息是存储在OCR中,11gR2 OCR可以存储在ASM中,所以以上信息需要在启动ASM之前能够获得。

2.The Grid Interprocess Communication (GIPC) daemon

此守护进程支持GI通信。

3.The multicast DNS (mDNS) service

4.The Grid Naming Service (GNS)

Cluster Ready Serevice Stack包含一下daemons and service

1.Cluster Ready Services (CRS):

Cluster Ready Service Daemon:CRSD(manage the culsterware resource ofr app and handle the process for starting,stopping,

and failing,maintail all configuration profiles,monitoring instance,listeners service).

2.Oracle Cluster Synchronization Service Daemon:OCSSD

provide the mechanism that grants access to node membership(group service,locking,concurrency operation.这个daemon终止，将引起机器重启)

3.The Cluster Synchronization Services Agent (cssdagent):

start,stop monitor the status of the ocssd daemon,if the cssdagent discovers the ocssd daemon has stopped,it shutdown the node.

4.The Cluster Synchronization Services Monitor (cssdmonitor) process

monitors the scheduling of CPU on the node,如果发现节点HANG,shutdown the node 保证数据一致性 .

5.The Disk Monitor (diskmon) daemon

The diskmon daemon monitors and performs I/O fencing for Exadata storage. On the basis that Exadata storage could be added

to any RAC node at any time, the diskmon daemon is always started when ocssd starts.

Therefore, the diskmon daemon will always be present, regardless of whether the cluster uses Exadata storage

6.The Oracle Clusterware Kill (oclskd) daemon

is used by CSS to stop processes associated with CSS group members for which stop requests have been received from other nodes in the cluster

7.The Cluster Time Synchronization Service (CTSS)

8.Event Monitor Daemon:EVMD

9.The Event Manager Logger (EVMLOGGER) daemon

The evmlogger daemon is started by the evmd daemon. It subscribes to a list of events read from a

configuration file, and it runs user-defined actions when those events occur. This daemon is intended to maintain backward compatibility

10.The Oracle Notification Service (ONS, eONS)

This publish-and-subscribe service distributes Fast Application Notification (FAN) events to interested clients in the

environment. Oracle plans to merge ONS and eONS into the evmd daemon with one of the next patchsets for 11.2

RAC事件和信息管理，EVMD产生一个持久的子进程evmlogger，此进程产生各种logfile，配置文件located:$ORACLE_HOME/evm/admin/conf/evmlogger.conf

四.启动顺序

1.initialization and shutdown scripts for clusterware

before 11G R2:

orginate from>init daemon(/etc/inittab)=

# Run xdm in runlevel 5

x:5:respawn:/etc/X11/prefdm -nodaemon

h1:35:respawn:/etc/init.d/init.evmd run >/dev/null 2>&1 </dev/null

h2:35:respawn:/etc/init.d/init.cssd fatal >/dev/null 2>&1 </dev/nul l

h3:35:respawn:/etc/init.d/init.crsd run >/dev/null 2>&1 </dev/null
1.启动守护进程

h1:35:respawn:/etc/init.d/init.ohasd run >/dev/null 2>&1 </dev/null.

2.调用/etc/init.d/ohasd script，启动$GRID_HOME/bin/ohasd.bin

3.ohasd.bin调用$GRID_HOME/bin/下相关文件，产生：

oraagent.bin, started as the Grid software owner

cssdmonitor, started as root

cssdagent, started as root

orarootagent.bin, started as root

4.Oracle Root Agent启动以下执行文件，来自于$GRID_HOME/bin/

crsd.bin: started as root

diskmon.bin: started as the Grid software owner

octssd.bin: started as root

5.Oracle Agent作为Grid用户启动如下：

evmd.bin: started as the Grid software owner

evmlogger.bin: started as the Grid software owner

gipcd.bin: started as the Grid software owner

gpnpd.bin: started as the Grid software owner

mdnsd.bin: started as the Grid software owner

6,一旦CRS守护进程被Oracel Root Agent创建，CRS开始启动，将创建Oracle Agent和root agent，

这个里的agent区别于OHASD创建的agent，可以通过log的位置来判断,$LOG_HOME/agent/crsd/,OHASD创建的agent log位于$LOG_HOME/agent/ohasd。

crsd.bin创建如下：

oraagent.bin: started as the Grid software owner

oraagent.bin: started as oracle

oraarootgent.bin: started as root

7,grid software owner的oracle agent(oraagent.bin)创建如下，拥有者为grid

Clustered ASM instance

ons

enhanced ONS (eONS), a Java process

tnslsnr: a SCAN listener

tnslsnr: a node listener

8,最后Oracle Root Agent will start the following resources:

The network resource

The SCAN virtual IP address

The Node virtual IP address

The GNS virtual IP address if GNS is configured

The GNS daemon if the cluster is configured to use GNS

The ASM Cluster File System Registry
五.管理工具介绍

1,Clusterware Control Utility

手动启动和停止clusterware

[root@node1]# ./crsctl start cluster -all

[root@node1]# ./crsctl stop cluster –all

检查状态

crsctl check cluster -all

**************************************************************

node1:

CRS-4537: Cluster Ready Services is online

CRS-4529: Cluster Synchronization Services is online

CRS-4533: Event Manager is online

**************************************************************

node2:

CRS-4537: Cluster Ready Services is online

CRS-4529: Cluster Synchronization Services is online

CRS-4533: Event Manager is online

**************************************************************

查看HA Service Stack状态：

[grid@node2 ~]$ crsctl status resource -t -init

--------------------------------------------------------------------------------

NAME TARGET STATE SERVER STATE_DETAILS 

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.asm

 1 ONLINE ONLINE node2 Started 

ora.cluster_interconnect.haip

 1 ONLINE ONLINE node2 

ora.crf

 1 ONLINE ONLINE node2 

ora.crsd

 1 ONLINE ONLINE node2 

ora.cssd

 1 ONLINE ONLINE node2 

ora.cssdmonitor

 1 ONLINE ONLINE node2 

ora.ctssd

 1 ONLINE ONLINE node2 OBSERVER 

ora.diskmon

 1 OFFLINE OFFLINE 

ora.drivers.acfs

 1 ONLINE ONLINE node2 

ora.evmd

 1 ONLINE ONLINE node2 

ora.gipcd

 1 ONLINE ONLINE node2 

ora.gpnpd

 1 ONLINE ONLINE node2 

ora.mdnsd

 1 ONLINE ONLINE node2 
使用SRVCTL管理resource

enable

disable

start

stop

relocate

status

add

remove

modify

config

getenv

setenv

unsetenv

Database

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Instance

Y

Y

Y

Y

Y

Y

Y

Y

Service

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Nodeapps

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Vip

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Asm

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Diskgroup

Y

Y

Y

Y

Y

Y

Listener

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Scan

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Scan_listener

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Srvpool

Y

Y

Y

Y

Y

Server

Y

Y

Oc4j

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Home

Y

Y

Y

Filesystem

Y

Y

Y

Y

Y

Y

Y

Y

Y

gns

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

其它工具续二.