IT街道－－遥远的博客

2009年4月参加了Oracle举办的RAC Master Workshop（3天，只有Oracle partner参加），见到了好多高手。课上根据老师的安排用虚拟机安装RAC，但ASM只能mount到一个节点。从网上搜索发现好多帖子都是类似的问题，但没有一个能找到root cause。本人经过一番周折终于把这个问题解决了，估计有很多人最后也解决了，但可能因为经过的时间比较长，就不想回复原来的帖子了。特写这个blog，希望有类似问题的人能看到后，能少走点弯路。
如果读者没有耐心就直接看最后一段的结论吧。

从第一个节点装ASM，在第二个节点不能mount，提示：
Could not mount the diskgroup on remount node rac1 using connection
service rac1:1521:+ASM1.Ensure that the listener is running on this node and the ASM
instance is registered to the listener. Received the following error:


ORA-15032:not all alterations performed
ORA=15063:ASM discovered an insufficient number of disks for diskgroup "ASM_DISK1"

从第二个节点装ASM，在第一个节点也出现类似的问题。

CRS的状态正常

$ /u/oracle/10gR2/crs/bin/crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora....C1.lsnr application    ONLINE    ONLINE    rac1
ora.rac1.gsd   application    ONLINE    ONLINE    rac1
ora.rac1.ons   application    ONLINE    ONLINE    rac1
ora.rac1.vip   application    ONLINE    ONLINE    rac1
ora....C2.lsnr application    ONLINE    ONLINE    rac2
ora.rac2.gsd   application    ONLINE    ONLINE    rac2
ora.rac2.ons   application    ONLINE    ONLINE    rac2
ora.rac2.vip   application    ONLINE    ONLINE    rac2

ASM的硬盘权限正常
[oracle@RAC1 ~]$ ll /dev/sd*1
brw-r----- 1 root   disk     8,  1 Apr 29 10:59 /dev/sda1
brw-rw---- 1 oracle oinstall 8, 17 Apr 29 11:01 /dev/sdb1
brw-rw---- 1 oracle oinstall 8, 33 Apr 29 11:54 /dev/sdc1
brw-r----- 1 oracle oinstall 8, 49 Apr 29 11:01 /dev/sdd1
brw-r----- 1 oracle oinstall 8, 65 Apr 29 11:01 /dev/sde1
brw-r----- 1 oracle oinstall 8, 81 Apr 29 11:01 /dev/sdf1

第一个节点的listener已经正常监听ASM1
$ lsnrctl status

LSNRCTL for Linux: Version 11.1.0.6.0 - Production on 29-APR-2009 11:53:47

Copyright (c) 1991, 2007, Oracle.  All rights reserved.

Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
STATUS of the LISTENER
------------------------
Alias                     LISTENER_RAC1
Version                   TNSLSNR for Linux: Version 11.1.0.6.0 - Production
Start Date                29-APR-2009 11:14:01
Uptime                    0 days 0 hr. 39 min. 47 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /u/oracle/10gR2/db/network/admin/listener.ora
Listener Log File         /u/oracle/diag/tnslsnr/RAC1/listener_rac1/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.15.188)(PORT=1521)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.15.88)(PORT=1521)))
Services Summary...
Service "+ASM" has 1 instance(s).
  Instance "+ASM1", status READY, has 1 handler(s) for this service...
Service "+ASM_XPT" has 1 instance(s).
  Instance "+ASM1", status READY, has 1 handler(s) for this service...
The command completed successfully


第二个节点的listener已经正常监听ASM2
[oracle@RAC2 ~]$ lsnrctl status

LSNRCTL for Linux: Version 11.1.0.6.0 - Production on 29-APR-2009 11:57:00

Copyright (c) 1991, 2007, Oracle.  All rights reserved.

Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
STATUS of the LISTENER
------------------------
Alias                     LISTENER_RAC2
Version                   TNSLSNR for Linux: Version 11.1.0.6.0 - Production
Start Date                29-APR-2009 11:14:05
Uptime                    0 days 0 hr. 42 min. 55 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /u/oracle/10gR2/db/network/admin/listener.ora
Listener Log File         /u/oracle/diag/tnslsnr/RAC2/listener_rac2/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.15.199)(PORT=1521)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.15.99)(PORT=1521)))
Services Summary...
Service "+ASM" has 1 instance(s).
  Instance "+ASM2", status READY, has 1 handler(s) for this service...
Service "+ASM_XPT" has 1 instance(s).
  Instance "+ASM2", status READY, has 1 handler(s) for this service...
The command completed successfully



在第一个节点读ASM磁盘正常

[oracle@RAC1 ~]$ dd if=/dev/raw/raw3 of=/tmp/sdd1.out count=1 bs=1024

1+0 records in

1+0 records out

1024 bytes (1.0 kB) copied, 0.0230128 seconds, 44.5 kB/s

[oracle@RAC1 ~]$

[oracle@RAC1 ~]$ dd if=/dev/raw/raw4 of=/tmp/sde1.out count=1 bs=1024

1+0 records in

1+0 records out

1024 bytes (1.0 kB) copied, 0.0539979 seconds, 19.0 kB/s

[oracle@RAC1 ~]$

[oracle@RAC1 ~]$ dd if=/dev/raw/raw5 of=/tmp/sdf1.out count=1 bs=1024

1+0 records in

1+0 records out

1024 bytes (1.0 kB) copied, 0.0190941 seconds, 53.6 kB/s


在第二个节点读ASM磁盘正常
 

[oracle@RAC2 ~]$ dd if=/dev/raw/raw3 of=/tmp/sdd1.out count=1 bs=1024

1+0 records in

1+0 records out

1024 bytes (1.0 kB) copied, 0.0339595 seconds, 30.2 kB/s

[oracle@RAC2 ~]$

[oracle@RAC2 ~]$ dd if=/dev/raw/raw4 of=/tmp/sde1.out count=1 bs=1024

1+0 records in

1+0 records out

1024 bytes (1.0 kB) copied, 0.0584076 seconds, 17.5 kB/s

[oracle@RAC2 ~]$

[oracle@RAC2 ~]$ dd if=/dev/raw/raw5 of=/tmp/sdf1.out count=1 bs=1024

1+0 records in

1+0 records out

1024 bytes (1.0 kB) copied, 0.0460453 seconds, 22.2 kB/s

[oracle@RAC2 ~]$

 




总结：
    经过以上分析，软件配置等各个方面都是正常的，那只能从基础环境上找原因了，开始怀疑的vmware的版本问题，本人用的是
workstation 6.5，据Oracle的老师讲workstation 6.0以上都能支持cluster的，而且前面的cfs已经是正常的了，应该也能支持
ASM的呀！但我抱着试一试的心态用了vmware的server版，结果好了，ASM两个节点都能mount上了！