Klaus,
What version or the RDBMS are you using for the ASM instance and for the
database? There are several known issues that may be the cause here.
Also how is raw11 presented to st02 and raw12 presented to st01?
-Bennett
Hahn, Klaus wrote:
> Hi list,
>
> I'm testing RAC + ASM + iSCSI-shared-storage on SUSE SLES10 SP1.
>
> Configuration of RAC:
> 1. 2 nodes (a + b) with 2 instances TEST1 and TEST2
>
> Configuration of ASM:
> 1. normal redundancy
> 2. Diskgroup data with 2 failure groups st01_data and st02_data.
> Each failure group has 1 disk (/dev/raw/raw11 and /dev/raw/raw12)
> raw11 is an iSCSI-Target from node st01.
> raw12 is an iSCSI-Target from node st02.
> 3. Diskgroup reco (recovery area) with 2 failure groups st01_reco and
> st02_reco.
> Each failure group has 1 disk (/dev/raw/raw6 and /dev/raw/raw7)
> raw6 is an iSCSI-Target from node st01.
> raw7 is an iSCSI-Target from node st02.
>
> Taking offline of the iSCSI-Target node st01 produce a strange error
> within the database:
> - no commit possible
> - alter system switch logfile not possible
> - database seems to be readonly
>
> srvctl says: All RAC-Services and Instances are running.
>
>
>
> alert.log of RAC-Instance TEST1:
>
>
>>>>>> ..
>>>>>>
>
ORA-27091: I/O kann nicht in Queue gestellt werden
>
ORA-27072: Datei-I/O-Fehler
> Linux-x86_64 Error: 5: Input/output error
> Additional information: 4
> Additional information: 192544
> Additional information: -1
> Tue Oct 30 11:57:10 2007
> Errors in file /opt/oracle/admin/TEST/udump/test1_ora_30342.trc:
>
ORA-27091: I/O kann nicht in Queue gestellt werden
>
ORA-27072: Datei-I/O-Fehler
> Linux-x86_64 Error: 5: Input/output error
> Additional information: 4
> Additional information: 192544
> Additional information: -1
> WARNING: offlining disk 2.2526723344 (RECO_0002) with mask 0x3
> Tue Oct 30 11:57:19 2007
> Errors in file /opt/oracle/admin/TEST/bdump/test1_ckpt_15122.trc:
>
ORA-27091: I/O kann nicht in Queue gestellt werden
>
ORA-27072: Datei-I/O-Fehler
> Linux-x86_64 Error: 5: Input/output error
> Additional information: 4
> Additional information: 192608
> Additional information: -1
> WARNING: offlining disk 2.2526723344 (RECO_0002) with mask 0x3
> Tue Oct 30 12:34:16 2007
>
>>>> WAITED TOO LONG FOR A ROW CACHE ENQUEUE LOCK! pid=16
>>>>
> System State dumped to trace file
> /opt/oracle/admin/TEST/bdump/test1_mmon_15153.trc
> Tue Oct 30 12:47:03 2007
>
>>>> WAITED TOO LONG FOR A ROW CACHE ENQUEUE LOCK! pid=37
>>>>
> System State dumped to trace file
> /opt/oracle/admin/TEST/udump/test1_ora_23960.trc
> Tue Oct 30 12:50:53 2007
>
>>>> WAITED TOO LONG FOR A ROW CACHE ENQUEUE LOCK! pid=54
>>>>
> System State dumped to trace file
> /opt/oracle/admin/TEST/udump/test1_ora_14206.trc
>
> <<<<<<
>
> alert.log of ASM-Instance: +ASM1
>
> <<<<<<<<<<<<
>
ORA-27091: unable to queue I/O
>
ORA-27072: File I/O error
> Linux-x86_64 Error: 5: Input/output error
> Additional information: 4
> Additional information: 2056
> Additional information: -1
> Tue Oct 30 11:57:19 2007
> Errors in file /opt/oracle/admin/+ASM/bdump/+asm1_gmon_28227.trc:
>
ORA-27091: unable to queue I/O
>
ORA-27072: File I/O error
> Linux-x86_64 Error: 5: Input/output error
> Additional information: 4
> Additional information: 2048
> Additional information: -1
> Tue Oct 30 11:57:19 2007
> Errors in file /opt/oracle/admin/+ASM/bdump/+asm1_gmon_28227.trc:
>
ORA-27091: unable to queue I/O
>
ORA-27072: File I/O error
> Linux-x86_64 Error: 5: Input/output error
> Additional information: 4
> Additional information: 2056
> Additional information: -1
> NOTE: cache closing disk 2 of grp 2: RECO_0002
> Tue Oct 30 11:57:19 2007
> NOTE: group RECO: relocated PST to: disk 0000 (PST copy 0)
> Tue Oct 30 11:57:22 2007
> NOTE: PST update: grp = 2, dsk = 2, mode = 0x4
> Tue Oct 30 11:57:22 2007
> NOTE: group RECO: relocated PST to: disk 0000 (PST copy 0)
> NOTE: cache closing disk 2 of grp 2: RECO_0002
> Tue Oct 30 11:57:52 2007
> NOTE: PST refresh pending for group 1/0x175a4df9 (DATA)
> SUCCESS: refreshed PST for 1/0x175a4df9 (DATA)
> NOTE: PST refresh pending for group 2/0x506a4dfe (RECO)
> SUCCESS: refreshed PST for 2/0x506a4dfe (RECO)
> Tue Oct 30 11:59:24 2007
> SUCCESS: refreshed PST for 1/0x175a4df9 (DATA)
> SUCCESS: refreshed PST for 2/0x506a4dfe (RECO)
> <<<<<<<<<<<
>
> Reconfiguration of ASM by " alter diskgroup add failgroup ..." (after
> deleting the asm-metadata on the raw device)
> takes no effect.
>
> After restart of the RAC-instances all work fine !
>
> Any hint ?
>
> Regards
> Klaus
>
>
>
>
