Tuesday, March 8, 2016

Griddisks not coming ONLINE after execution of alter griddisk all active

SYMPTOMS:

A cell-to-cell offload operation occurs, such as when performing a resilvering operation during ASM disk resync, or when issuing "alter diskgroup drop disks" statement in ASM

Affected Versions:

Grid Infrastructure version is 12.1.0.2, lower than 12.1.0.2.160119

The fix for bug 21218243 is installed in the Grid Infrastructure home.  This fix was included in 12.1.0.2.11, 12.1.0.2.12, and 12.1.0.2.13, but was also provided as a backport to some previous bundle patches. The fix for bug 22304421 is not installed in the Grid Infrastructure home.  This is the resolution fix for this issue.

Workaround:

Step 1 - Verify cellsrv process is running on all storage servers, and restart, if necessary.

CellCLI> list cell attributes cellsrvStatus
         stopped

CellCLI> alter cell startup services cellsrv

Starting CELLSRV services...
The STARTUP of CELLSRV services was successful.

Step 2 - Set initialization parameter _kxdbio_disable_offload_opcode=1 in all ASM instances.
This parameter may be set while ASM remains online.  For example,
SQL> alter system set "_kxdbio_disable_offload_opcode"=1;
System altered.
_kxdbio_disable_offload_opcode=1 disables cell-to-cell offload operations, such as cell-to-cell offloaded resilvering during ASM disk resync.  Such operations will proceed without direct cell-to-cell communication when this parameter is set as directed.  Other offloaded operations, such as query, incremental backup, and file creation, are unaffected when setting _kxdbio_disable_offload_opcode=1.


Note: This workaround should be used as a temporary remedy only and should be removed upon using the recommended resolution

 On 2015-Nov-09 patch 21976960 was released to correct this issue.  Subsequently, it was discovered that patch 21976960 did not prevent the issue in all possible scenarios and was withdrawn.  On 2015-Dec-22 patch 22304421 replaced patch 21976960.  If patch 21976960 was previously installed, it is not necessary to rollback prior to taking additional action.

Reference: