ASM - Online OCR and Voting Adding on ASM Diskgroup using Normal Redundancy

 ASM -   Online OCR and Voting disk SAN LUN Migration on ASM Diskgroup Normal Redundancy

Once 3 ASM disks are created at OS level using oracleasm utility. We will add to CRS ASM diskgroup online as OCR and voting disk is stored on it.

See also  blog link for details in alert log: ASM - Online OCR and Voting disk SAN LUN Migration on ASM Diskgroup

Redundancy Level: Normal

Grid Version: 12.1.0.2.0

• Create ASM disk of same size using oracleasm utility

[root@srv1 ~]# oracleasm createdisk CRSNEW1 /dev/sdc1

Writing disk header: done

Instantiating disk: done

[root@srv1 ~]# oracleasm createdisk CRSNEW2 /dev/sdd1

Writing disk header: done

Instantiating disk: done

[root@srv1 ~]# oracleasm createdisk CRSNEW3 /dev/sdk1

Writing disk header: done

Instantiating disk: done

[root@srv1 ~]# oracleasm listdisks

CRS1

CRS2

CRS3

CRSNEW

CRSNEW1

CRSNEW2

CRSNEW3

DATANEW

FRANEW

• Login to the ASM instance using grid user and check the current diskgroup status.

[root@srv1 ~]# su grid

[grid@srv1 root]$ cd

[grid@srv1 ~]$ . oraenv

ORACLE_SID = [+ASM1] ?

The Oracle base remains unchanged with value /u01/app/grid

[grid@srv1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 12.1.0.2.0 Production on Fri Apr 2 17:45:51 2021

Copyright (c) 1982, 2014, Oracle.  All rights reserved.

Connected to:

Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production

With the Real Application Clusters and Automatic Storage Management options

SQL> 

set lines 999;

col diskgroup for a10

col diskname for a12

col path for a30

select a.name DiskGroup,b.name DiskName, b.total_mb, (b.total_mb-b.free_mb) Used_MB, b.free_mb,b.path,b.header_status from v$asm_disk b, v$asm_diskgroup a where a.group_number (+) =b.group_number order by b.group_number,b.name;

DISKGROUP  DISKNAME       TOTAL_MB    USED_MB    FREE_MB PATH                           HEADER_STATU

---------- ------------ ---------- ---------- ---------- ------------------------------ ------------

                                 0          0          0 /dev/oracleasm/disks/CRSNEW    FORMER

                                 0          0          0 /dev/oracleasm/disks/CRSNEW1   PROVISIONED

                                 0          0          0 /dev/oracleasm/disks/CRSNEW3   PROVISIONED

                                 0          0          0 /dev/oracleasm/disks/CRSNEW2   PROVISIONED

CRS        CRS1              12284       1519      10765 /dev/oracleasm/disks/CRS1      MEMBER

CRS        CRS2              12284       1484      10800 /dev/oracleasm/disks/CRS2      MEMBER

CRS        CRS3              12284       1485      10799 /dev/oracleasm/disks/CRS3      MEMBER

DATA       DATA_0001         20473       2476      17997 /dev/oracleasm/disks/DATANEW   MEMBER

FRA        FRA_0001          20473       1075      19398 /dev/oracleasm/disks/FRANEW    MEMBER

9 rows selected.

SQL>  set pages 40000 lines 120

col PATH for a30

select DISK_NUMBER,MOUNT_STATUS,HEADER_STATUS,MODE_STATUS,STATE,PATH FROM V$ASM_DISK;

DISK_NUMBER MOUNT_S HEADER_STATU MODE_ST STATE    PATH

----------- ------- ------------ ------- -------- ------------------------------

          0 CLOSED  FORMER       ONLINE  NORMAL   /dev/oracleasm/disks/CRSNEW

          1 CLOSED  PROVISIONED  ONLINE  NORMAL   /dev/oracleasm/disks/CRSNEW1

          2 CLOSED  PROVISIONED  ONLINE  NORMAL   /dev/oracleasm/disks/CRSNEW3

          3 CLOSED  PROVISIONED  ONLINE  NORMAL   /dev/oracleasm/disks/CRSNEW2

          1 CACHED  MEMBER       ONLINE  NORMAL   /dev/oracleasm/disks/FRANEW

          3 CACHED  MEMBER       ONLINE  NORMAL   /dev/oracleasm/disks/CRS3

          1 CACHED  MEMBER       ONLINE  NORMAL   /dev/oracleasm/disks/DATANEW

          2 CACHED  MEMBER       ONLINE  NORMAL   /dev/oracleasm/disks/CRS2

          0 CACHED  MEMBER       ONLINE  NORMAL   /dev/oracleasm/disks/CRS1

9 rows selected.

• Add these provisioned disks in CRS diskgroup 

SQL> ALTER DISKGROUP CRS ADD DISK '/dev/oracleasm/disks/CRSNEW1' NAME CRSNEW1,'/dev/oracleasm/disks/CRSNEW2' NAME CRSNEW2,'/dev/oracleasm/disks/CRSNEW3' NAME CRSNEW3;

Diskgroup altered.

• Check Rebalance operation to complete using below query.

SQL> Select operation, state, est_work, est_minutes from v$asm_operation;

OPERA STAT   EST_WORK EST_MINUTES

----- ---- ---------- -----------

REBAL RUN        2218           3

REBAL WAIT          0           0

SQL> /

OPERA STAT   EST_WORK EST_MINUTES

----- ---- ---------- -----------

REBAL RUN        2218           1

REBAL WAIT          0           0

SQL> /

OPERA STAT   EST_WORK EST_MINUTES

----- ---- ---------- -----------

REBAL DONE       2218           0

REBAL RUN           0           0

SQL> Select operation, state, est_work, est_minutes from v$asm_operation;

OPERA STAT   EST_WORK EST_MINUTES

----- ---- ---------- -----------

REBAL DONE       2218           0

REBAL RUN           0           0

SQL> /

no rows selected

• Check the status of Diskgroup

SQL> 

set pages 40000 lines 120

col PATH for a30

select DISK_NUMBER,MOUNT_STATUS,HEADER_STATUS,MODE_STATUS,STATE,PATH FROM V$ASM_DISK;

DISK_NUMBER MOUNT_S HEADER_STATU MODE_ST STATE    PATH

----------- ------- ------------ ------- -------- ------------------------------

          0 CLOSED  FORMER       ONLINE  NORMAL   /dev/oracleasm/disks/CRSNEW

          1 CACHED  MEMBER       ONLINE  NORMAL   /dev/oracleasm/disks/FRANEW

          3 CACHED  MEMBER       ONLINE  NORMAL   /dev/oracleasm/disks/CRS3

          1 CACHED  MEMBER       ONLINE  NORMAL   /dev/oracleasm/disks/DATANEW

          2 CACHED  MEMBER       ONLINE  NORMAL   /dev/oracleasm/disks/CRS2

          5 CACHED  MEMBER       ONLINE  NORMAL   /dev/oracleasm/disks/CRSNEW3

          0 CACHED  MEMBER       ONLINE  NORMAL   /dev/oracleasm/disks/CRS1

          4 CACHED  MEMBER       ONLINE  NORMAL   /dev/oracleasm/disks/CRSNEW2

          1 CACHED  MEMBER       ONLINE  NORMAL   /dev/oracleasm/disks/CRSNEW1

9 rows selected.

SQL> 

set lines 999;

col diskgroup for a10

col diskname for a12

 col path for a30

select a.name DiskGroup,b.name DiskName, b.total_mb, (b.total_mb-b.free_mb) Used_MB, b.free_mb,b.path,b.header_status from v$asm_disk b, v$asm_diskgroup a where a.group_number (+) =b.group_number order by b.group_number,b.name;SQL>

DISKGROUP  DISKNAME       TOTAL_MB    USED_MB    FREE_MB PATH                           HEADER_STATU

---------- ------------ ---------- ---------- ---------- ------------------------------ ------------

                                 0          0          0 /dev/oracleasm/disks/CRSNEW    FORMER

CRS        CRS1              12284        778      11506 /dev/oracleasm/disks/CRS1      MEMBER

CRS        CRS2              12284        746      11538 /dev/oracleasm/disks/CRS2      MEMBER

CRS        CRS3              12284        746      11538 /dev/oracleasm/disks/CRS3      MEMBER

CRS        CRSNEW1           12284        743      11541 /dev/oracleasm/disks/CRSNEW1   MEMBER

CRS        CRSNEW2           12284        742      11542 /dev/oracleasm/disks/CRSNEW2   MEMBER

CRS        CRSNEW3           12284        742      11542 /dev/oracleasm/disks/CRSNEW3   MEMBER

DATA       DATA_0001         20473       2476      17997 /dev/oracleasm/disks/DATANEW   MEMBER

FRA        FRA_0001          20473       1075      19398 /dev/oracleasm/disks/FRANEW    MEMBER

9 rows selected.

• Drop old Diskgroup of CRS with disk CRS1,CRS2,CRS3

SQL> alter diskgroup CRS drop disk 'CRS1';

Diskgroup altered.

SQL> alter diskgroup CRS drop disk 'CRS2';

Diskgroup altered.

SQL> alter diskgroup CRS drop disk 'CRS3';

Diskgroup altered.

• Check query to complete rebalance operations.

SQL> Select operation, state, est_work, est_minutes from v$asm_operation;

OPERA STAT   EST_WORK EST_MINUTES

----- ---- ---------- -----------

REBAL RUN        2177           2

REBAL WAIT          0           0

SQL> Select operation, state, est_work, est_minutes from v$asm_operation;

OPERA STAT   EST_WORK EST_MINUTES

----- ---- ---------- -----------

REBAL RUN        2177           2

REBAL WAIT          0           0

SQL> /

OPERA STAT   EST_WORK EST_MINUTES

----- ---- ---------- -----------

REBAL RUN        2177           1

REBAL WAIT          0           0

SQL> /

no rows selected

• Check the current status of disk CRS1, CRS2,CRS3

SQL>

 set lines 999;

col diskgroup for a10

col diskname for a12

col path for a30

select a.name DiskGroup,b.name DiskName, b.total_mb, (b.total_mb-b.free_mb) Used_MB, b.free_mb,b.path,b.header_status from v$asm_disk b, v$asm_diskgroup a where a.group_number (+) =b.group_number order by b.group_number,b.name;

DISKGROUP  DISKNAME       TOTAL_MB    USED_MB    FREE_MB PATH                           HEADER_STATU

---------- ------------ ---------- ---------- ---------- ------------------------------ ------------

                                 0          0          0 /dev/oracleasm/disks/CRSNEW    FORMER

                                 0          0          0 /dev/oracleasm/disks/CRS1      FORMER

                                 0          0          0 /dev/oracleasm/disks/CRS3      FORMER

                                 0          0          0 /dev/oracleasm/disks/CRS2      FORMER

CRS        CRSNEW1           12284       1517      10767 /dev/oracleasm/disks/CRSNEW1   MEMBER

CRS        CRSNEW2           12284       1485      10799 /dev/oracleasm/disks/CRSNEW2   MEMBER

CRS        CRSNEW3           12284       1486      10798 /dev/oracleasm/disks/CRSNEW3   MEMBER

DATA       DATA_0001         20473       2476      17997 /dev/oracleasm/disks/DATANEW   MEMBER

FRA        FRA_0001          20473       1075      19398 /dev/oracleasm/disks/FRANEW    MEMBER

9 rows selected.

SQL> /

DISKGROUP  DISKNAME       TOTAL_MB    USED_MB    FREE_MB PATH                           HEADER_STATU

---------- ------------ ---------- ---------- ---------- ------------------------------ ------------

                                 0          0          0 /dev/oracleasm/disks/CRSNEW    FORMER

                                 0          0          0 /dev/oracleasm/disks/CRS1      FORMER

                                 0          0          0 /dev/oracleasm/disks/CRS3      FORMER

                                 0          0          0 /dev/oracleasm/disks/CRS2      FORMER

CRS        CRSNEW1           12284       1517      10767 /dev/oracleasm/disks/CRSNEW1   MEMBER

CRS        CRSNEW2           12284       1485      10799 /dev/oracleasm/disks/CRSNEW2   MEMBER

CRS        CRSNEW3           12284       1486      10798 /dev/oracleasm/disks/CRSNEW3   MEMBER

DATA       DATA_0001         20473       2476      17997 /dev/oracleasm/disks/DATANEW   MEMBER

FRA        FRA_0001          20473       1075      19398 /dev/oracleasm/disks/FRANEW    MEMBER

9 rows selected.

SQL> /

DISKGROUP  DISKNAME       TOTAL_MB    USED_MB    FREE_MB PATH                           HEADER_STATU

---------- ------------ ---------- ---------- ---------- ------------------------------ ------------

                                 0          0          0 /dev/oracleasm/disks/CRSNEW    FORMER

                                 0          0          0 /dev/oracleasm/disks/CRS1      FORMER

                                 0          0          0 /dev/oracleasm/disks/CRS3      FORMER

                                 0          0          0 /dev/oracleasm/disks/CRS2      FORMER

CRS        CRSNEW1           12284       1517      10767 /dev/oracleasm/disks/CRSNEW1   MEMBER

CRS        CRSNEW2           12284       1485      10799 /dev/oracleasm/disks/CRSNEW2   MEMBER

CRS        CRSNEW3           12284       1486      10798 /dev/oracleasm/disks/CRSNEW3   MEMBER

DATA       DATA_0001         20473       2476      17997 /dev/oracleasm/disks/DATANEW   MEMBER

FRA        FRA_0001          20473       1075      19398 /dev/oracleasm/disks/FRANEW    MEMBER

9 rows selected.

• Delete Disk at OS level using oracleasm utility

srv1:/home/grid # oracleasm deletedisk CRS1

Disk "CRS1" defines an unmarked device

Dropping disk: done

srv1:/home/grid # oracleasm deletedisk CRS2

Clearing disk header: done

Dropping disk: done

srv1:/home/grid # oracleasm deletedisk CRS3

Clearing disk header: done

Dropping disk: done

SQL> /

• Check query for current status of ASM Disk group

SQL>

 set lines 999;

col diskgroup for a10

col diskname for a12

col path for a30

select a.name DiskGroup,b.name DiskName, b.total_mb, (b.total_mb-b.free_mb) Used_MB, b.free_mb,b.path,b.header_status from v$asm_disk b, v$asm_diskgroup a where a.group_number (+) =b.group_number order by b.group_number,b.name;

DISKGROUP  DISKNAME       TOTAL_MB    USED_MB    FREE_MB PATH                           HEADER_STATU

---------- ------------ ---------- ---------- ---------- ------------------------------ ------------

                                 0          0          0 /dev/oracleasm/disks/CRSNEW    FORMER

CRS        CRSNEW1           12284       1517      10767 /dev/oracleasm/disks/CRSNEW1   MEMBER

CRS        CRSNEW2           12284       1485      10799 /dev/oracleasm/disks/CRSNEW2   MEMBER

CRS        CRSNEW3           12284       1486      10798 /dev/oracleasm/disks/CRSNEW3   MEMBER

DATA       DATA_0001         20473       2476      17997 /dev/oracleasm/disks/DATANEW   MEMBER

FRA        FRA_0001          20473       1075      19398 /dev/oracleasm/disks/FRANEW    MEMBER

6 rows selected.

############## Another Example of Online ASM disk group migration ###############

SQL> set lines 999;

SQL> col diskgroup for a10

SQL> col diskname for a12

SQL> col path for a30

SQL> select a.name DiskGroup,b.name DiskName, b.total_mb, (b.total_mb-b.free_mb) Used_MB, b.free_mb,b.path,b.header_status from v$asm_disk b, v$asm_diskgroup a where a.group_number (+) =b.group_number order by b.group_number,b.name;

DISKGROUP  DISKNAME       TOTAL_MB    USED_MB    FREE_MB PATH                           HEADER_STATU

---------- ------------ ---------- ---------- ---------- ------------------------------ ------------

                                 0          0          0 ORCL:CRSNEW1                   PROVISIONED

                                 0          0          0 ORCL:CRSNEW2                   PROVISIONED

                                 0          0          0 ORCL:CRSNEW3                   PROVISIONED

BACKUP     BACKUP            15359         96      15263 ORCL:BACKUP                    MEMBER

CRS        CRS1              12287       3047       9240 ORCL:CRS1                      MEMBER

CRS        CRS2              12287       3046       9241 ORCL:CRS2                      MEMBER

CRS        CRS3              12287       3043       9244 ORCL:CRS3                      MEMBER

DATA       DATA              20479       4249      16230 ORCL:DATA                      MEMBER

FRA        FRA               20479        937      19542 ORCL:FRA                       MEMBER

9 rows selected.

SQL> ALTER DISKGROUP CRS ADD DISK 'ORCL:CRSNEW1' NAME CRSNEW1,'ORCL:CRSNEW2' NAME CRSNEW2,'ORCL:CRSNEW3' NAME CRSNEW3;

Diskgroup altered.

SQL> set lines 999;

col diskgroup for a10

SQL> SQL> SQL> SQL> col diskname for a12

col path for a30

SQL> SQL> SQL> SQL> select a.name DiskGroup,b.name DiskName, b.total_mb, (b.total_mb-b.free_mb) Used_MB, b.free_mb,b.path,b.header_status from v$asm_disk b, v$asm_diskgroup a where a.group_number (+) =b.group_number order by b.group_number,b.name;

DISKGROUP  DISKNAME       TOTAL_MB    USED_MB    FREE_MB PATH                           HEADER_STATU

---------- ------------ ---------- ---------- ---------- ------------------------------ ------------

BACKUP     BACKUP            15359         96      15263 ORCL:BACKUP                    MEMBER

CRS        CRS1              12287       3017       9270 ORCL:CRS1                      MEMBER

CRS        CRS2              12287       3014       9273 ORCL:CRS2                      MEMBER

CRS        CRS3              12287       3011       9276 ORCL:CRS3                      MEMBER

CRS        CRSNEW1           12287         34      12253 ORCL:CRSNEW1                   MEMBER

CRS        CRSNEW2           12287         34      12253 ORCL:CRSNEW2                   MEMBER

CRS        CRSNEW3           12287         35      12252 ORCL:CRSNEW3                   MEMBER

DATA       DATA              20479       4249      16230 ORCL:DATA                      MEMBER

FRA        FRA               20479        937      19542 ORCL:FRA                       MEMBER

9 rows selected.

srv1:/home/grid # oracleasm deletedisk CRS1

Disk "CRS1" defines an unmarked device

Dropping disk: done

srv1:/home/grid # oracleasm deletedisk CRS2

Clearing disk header: done

Dropping disk: done

srv1:/home/grid # oracleasm deletedisk CRS3

Clearing disk header: done

Dropping disk: done

srv1:/home/grid # su grid

grid@srv1:~> sqlplus / as sysasm

SQL*Plus: Release 12.1.0.2.0 Production on Thu Apr 8 13:16:53 2021

Copyright (c) 1982, 2014, Oracle.  All rights reserved.

Connected to:

Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production

With the Real Application Clusters and Automatic Storage Management options

SQL> set lines 999;

SQL> SQL> col diskgroup for a10

col diskname for a12

SQL> SQL> SQL> SQL> col path for a30

select a.name DiskGroup,b.name DiskName, b.total_mb, (b.total_mb-b.free_mb) Used_MB, b.free_mb,b.path,b.header_status from v$asm_disk b, v$asm_diskgroup a where a.group_number (+) =b.group_number order by b.group_number,b.name;

SQL> SQL>

DISKGROUP  DISKNAME       TOTAL_MB    USED_MB    FREE_MB PATH                           HEADER_STATU

---------- ------------ ---------- ---------- ---------- ------------------------------ ------------

BACKUP     BACKUP            15359         96      15263 ORCL:BACKUP                    MEMBER

CRS        CRSNEW1           12287       3045       9242 ORCL:CRSNEW1                   MEMBER

CRS        CRSNEW2           12287       3046       9241 ORCL:CRSNEW2                   MEMBER

CRS        CRSNEW3           12287       3045       9242 ORCL:CRSNEW3                   MEMBER

DATA       DATA              20479       4249      16230 ORCL:DATA                      MEMBER

FRA        FRA               20479        937      19542 ORCL:FRA                       MEMBER

6 rows selected.

SQL>

srv2:~ # oracleasm scandisks

Reloading disk partitions: done

Cleaning any stale ASM disks...

Cleaning disk "CRS1"

Cleaning disk "CRS2"

Cleaning disk "CRS3"

Scanning system for ASM disks...

srv2:~ # oracleasm listdisks

BACKUP

CRSNEW1

CRSNEW2

CRSNEW3

DATA

FRA

srv2:~ #

ASM - Online OCR and Voting disk SAN LUN Migration on ASM Diskgroup

ASM - Online OCR and Voting disk SAN LUN Migration on ASM Diskgroup

We are following oracle metalink doc for OCR and voting disk and mgmt database migration below in our test envirnment.

Exact Steps To Migrate ASM Diskgroups To Another SAN/Disk-Array/DAS/etc Without Downtime. (Doc ID 837308.1)

• Login using grid user and check diskgroup status , Here in CRS diskgroup OCR and voting disk stored.

SQL> set lines 999;

col diskgroup for a10

col diskname for a12

col path for a30

SQL> SQL> SQL> SQL> select a.name DiskGroup,b.name DiskName, b.total_mb, (b.total_mb-b.free_mb) Used_MB, b.free_mb,b.path,b.header_status from v$asm_disk b, v$asm_diskgroup a where a.group_number (+) =b.group_number order by b.group_number,b.name;

DISKGROUP  DISKNAME       TOTAL_MB    USED_MB    FREE_MB PATH                           HEADER_STATU

---------- ------------ ---------- ---------- ---------- ------------------------------ ------------

                                 0          0          0 /dev/oracleasm/disks/CRSNEW    FORMER

                                 0          0          0 /dev/oracleasm/disks/BACKUPNEW FORMER

BACKUP     BACKUP_0000       12284         96      12188 /dev/oracleasm/disks/BACKUP    MEMBER

CRS        CRS_0000          12284       4472       7812 /dev/oracleasm/disks/CRS       MEMBER

DATA       DATA_0001         20473       2281      18192 /dev/oracleasm/disks/DATANEW   MEMBER

FRA        FRA_0001          20473        606      19867 /dev/oracleasm/disks/FRANEW    MEMBER

6 rows selected.

SQL> set pages 40000 lines 120

col PATH for a30

select DISK_NUMBER,MOUNT_STATUS,HEADER_STATUS,MODE_STATUS,STATE,PATH FROM V$ASM_DISK;SQL> SQL>

DISK_NUMBER MOUNT_S HEADER_STATU MODE_ST STATE    PATH

----------- ------- ------------ ------- -------- ------------------------------

          0 CLOSED  FORMER       ONLINE  NORMAL   /dev/oracleasm/disks/CRSNEW

          1 CLOSED  FORMER       ONLINE  NORMAL   /dev/oracleasm/disks/BACKUPNEW

          1 CACHED  MEMBER       ONLINE  NORMAL   /dev/oracleasm/disks/FRANEW

          1 CACHED  MEMBER       ONLINE  NORMAL   /dev/oracleasm/disks/DATANEW

          0 CACHED  MEMBER       ONLINE  NORMAL   /dev/oracleasm/disks/CRS

          0 CACHED  MEMBER       ONLINE  NORMAL   /dev/oracleasm/disks/BACKUP

6 rows selected.

• Add disk on online in CRS diskgroup

SQL> alter diskgroup CRS add disk '/dev/oracleasm/disks/CRSNEW';

Diskgroup altered.

• Check rebalance operation status to complete. 

SQL> Select operation, state, est_work, est_minutes from v$asm_operation;

OPERA STAT   EST_WORK EST_MINUTES

----- ---- ---------- -----------

REBAL RUN        2217           1

REBAL WAIT          0           0

SQL> /

OPERA STAT   EST_WORK EST_MINUTES

----- ---- ---------- -----------

REBAL DONE       2217           0

REBAL RUN           0           0

SQL> /

OPERA STAT   EST_WORK EST_MINUTES

----- ---- ---------- -----------

REBAL DONE       2217           0

REBAL RUN           0           0

SQL> /

no rows selected

SQL>

• ASM Alert log details:

[root@srv2 trace]# tail -1000f alert_+ASM2.log

Wed Mar 31 20:34:20 2021

SQL> alter diskgroup CRS add disk '/dev/oracleasm/disks/CRSNEW'

Wed Mar 31 20:34:20 2021

NOTE: GroupBlock outside rolling migration privileged region

NOTE: Assigning number (4,1) to disk (/dev/oracleasm/disks/CRSNEW)

NOTE: requesting all-instance membership refresh for group=4

NOTE: Disk 1 in group 4 is assigned fgnum=2

NOTE: discarding redo for group 4 disk 1

NOTE: initializing header (replicated) on grp 4 disk CRS_0001

NOTE: initializing header on grp 4 disk CRS_0001

NOTE: requesting all-instance disk validation for group=4

Wed Mar 31 20:34:21 2021

NOTE: skipping rediscovery for group 4/0xb6a1c3c6 (CRS) on local instance.

Wed Mar 31 20:34:21 2021

NOTE: requesting all-instance disk validation for group=4

Wed Mar 31 20:34:21 2021

NOTE: skipping rediscovery for group 4/0xb6a1c3c6 (CRS) on local instance.

Wed Mar 31 20:34:21 2021

GMON updating for reconfiguration, group 4 at 55 for pid 30, osid 26667

Wed Mar 31 20:34:21 2021

NOTE: group 4 PST updated.

Wed Mar 31 20:34:21 2021

NOTE: membership refresh pending for group 4/0xb6a1c3c6 (CRS)

Wed Mar 31 20:34:21 2021

GMON querying group 4 at 56 for pid 22, osid 9992

NOTE: cache opening disk 1 of grp 4: CRS_0001 path:/dev/oracleasm/disks/CRSNEW

Wed Mar 31 20:34:22 2021

NOTE: Attempting voting file refresh on diskgroup CRS

NOTE: Refresh completed on diskgroup CRS

. Found 1 voting file(s).

Wed Mar 31 20:34:22 2021

GMON querying group 4 at 57 for pid 22, osid 9992

Wed Mar 31 20:34:22 2021

SUCCESS: refreshed membership for 4/0xb6a1c3c6 (CRS)

Wed Mar 31 20:34:22 2021

SUCCESS: alter diskgroup CRS add disk '/dev/oracleasm/disks/CRSNEW'

Wed Mar 31 20:34:22 2021

NOTE: starting rebalance of group 4/0xb6a1c3c6 (CRS) at power 1

Starting background process ARB0

Wed Mar 31 20:34:22 2021

ARB0 started with pid=34, OS id=28512

NOTE: assigning ARB0 to group 4/0xb6a1c3c6 (CRS) with 1 parallel I/O

Wed Mar 31 20:34:23 2021

NOTE: header on disk 0 advanced to format #2 using fcn 0.713

NOTE: header on disk 1 advanced to format #2 using fcn 0.9951

Wed Mar 31 20:34:29 2021

NOTE: Attempting voting file refresh on diskgroup CRS

NOTE: Refresh completed on diskgroup CRS

. Found 1 voting file(s).

Wed Mar 31 20:36:47 2021

NOTE: GroupBlock outside rolling migration privileged region

NOTE: requesting all-instance membership refresh for group=4

Wed Mar 31 20:36:47 2021

NOTE: membership refresh pending for group 4/0xb6a1c3c6 (CRS)

Wed Mar 31 20:36:47 2021

GMON querying group 4 at 58 for pid 22, osid 9992

Wed Mar 31 20:36:47 2021

SUCCESS: refreshed membership for 4/0xb6a1c3c6 (CRS)

NOTE: Attempting voting file refresh on diskgroup CRS

NOTE: Refresh completed on diskgroup CRS

. Found 1 voting file(s).

Wed Mar 31 20:38:31 2021

NOTE: stopping process ARB0

Wed Mar 31 20:38:32 2021

SUCCESS: rebalance completed for group 4/0xb6a1c3c6 (CRS)

• Drop CRS diskgroup using diskname, By using  output above

SQL> alter diskgroup CRS drop disk 'CRS_0000';

Diskgroup altered.

SQL> Select operation, state, est_work, est_minutes from v$asm_operation;

OPERA STAT   EST_WORK EST_MINUTES

----- ---- ---------- -----------

REBAL RUN        2220           1

REBAL WAIT          0           0

SQL> set lines 999;

col diskgroup for a10

SQL> SQL> col diskname for a12

col path for a30

select a.name DiskGroup,b.name DiskName, b.total_mb, (b.total_mb-b.free_mb) Used_MB, b.free_mb,b.path,

b.header_status

from v$asm_disk b, v$asm_diskgroup a

where a.group_number (+) =b.group_number

order by b.group_number,b.name;SQL> SQL>   2    3    4    5

DISKGROUP  DISKNAME       TOTAL_MB    USED_MB    FREE_MB PATH                           HEADER_STATU

---------- ------------ ---------- ---------- ---------- ------------------------------ ------------

                                 0          0          0 /dev/oracleasm/disks/BACKUPNEW FORMER

BACKUP     BACKUP_0000       12284         96      12188 /dev/oracleasm/disks/BACKUP    MEMBER

CRS        CRS_0000          12284       1756      10528 /dev/oracleasm/disks/CRS       MEMBER

CRS        CRS_0001          12284       2719       9565 /dev/oracleasm/disks/CRSNEW    MEMBER

DATA       DATA_0001         20473       2291      18182 /dev/oracleasm/disks/DATANEW   MEMBER

FRA        FRA_0001          20473        606      19867 /dev/oracleasm/disks/FRANEW    MEMBER

6 rows selected.

• Check status to complete rebalance operation

SQL> Select operation, state, est_work, est_minutes from v$asm_operation;

OPERA STAT   EST_WORK EST_MINUTES

----- ---- ---------- -----------

REBAL RUN        2220           1

REBAL WAIT          0           0

SQL> /

OPERA STAT   EST_WORK EST_MINUTES

----- ---- ---------- -----------

REBAL RUN        2220           0

REBAL WAIT          0           0

SQL> /

no rows selected

SQL> Select operation, state, est_work, est_minutes from v$asm_operation;

no rows selected

SQL>

• ASM alert log on local node

[root@srv2 trace]# tail -1000f alert_+ASM2.log

SQL> alter diskgroup CRS drop disk 'CRS_0000'

Wed Mar 31 20:43:20 2021

NOTE: GroupBlock outside rolling migration privileged region

NOTE: requesting all-instance membership refresh for group=4

Wed Mar 31 20:43:20 2021

NOTE: Attempting voting file relocation on diskgroup CRS

NOTE: Successful voting file relocation on diskgroup CRS

NOTE: membership refresh pending for group 4/0xb6a1c3c6 (CRS)

Wed Mar 31 20:43:20 2021

GMON querying group 4 at 59 for pid 22, osid 9992

Wed Mar 31 20:43:20 2021

SUCCESS: refreshed membership for 4/0xb6a1c3c6 (CRS)

Wed Mar 31 20:43:20 2021

SUCCESS: alter diskgroup CRS drop disk 'CRS_0000'

Wed Mar 31 20:43:21 2021

NOTE: starting rebalance of group 4/0xb6a1c3c6 (CRS) at power 1

Starting background process ARB0

Wed Mar 31 20:43:21 2021

ARB0 started with pid=34, OS id=31791

NOTE: assigning ARB0 to group 4/0xb6a1c3c6 (CRS) with 1 parallel I/O

Wed Mar 31 20:43:22 2021

NOTE: F1X0 on disk 0 (fmt 2) relocated at fcn 0.19937: AU 10 -> AU 0

NOTE: F1X0 on disk 1 (fmt 2) relocated at fcn 0.19937: AU 0 -> AU 2220

NOTE: 03/31/21 20:43:22 CRS.F1X0 copy 1 relocating from 0:10 to 1:2220 at FCN 0.19937

Wed Mar 31 20:43:27 2021

NOTE: Attempting voting file refresh on diskgroup CRS

NOTE: Refresh completed on diskgroup CRS

. Found 1 voting file(s).

Wed Mar 31 20:45:28 2021

NOTE: stopping process ARB0

Wed Mar 31 20:45:28 2021

NOTE: GroupBlock outside rolling migration privileged region

NOTE: requesting all-instance membership refresh for group=4

Wed Mar 31 20:45:28 2021

SUCCESS: rebalance completed for group 4/0xb6a1c3c6 (CRS)

Wed Mar 31 20:45:28 2021

GMON updating for reconfiguration, group 4 at 60 for pid 36, osid 32335

Wed Mar 31 20:45:28 2021

NOTE: group 4 PST updated.

SUCCESS: grp 4 disk CRS_0000 emptied

NOTE: erasing header (replicated) on grp 4 disk CRS_0000

NOTE: erasing header on grp 4 disk CRS_0000

NOTE: process _x000_+asm2 (32335) initiating offline of disk 0.3940627262 (CRS_0000) with mask 0x7e in group 4 (CRS) without client assisting

NOTE: initiating PST update: grp 4 (CRS), dsk = 0/0xeae1333e, mask = 0x6a, op = clear

Wed Mar 31 20:45:28 2021

GMON updating disk modes for group 4 at 61 for pid 36, osid 32335

NOTE: group CRS: updated PST location: disk 0001 (PST copy 0)

Wed Mar 31 20:45:28 2021

NOTE: PST update grp = 4 completed successfully

NOTE: initiating PST update: grp 4 (CRS), dsk = 0/0xeae1333e, mask = 0x7e, op = clear

Wed Mar 31 20:45:28 2021

GMON updating disk modes for group 4 at 62 for pid 36, osid 32335

Wed Mar 31 20:45:28 2021

NOTE: cache closing disk 0 of grp 4: CRS_0000

Wed Mar 31 20:45:28 2021

NOTE: PST update grp = 4 completed successfully

Wed Mar 31 20:45:28 2021

GMON updating for reconfiguration, group 4 at 63 for pid 36, osid 32335

Wed Mar 31 20:45:28 2021

NOTE: cache closing disk 0 of grp 4: (not open) CRS_0000

Wed Mar 31 20:45:28 2021

NOTE: group 4 PST updated.

Wed Mar 31 20:45:29 2021

NOTE: membership refresh pending for group 4/0xb6a1c3c6 (CRS)

NOTE: Attempting voting file refresh on diskgroup CRS

NOTE: Refresh completed on diskgroup CRS

. Found 1 voting file(s).

NOTE: Voting file relocation is required in diskgroup CRS

Wed Mar 31 20:45:29 2021

GMON querying group 4 at 64 for pid 22, osid 9992

GMON querying group 4 at 65 for pid 22, osid 9992

Wed Mar 31 20:45:29 2021

NOTE: Disk CRS_0000 in mode 0x0 marked for de-assignment

SUCCESS: refreshed membership for 4/0xb6a1c3c6 (CRS)

NOTE: Attempting voting file refresh on diskgroup CRS

NOTE: Refresh completed on diskgroup CRS

. Found 1 voting file(s).

NOTE: Voting file relocation is required in diskgroup CRS

NOTE: Attempting voting file relocation on diskgroup CRS

NOTE: Successful voting file relocation on diskgroup CRS

• CRS alter log details below:

[root@srv2 ~]# tail -300f /u01/app/grid/diag/crs/srv2/crs/trace/alert.log

2021-03-31 20:43:21.965 [OCSSD(9604)]CRS-1605: CSSD voting file is online: /dev/oracleasm/disks/CRSNEW; details in /u01/app/grid/diag/crs/srv2/crs/trace/ocssd.trc.

2021-03-31 20:43:21.965 [OCSSD(9604)]CRS-1604: CSSD voting file is offline: /dev/oracleasm/disks/CRS; details at (:CSSNM00069:) in /u01/app/grid/diag/crs/srv2/crs/trace/ocssd.trc.

2021-03-31 20:43:21.965 [OCSSD(9604)]CRS-1626: A Configuration change request completed successfully

2021-03-31 20:43:21.972 [OCSSD(9604)]CRS-1601: CSSD Reconfiguration complete. Active nodes are srv1 srv2 .

• login using grid user and check diskgroup user on asm instance.

SQL> set lines 999;

col diskgroup for a10

col diskname for a12

col path for a30

select a.name DiskGroup,b.name DiskName, b.total_mb, (b.total_mb-b.free_mb) Used_MB, b.free_mb,b.path,

b.header_status

from v$asm_disk b, v$asm_diskgroup a

where a.group_number (+) =b.group_number

order by b.group_number,b.name;SQL> SQL> SQL> SQL>   2    3    4    5

DISKGROUP  DISKNAME       TOTAL_MB    USED_MB    FREE_MB PATH                           HEADER_STATU

---------- ------------ ---------- ---------- ---------- ------------------------------ ------------

                                 0          0          0 /dev/oracleasm/disks/BACKUPNEW FORMER

                                 0          0          0 /dev/oracleasm/disks/CRS       FORMER

BACKUP     BACKUP_0000       12284         96      12188 /dev/oracleasm/disks/BACKUP    MEMBER

CRS        CRS_0001          12284       4472       7812 /dev/oracleasm/disks/CRSNEW    MEMBER

DATA       DATA_0001         20473       2291      18182 /dev/oracleasm/disks/DATANEW   MEMBER

FRA        FRA_0001          20473        606      19867 /dev/oracleasm/disks/FRANEW    MEMBER

6 rows selected.

SQL>

• Delete disk CRS using oracleasm utility and OS user is root

[root@srv1 ~]# oracleasm deletedisk CRS

Clearing disk header: done

Dropping disk: done

[root@srv1 ~]# oracleasm listdisks

BACKUP

BACKUPNEW

CRSNEW

DATANEW

FRANEW

[root@srv1 ~]#

• Login and check the disk status as we have not perform oracleasm scandsk so CRS showing candidate status

[grid@srv2 ~]$ sqlplus / as sysasm

SQL*Plus: Release 12.1.0.2.0 Production on Wed Mar 31 20:56:36 2021

Copyright (c) 1982, 2014, Oracle.  All rights reserved.

Connected to:

Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production

With the Real Application Clusters and Automatic Storage Management options

SQL> set lines 999;

col diskgroup for a10

SQL> col diskname for a12

col path for a30

select a.name DiskGroup,b.name DiskName, b.total_mb, (b.total_mb-b.free_mb) Used_MB, b.free_mb,b.path,

b.header_status

from v$asm_disk b, v$asm_diskgroup a

where a.group_number (+) =b.group_number

order by b.group_number,b.name;SQL> SQL> SQL>   2    3    4    5

DISKGROUP  DISKNAME       TOTAL_MB    USED_MB    FREE_MB PATH                           HEADER_STATU

---------- ------------ ---------- ---------- ---------- ------------------------------ ------------

                                 0          0          0 /dev/oracleasm/disks/BACKUPNEW FORMER

                                 0          0          0 /dev/oracleasm/disks/CRS       CANDIDATE

BACKUP     BACKUP_0000       12284         96      12188 /dev/oracleasm/disks/BACKUP    MEMBER

CRS        CRS_0001          12284       4472       7812 /dev/oracleasm/disks/CRSNEW    MEMBER

DATA       DATA_0001         20473       2291      18182 /dev/oracleasm/disks/DATANEW   MEMBER

FRA        FRA_0001          20473        606      19867 /dev/oracleasm/disks/FRANEW    MEMBER

6 rows selected.

SQL> exit

Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production

With the Real Application Clusters and Automatic Storage Management options

• Oracleasm listdisks and scandisk status

[grid@srv2 ~]$ su root

Password:

[root@srv2 grid]# oracleasm listdisks

BACKUP

BACKUPNEW

CRS

CRSNEW

DATANEW

FRANEW

[root@srv2 grid]# oracleasmscandisks

bash: oracleasmscandisks: command not found

[root@srv2 grid]# oracleasm scandisks

Reloading disk partitions: done

Cleaning any stale ASM disks...

Cleaning disk "CRS"

Scanning system for ASM disks...

[root@srv2 grid]# oracleasm listdisks

BACKUP

BACKUPNEW

CRSNEW

DATANEW

FRANEW

[root@srv2 grid]#

ASM - Online ASM disk migration 12.1.0.2.0 on 2 node RAC

 ASM - Online ASM disk migration 12.1.0.2.0 on 2 node RAC

• Login grid user to particular node and check current disk status

SQL> select path from v$asm_disk;

PATH

------------------------------

/dev/oracleasm/disks/CRSNEW

/dev/oracleasm/disks/DATANEW

/dev/oracleasm/disks/BACKUPNEW

/dev/oracleasm/disks/FRANEW

/dev/oracleasm/disks/FRA

/dev/oracleasm/disks/DATA

/dev/oracleasm/disks/CRS

/dev/oracleasm/disks/BACKUP

8 rows selected.

SQL> set pages 40000 lines 120

col PATH for a30

select DISK_NUMBER,MOUNT_STATUS,HEADER_STATUS,MODE_STATUS,STATE,

PATH FROM V$ASM_DISK;SQL> SQL>   2

DISK_NUMBER MOUNT_S HEADER_STATU MODE_ST STATE    PATH

----------- ------- ------------ ------- -------- ------------------------------

          0 CLOSED  FORMER       ONLINE  NORMAL   /dev/oracleasm/disks/CRSNEW

          1 CLOSED  FORMER       ONLINE  NORMAL   /dev/oracleasm/disks/DATANEW

          2 CLOSED  FORMER       ONLINE  NORMAL   /dev/oracleasm/disks/BACKUPNEW

          3 CLOSED  FORMER       ONLINE  NORMAL   /dev/oracleasm/disks/FRANEW

          0 CACHED  MEMBER       ONLINE  NORMAL   /dev/oracleasm/disks/FRA

          0 CACHED  MEMBER       ONLINE  NORMAL   /dev/oracleasm/disks/DATA

          0 CACHED  MEMBER       ONLINE  NORMAL   /dev/oracleasm/disks/CRS

          0 CACHED  MEMBER       ONLINE  NORMAL   /dev/oracleasm/disks/BACKUP

8 rows selected.

• Add new diskgroup DATA  and  check rebalance status. By default asm power limit is 1

SQL> alter diskgroup DATA add disk '/dev/oracleasm/disks/DATANEW';

Diskgroup altered.

SQL> Select operation, state, est_work, est_minutes from v$asm_operation;

OPERA STAT   EST_WORK EST_MINUTES

----- ---- ---------- -----------

REBAL RUN        1137           0

REBAL WAIT          0           0

SQL> Select operation, state, est_work, est_minutes from v$asm_operation;

OPERA STAT   EST_WORK EST_MINUTES

----- ---- ---------- -----------

REBAL RUN        1137           0

REBAL WAIT          0           0

SQL> /

OPERA STAT   EST_WORK EST_MINUTES

----- ---- ---------- -----------

REBAL RUN        1137           0

REBAL WAIT          0           0

SQL> /

OPERA STAT   EST_WORK EST_MINUTES

----- ---- ---------- -----------

REBAL RUN        1137           0

REBAL WAIT          0           0

SQL> /

OPERA STAT   EST_WORK EST_MINUTES

----- ---- ---------- -----------

REBAL DONE       1137           0

REBAL RUN           0           0

SQL> /

no rows selected

SQL>

• Check asm alert log details :

[root@srv2 trace]# tail -1000f alert_+ASM2.log

Wed Mar 31 16:51:02 2021

SQL> alter diskgroup DATA add disk '/dev/oracleasm/disks/DATANEW'

Wed Mar 31 16:51:02 2021

NOTE: GroupBlock outside rolling migration privileged region

NOTE: Assigning number (6,1) to disk (/dev/oracleasm/disks/DATANEW)

NOTE: requesting all-instance membership refresh for group=6

NOTE: Disk 1 in group 6 is assigned fgnum=2

NOTE: discarding redo for group 6 disk 1

NOTE: initializing header (replicated) on grp 6 disk DATA_0001

NOTE: initializing header on grp 6 disk DATA_0001

NOTE: requesting all-instance disk validation for group=6

Wed Mar 31 16:51:03 2021

NOTE: skipping rediscovery for group 6/0xb6b1c3c8 (DATA) on local instance.

Wed Mar 31 16:51:03 2021

NOTE: requesting all-instance disk validation for group=6

Wed Mar 31 16:51:03 2021

NOTE: skipping rediscovery for group 6/0xb6b1c3c8 (DATA) on local instance.

Wed Mar 31 16:51:03 2021

GMON updating for reconfiguration, group 6 at 33 for pid 30, osid 23425

Wed Mar 31 16:51:03 2021

NOTE: group 6 PST updated.

Wed Mar 31 16:51:03 2021

NOTE: membership refresh pending for group 6/0xb6b1c3c8 (DATA)

Wed Mar 31 16:51:03 2021

GMON querying group 6 at 34 for pid 22, osid 9992

NOTE: cache opening disk 1 of grp 6: DATA_0001 path:/dev/oracleasm/disks/DATANEW

Wed Mar 31 16:51:04 2021

NOTE: Attempting voting file refresh on diskgroup DATA

NOTE: Refresh completed on diskgroup DATA. No voting file found.

Wed Mar 31 16:51:04 2021

GMON querying group 6 at 35 for pid 22, osid 9992

Wed Mar 31 16:51:04 2021

SUCCESS: refreshed membership for 6/0xb6b1c3c8 (DATA)

Wed Mar 31 16:51:04 2021

SUCCESS: alter diskgroup DATA add disk '/dev/oracleasm/disks/DATANEW'

Wed Mar 31 16:51:04 2021

NOTE: starting rebalance of group 6/0xb6b1c3c8 (DATA) at power 1

Starting background process ARB0

Wed Mar 31 16:51:04 2021

ARB0 started with pid=34, OS id=11500

NOTE: assigning ARB0 to group 6/0xb6b1c3c8 (DATA) with 1 parallel I/O

Wed Mar 31 16:51:05 2021

NOTE: header on disk 0 advanced to format #2 using fcn 0.752

Wed Mar 31 16:51:10 2021

NOTE: Attempting voting file refresh on diskgroup DATA

NOTE: Refresh completed on diskgroup DATA. No voting file found.

Wed Mar 31 16:52:02 2021

NOTE: GroupBlock outside rolling migration privileged region

NOTE: requesting all-instance membership refresh for group=6

Wed Mar 31 16:52:02 2021

NOTE: membership refresh pending for group 6/0xb6b1c3c8 (DATA)

Wed Mar 31 16:52:02 2021

GMON querying group 6 at 36 for pid 22, osid 9992

Wed Mar 31 16:52:02 2021

SUCCESS: refreshed membership for 6/0xb6b1c3c8 (DATA)

NOTE: Attempting voting file refresh on diskgroup DATA

NOTE: Refresh completed on diskgroup DATA. No voting file found.

Wed Mar 31 16:52:49 2021

NOTE: stopping process ARB0

Wed Mar 31 16:52:50 2021

SUCCESS: rebalance completed for group 6/0xb6b1c3c8 (DATA)

• Login grid user and check the diskgroup user as we will delete diskgroup data

SQL> set lines 999;

col diskgroup for a10

col diskname for a12

col path for a30

select a.name DiskGroup,b.name DiskName, b.total_mb, (b.total_mb-b.free_mb) Used_MB, b.free_mb,b.path,

b.header_status

from v$asm_disk b, v$asm_diskgroup a

where a.group_number (+) =b.group_number

order by b.group_number,b.name;SQL> SQL> SQL> SQL>   2    3    4    5

DISKGROUP  DISKNAME       TOTAL_MB    USED_MB    FREE_MB PATH                           HEADER_STATU

---------- ------------ ---------- ---------- ---------- ------------------------------ ------------

                                 0          0          0 /dev/oracleasm/disks/CRSNEW    FORMER

                                 0          0          0 /dev/oracleasm/disks/BACKUPNEW FORMER

                                 0          0          0 /dev/oracleasm/disks/FRANEW    FORMER

BACKUP     BACKUP_0000       12284         96      12188 /dev/oracleasm/disks/BACKUP    MEMBER

CRS        CRS_0000          12284       4472       7812 /dev/oracleasm/disks/CRS       MEMBER

DATA       DATA_0000         20473       1144      19329 /dev/oracleasm/disks/DATA      MEMBER

DATA       DATA_0001         20473       1140      19333 /dev/oracleasm/disks/DATANEW   MEMBER

FRA        FRA_0000          20473        580      19893 /dev/oracleasm/disks/FRA       MEMBER

8 rows selected.

SQL> alter diskgroup DATA drop disk 'DATA_0000';

Diskgroup altered.

• Query to check ASM rebalance status

SQL> Select operation, state, est_work, est_minutes from v$asm_operation;

OPERA STAT   EST_WORK EST_MINUTES

----- ---- ---------- -----------

REBAL RUN        1141           0

REBAL WAIT          0           0

SQL> /

no rows selected

SQL>

• ASM alert log details below:

[root@srv2 trace]# tail -1000f /u01/app/grid/diag/asm/+asm/+ASM2/trace/alert_+ASM2.log

Wed Mar 31 17:02:30 2021

SUCCESS: alter diskgroup DATA drop disk 'DATA_0000'

Wed Mar 31 17:02:30 2021

NOTE: starting rebalance of group 6/0xb6b1c3c8 (DATA) at power 1

Starting background process ARB0

Wed Mar 31 17:02:30 2021

ARB0 started with pid=34, OS id=15571

NOTE: assigning ARB0 to group 6/0xb6b1c3c8 (DATA) with 1 parallel I/O

Wed Mar 31 17:02:30 2021

NOTE: F1X0 on disk 0 (fmt 2) relocated at fcn 0.11628: AU 10 -> AU 0

NOTE: header on disk 1 advanced to format #2 using fcn 0.0

NOTE: F1X0 on disk 1 (fmt 2) relocated at fcn 0.11628: AU 0 -> AU 1140

NOTE: 03/31/21 17:02:30 DATA.F1X0 copy 1 relocating from 0:10 to 1:1140 at FCN 0.11628

Wed Mar 31 17:02:36 2021

NOTE: Attempting voting file refresh on diskgroup DATA

NOTE: Refresh completed on diskgroup DATA. No voting file found.

Wed Mar 31 17:03:25 2021

NOTE: stopping process ARB0

Wed Mar 31 17:03:25 2021

NOTE: GroupBlock outside rolling migration privileged region

NOTE: requesting all-instance membership refresh for group=6

Wed Mar 31 17:03:25 2021

SUCCESS: rebalance completed for group 6/0xb6b1c3c8 (DATA)

Wed Mar 31 17:03:25 2021

GMON updating for reconfiguration, group 6 at 38 for pid 36, osid 15785

Wed Mar 31 17:03:25 2021

NOTE: group 6 PST updated.

SUCCESS: grp 6 disk DATA_0000 emptied

NOTE: erasing header (replicated) on grp 6 disk DATA_0000

NOTE: erasing header on grp 6 disk DATA_0000

NOTE: process _x000_+asm2 (15785) initiating offline of disk 0.3940627264 (DATA_0000) with mask 0x7e in group 6 (DATA) without client assisting

NOTE: initiating PST update: grp 6 (DATA), dsk = 0/0xeae13340, mask = 0x6a, op = clear

Wed Mar 31 17:03:26 2021

GMON updating disk modes for group 6 at 39 for pid 36, osid 15785

NOTE: group DATA: updated PST location: disk 0001 (PST copy 0)

Wed Mar 31 17:03:26 2021

NOTE: PST update grp = 6 completed successfully

NOTE: initiating PST update: grp 6 (DATA), dsk = 0/0xeae13340, mask = 0x7e, op = clear

Wed Mar 31 17:03:26 2021

GMON updating disk modes for group 6 at 40 for pid 36, osid 15785

Wed Mar 31 17:03:26 2021

NOTE: cache closing disk 0 of grp 6: DATA_0000

Wed Mar 31 17:03:26 2021

NOTE: PST update grp = 6 completed successfully

Wed Mar 31 17:03:26 2021

GMON updating for reconfiguration, group 6 at 41 for pid 36, osid 15785

Wed Mar 31 17:03:26 2021

NOTE: cache closing disk 0 of grp 6: (not open) DATA_0000

Wed Mar 31 17:03:26 2021

NOTE: group 6 PST updated.

Wed Mar 31 17:03:26 2021

NOTE: membership refresh pending for group 6/0xb6b1c3c8 (DATA)

Wed Mar 31 17:03:26 2021

GMON querying group 6 at 42 for pid 22, osid 9992

GMON querying group 6 at 43 for pid 22, osid 9992

Wed Mar 31 17:03:26 2021

NOTE: Disk DATA_0000 in mode 0x0 marked for de-assignment

SUCCESS: refreshed membership for 6/0xb6b1c3c8 (DATA)

NOTE: Attempting voting file refresh on diskgroup DATA

NOTE: Refresh completed on diskgroup DATA. No voting file found.

• Query to check ASM disk group status

SQL> set lines 999;

col diskgroup for a10

SQL> SQL> col diskname for a12

col path for a30

select a.name DiskGroup,b.name DiskName, b.total_mb, (b.total_mb-b.free_mb) Used_MB, b.free_mb,b.path,

b.header_status

from v$asm_disk b, v$asm_diskgroup a

where a.group_number (+) =b.group_number

order by b.group_number,b.name;SQL> SQL>   2    3    4    5

DISKGROUP  DISKNAME       TOTAL_MB    USED_MB    FREE_MB PATH                           HEADER_STATU

---------- ------------ ---------- ---------- ---------- ------------------------------ ------------

                                 0          0          0 /dev/oracleasm/disks/DATA      FORMER

                                 0          0          0 /dev/oracleasm/disks/CRSNEW    FORMER

                                 0          0          0 /dev/oracleasm/disks/BACKUPNEW FORMER

                                 0          0          0 /dev/oracleasm/disks/FRANEW    FORMER

BACKUP     BACKUP_0000       12284         96      12188 /dev/oracleasm/disks/BACKUP    MEMBER

CRS        CRS_0000          12284       4472       7812 /dev/oracleasm/disks/CRS       MEMBER

DATA       DATA_0001         20473       2281      18192 /dev/oracleasm/disks/DATANEW   MEMBER

FRA        FRA_0000          20473        580      19893 /dev/oracleasm/disks/FRA       MEMBER

8 rows selected.

SQL>

• Delete disk at OS level using root user

[root@srv2 ~]# oracleasm deletedisk DATA

Disk "DATA" defines an unmarked device

Dropping disk: done

[root@srv2 ~]# oracleasm listdisks

BACKUP

BACKUPNEW

CRS

CRSNEW

DATANEW

FRA

FRANEW

[root@srv2 ~]#

• oracleasm scandisks on remote node to check the status

[root@srv1 ~]# oracleasm scandisks

Reloading disk partitions: done

Cleaning any stale ASM disks...

Cleaning disk "DATA"

Scanning system for ASM disks...

[root@srv1 ~]# oracleasm listdisks

BACKUP

BACKUPNEW

CRS

CRSNEW

DATANEW

FRA

FRANEW

ASM drop diskgroup ORA-15032: not all alterations performed

 

Hi all we recently wants to perform ASM LUN migration activity we face some error,  Although this error not related the ASM LUN migration.  When we try to delete asm diskgroup name is NEWBACKUP using asmca we got error as well is in command prompt. Earlier in another scenario we deleted the same but do not encountered any issue. After troubleshooting  we dismount the disk group from another instance and deleted the same. 

Error:

ERROR at line 1:

ORA-15032: not all alterations performed

ORA-15250: insufficient diskgroup space for rebalance completion

ORA-15039: diskgroup not dropped

ORA-15073: diskgroup BACKUPNEW is mounted by another ASM instance

Dropping disks from disk group BACKUPNEW failed with the following message:

ORA-15032: not all alterations performed

ORA-15250: insufficient diskgroup space for rebalance completion

ASM - Add and remove asmdiskgroup in 12c

 

• Add and remove asmdiskgroup in 12c using command line

Make raw partition of disk

[root@srv1 ~]# fdisk /dev/sdf

Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel

Building a new DOS disklabel with disk identifier 0x20717bf6.

Changes will remain in memory only, until you decide to write them.

After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to

         switch off the mode (command 'c') and change display units to

         sectors (command 'u').

Command (m for help): n

Command action

   e   extended

   p   primary partition (1-4)

p

Partition number (1-4): 1 dsdsdsd                           

First cylinder (1-1566, default 1):

Using default value 1

Last cylinder, +cylinders or +size{K,M,G} (1-1566, default 1566):

Using default value 1566

Command (m for help): w

The partition table has been altered!

Calling ioctl() to re-read partition table.

Syncing disks.

[root@srv1 ~]# fdisk -l

Disk /dev/sda: 85.9 GB, 85899345920 bytes

255 heads, 63 sectors/track, 10443 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x00047a8e

   Device Boot      Start         End      Blocks   Id  System

/dev/sda1   *           1          77      614400   83  Linux

Partition 1 does not end on cylinder boundary.

/dev/sda2              77         599     4194304   83  Linux

Partition 2 does not end on cylinder boundary.

/dev/sda3             599       10444    79076352   83  Linux

Disk /dev/sdb: 12.9 GB, 12884901888 bytes

255 heads, 63 sectors/track, 1566 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0xb374f6ac

   Device Boot      Start         End      Blocks   Id  System

/dev/sdb1               1        1566    12578863+  83  Linux

Disk /dev/sdc: 21.5 GB, 21474836480 bytes

255 heads, 63 sectors/track, 2610 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0xbf063e34

   Device Boot      Start         End      Blocks   Id  System

/dev/sdc1               1        2610    20964793+  83  Linux

Disk /dev/sdd: 21.5 GB, 21474836480 bytes

255 heads, 63 sectors/track, 2610 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x8af70889

   Device Boot      Start         End      Blocks   Id  System

/dev/sdd1               1        2610    20964793+  83  Linux

Disk /dev/sde: 4294 MB, 4294967296 bytes

255 heads, 63 sectors/track, 522 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0xff585e78

   Device Boot      Start         End      Blocks   Id  System

/dev/sde1               1         522     4192933+  83  Linux

Disk /dev/sdf: 12.9 GB, 12884901888 bytes

255 heads, 63 sectors/track, 1566 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x20717bf6

   Device Boot      Start         End      Blocks   Id  System

/dev/sdf1               1        1566    12578863+  83  Linux

• Create ASM disk using raw partition

[root@srv1 ~]# oracleasm createdisk backup /dev/sdf1

Writing disk header: done

Instantiating disk: done

[root@srv1 ~]# oracleasm listdisks

BACKUP

CRS

DATA

FRA

[root@srv1 ~]# su grid

[grid@srv1 root]$ cd

[grid@srv1 ~]$ . oraenv

ORACLE_SID = [+ASM1] ?

The Oracle base remains unchanged with value /u01/app/grid

[grid@srv1 ~]$ asmcmd

ASMCMD> lsdg

State    Type    Rebal  Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name

MOUNTED  EXTERN  N         512   4096  1048576     12284     7807                0            7807              0             Y  CRS/

MOUNTED  EXTERN  N         512   4096  1048576     20473    17997                0           17997              0             N  DATA/

MOUNTED  EXTERN  N         512   4096  1048576     20473    20142                0           20142              0             N  FRA/

ASMCMD> exit

[grid@srv1 ~]$

SQL> col PATH for a40

SQL> /

PATH                                     HEADER_STATU

---------------------------------------- ------------

/dev/oracleasm/disks/BACKUP              PROVISIONED

/dev/oracleasm/disks/FRA                 MEMBER

/dev/oracleasm/disks/DATA                MEMBER

/dev/oracleasm/disks/CRS                 MEMBER

SQL> sho parameter ASM_DISKSTRING

NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

asm_diskstring                       string      /dev/oracleasm/disks

• Add ASM disk group with EXTERNAL redundancy level.

SQL> CREATE DISKGROUP backup EXTERNAL REDUNDANCY DISK '/dev/oracleasm/disks/BACKUP';

Diskgroup created.

SQL> select PATH,HEADER_STATUS from v$asm_disk;

PATH                                     HEADER_STATU

---------------------------------------- ------------

/dev/oracleasm/disks/FRA                 MEMBER

/dev/oracleasm/disks/DATA                MEMBER

/dev/oracleasm/disks/CRS                 MEMBER

/dev/oracleasm/disks/BACKUP              MEMBER

[grid@srv1 ~]$ asmcmd

ASMCMD> lsdg

State    Type    Rebal  Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name

MOUNTED  EXTERN  N         512   4096  1048576     12284    12234                0           12234              0             N  BACKUP/

MOUNTED  EXTERN  N         512   4096  1048576     12284     7807                0            7807              0             Y  CRS/

MOUNTED  EXTERN  N         512   4096  1048576     20473    17997                0           17997              0             N  DATA/

MOUNTED  EXTERN  N         512   4096  1048576     20473    20142                0           20142              0             N  FRA/

ASMCMD>

SQL> set lines 200

SQL> col GROUP_NUMBER for a60

SQL> col GROUP_NUMBER for a40

SQL> col COMPATIBILITY for a40

SQL> col DATABASE_COMPATIBILITY for a40

SQL> select group_number, name, compatibility, database_compatibility from v$asm_diskgroup;

GROUP_NUMBER NAME                           COMPATIBILITY                            DATABASE_COMPATIBILITY

------------ ------------------------------ ---------------------------------------- ----------------------------------------

  ########## BACKUP                         10.1.0.0.0                               10.1.0.0.0

  ########## FRA                            12.1.0.0.0                               10.1.0.0.0

  ########## DATA                           12.1.0.0.0                               10.1.0.0.0

  ########## CRS                            12.1.0.0.0                               10.1.0.0.0

SQL> ALTER DISKGROUP BACKUP SET ATTRIBUTE 'compatible.asm' = '12.1.0.0.0';

Diskgroup altered.

SQL> select group_number, name, compatibility, database_compatibility from v$asm_diskgroup;

GROUP_NUMBER NAME                           COMPATIBILITY                            DATABASE_COMPATIBILITY

------------ ------------------------------ ---------------------------------------- ----------------------------------------

  ########## BACKUP                         12.1.0.0.0                               10.1.0.0.0

  ########## FRA                            12.1.0.0.0                               10.1.0.0.0

  ########## DATA                           12.1.0.0.0                               10.1.0.0.0

  ########## CRS                            12.1.0.0.0                               10.1.0.0.0

• Drop diskgroup backup

SQL> DROP DISKGROUP BACKUP INCLUDING CONTENTS;

Diskgroup dropped.

SQL> select  name, compatibility, database_compatibility from v$asm_diskgroup;

NAME                           COMPATIBILITY                            DATABASE_COMPATIBILITY

------------------------------ ---------------------------------------- ----------------------------------------

FRA                            12.1.0.0.0                               10.1.0.0.0

DATA                           12.1.0.0.0                               10.1.0.0.0

CRS                            12.1.0.0.0                               10.1.0.0.0

• Delete at OS level

[grid@srv1 ~]$ oracleasm deletedisk BACKUP

Disk "BACKUP" defines an unmarked device

Dropping disk: failed

Unable to delete disk "BACKUP"

[grid@srv1 ~]$ su root

Password:

[root@srv1 grid]# +ASM1

bash: +ASM1: command not found

[root@srv1 grid]# . oraenv

ORACLE_SID = [+ASM1] ?

The Oracle base remains unchanged with value /u01/app/grid

[root@srv1 grid]# oracleasm deletedisk BACKUP

Disk "BACKUP" defines an unmarked device

Dropping disk: done

[root@srv1 grid]# oracleasm listdisks

CRS

DATA

FRA

[root@srv1 grid]#