Monday 4 October 2021

standby redo log file header 19.0.0.0.0 exceeded the one currently supported on this standby 12.1.0.2.0.


Recently we got error after upgrade from 12c to 19c.  compatible parameter not set to 19c  on physical standby database , Due to this after applying MRP we got error.


Error : The compatibility version in redo log file header 19.0.0.0.0 exceeded the one currently supported on this standby 12.1.0.2.0. 



2021-09-19T17:24:47.440414+05:30

 rfs (PID:5459): Opened log for T-1.S-302 dbid 912183804 branch 1081693119

 rfs (PID:5459): The compatibility version in redo log file header 19.0.0.0.0 exceeded the one currently supported on this standby 12.1.0.2.0.

2021-09-19T17:25:31.765108+05:30

Deleted Oracle managed file /u01/app/oracle/fast_recovery_area/ORCL12CDR/archivelog/2021_09_19/o1_mf_1_302_jng987fp_.arc

2021-09-19T17:30:49.929833+05:30

Deleted Oracle managed file /u01/app/oracle/fast_recovery_area/ORCL12CDR/archivelog/2021_09_19/o1_mf_0_0_jng9mkwx_.arc

2021-09-19T17:30:50.019755+05:30

 rfs (PID:5816): Primary database is in MAXIMUM PERFORMANCE mode

 rfs (PID:5816): No SRLs available for T-1

2021-09-19T17:30:50.068208+05:30


Scenario : 


SQL> show parameter compatible


NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

compatible                           string      12.1.0.2.0

noncdb_compatible                    boolean     FALSE


  • Change compatible parameter to  '19.0.0'
  • 
    
    
    

    SQL> alter system set compatible='19.0.0' scope=spfile;


    shudown standby database 

    startup mount



    SQL>  select process, thread#, sequence#, status from v$managed_standby;


    PROCESS      THREAD#  SEQUENCE# STATUS

    --------- ---------- ---------- ------------

    ARCH               0          0 CONNECTED

    DGRD               0          0 ALLOCATED

    DGRD               0          0 ALLOCATED

    ARCH               0          0 CONNECTED

    ARCH               1        110 CLOSING

    ARCH               0          0 CONNECTED

    RFS                1        302 IDLE


    7 rows selected.



    • Apply MRP on standby database 

    
    
    

    SQL> alter database recover managed standby database disconnect from session;


    Database altered.


    SQL> SELECT ARCH.THREAD# "Thread", ARCH.SEQUENCE# "Last Sequence Received", APPL.SEQUENCE# "Last Sequence Applied", (ARCH.SEQUENCE# - APPL.SEQUENCE#) "Difference"

    FROM

    (SELECT THREAD# ,SEQUENCE# FROM V$ARCHIVED_LOG WHERE (THREAD#,FIRST_TIME ) IN (SELECT THREAD#,MAX(FIRST_TIME) FROM V$ARCHIVED_LOG GROUP BY THREAD#)) ARCH,

      2    3    4  (SELECT THREAD# ,SEQUENCE# FROM V$LOG_HISTORY WHERE (THREAD#,FIRST_TIME ) IN (SELECT THREAD#,MAX(FIRST_TIME) FROM V$LOG_HISTORY GROUP BY THREAD#)) APPL

      5  WHERE

      6  ARCH.THREAD# = APPL.THREAD#

      7  ;


        Thread Last Sequence Received Last Sequence Applied Difference

    ---------- ---------------------- --------------------- ----------

             1                    196                   110         86


    SQL> /


        Thread Last Sequence Received Last Sequence Applied Difference

    ---------- ---------------------- --------------------- ----------

             1                    301                   199        102


    SQL> /


        Thread Last Sequence Received Last Sequence Applied Difference

    ---------- ---------------------- --------------------- ----------

             1                    302                   302          0


    SQL>

    SQL> select process, thread#, sequence#, status from v$managed_standby;


    PROCESS      THREAD#  SEQUENCE# STATUS

    --------- ---------- ---------- ------------

    ARCH               1        298 CLOSING

    DGRD               0          0 ALLOCATED

    DGRD               0          0 ALLOCATED

    ARCH               0          0 CONNECTED

    ARCH               1        110 CLOSING

    ARCH               1        301 CLOSING

    MRP0               1        303 APPLYING_LOG

    RFS                1        303 IDLE

    RFS                1          0 IDLE

    RFS                0          0 IDLE

    RFS                0          0 IDLE


    PROCESS      THREAD#  SEQUENCE# STATUS

    --------- ---------- ---------- ------------

    RFS                0          0 IDLE


    12 rows selected.


    SQL>


    Dataguard: physical standby database restoration using full rman backup


    Scenario Preview: Resolving gap between primary and physical standby database  resolved  by taking incremental backup  and perform recovery using rman incremental backup.

    • Query to check current SCN status of standby database. Run on standby DB.

    SQL> SELECT CURRENT_SCN FROM V$DATABASE;


    CURRENT_SCN

    -----------

      339341431


    SQL>  select min(fhscn) from x$kcvfh;


    MIN(FHSCN)

    ----------------

    339341432


    SQL>  select min(f.fhscn) from x$kcvfh f, v$datafile d where f.hxfil =d.file# and d.enabled != 'READ ONLY';


    MIN(F.FHSCN)

    ----------------

    339341432

    • SCN  based  rman incremental backup  command below for reference.


    run

    {

    allocate channel d0 type disk FORMAT 'backupscn_based%d_%s_%t.bak';

    allocate channel d1 type disk FORMAT 'backupscn_based%d_%s_%t.bak';

    allocate channel d2 type disk FORMAT 'backupscn_based%d_%s_%t.bak';

    allocate channel d3 type disk FORMAT 'backupscn_based%d_%s_%t.bak';

    backup as compressed backupset incremental from scn 339341432 database;

    copy current controlfile to 'backupscn_basedcontrol01full_%d_%t_%U.bak';

    backup current controlfile for standby format 'backupscn_basedstandbycontrol_%d_%T_%U.ctl';

    RELEASE CHANNEL d0;

    RELEASE CHANNEL d1;

    RELEASE CHANNEL d2;

    RELEASE CHANNEL d3;

    }


    • Open database in nomount stage and restore control file 

    [oracle@srv4 backup]$ . oraenv

    ORACLE_SID = [ZABBIXDR] ?

    The Oracle base remains unchanged with value /u01/app/oracle

    [oracle@srv4 backup]$ sqlplus / as sysdba


    SQL*Plus: Release 12.1.0.2.0 Production on Tue Jul 27 18:37:36 2021


    Copyright (c) 1982, 2014, Oracle.  All rights reserved.



    Connected to:

    Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production

    With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options


    SQL> shut immediate;

    ORA-01109: database not open



    Database dismounted.

    ORACLE instance shut down.

    SQL> startup nomount;

    ORACLE instance started.


    Total System Global Area 1795162112 bytes

    Fixed Size                  2925456 bytes

    Variable Size             620760176 bytes

    Database Buffers         1157627904 bytes

    Redo Buffers               13848576 bytes

    SQL> exit


    Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production

    With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

    [oracle@srv4 backup]$ rman target /


    Recovery Manager: Release 12.1.0.2.0 - Production on Tue Jul 27 18:40:44 2021


    Copyright (c) 1982, 2014, Oracle and/or its affiliates.  All rights reserved.


    connected to target database: ZABBIX (not mounted)


    RMAN> restore standby controlfile from '/u01/backup/standbycontrol_ZABBIX_1d051c76_1_1.ctl';

    Starting restore at 27-JUL-21

    using target database control file instead of recovery catalog

    allocated channel: ORA_DISK_1

    channel ORA_DISK_1: SID=23 device type=DISK

    channel ORA_DISK_1: restoring control file

    channel ORA_DISK_1: restore complete, elapsed time: 00:00:01

    output file name=/u01/app/oracle/oradata/ZABBIXDR/control01.ctl

    output file name=/u01/app/oracle/fast_recovery_area/ZABBIXDR/control02.ctl

    Finished restore at 27-JUL-21

    • Open database in mount stage and restore  standby database 

    RMAN> sql 'alter database mount standby database';


    sql statement: alter database mount standby database

    released channel: ORA_DISK_1


    RMAN> catalog start with '/u01/backup';


    Starting implicit crosscheck backup at 27-JUL-21

    allocated channel: ORA_DISK_1

    channel ORA_DISK_1: SID=24 device type=DISK

    Crosschecked 13 objects

    Finished implicit crosscheck backup at 27-JUL-21


    Starting implicit crosscheck copy at 27-JUL-21

    using channel ORA_DISK_1

    Crosschecked 6 objects

    Finished implicit crosscheck copy at 27-JUL-21


    searching for all files in the recovery area

    cataloging files...

    cataloging done


    List of Cataloged Files

    =======================

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/autobackup/2021_07_25/o1_mf_s_1078857835_jhts676p_.bkp

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/autobackup/2021_07_23/o1_mf_s_1078691869_jhopt5fk_.bkp

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/autobackup/2021_07_19/o1_mf_s_1078341966_jhc13pjr_.bkp

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_17/o1_mf_1_51_jh5txgn3_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_17/o1_mf_1_52_jh5w956q_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_27/o1_mf_1_33_jhz922g1_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_27/o1_mf_1_34_jhz923jb_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_27/o1_mf_1_32_jhz87wwc_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_27/o1_mf_1_35_jhzcszvk_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_27/o1_mf_1_31_jhz87yd4_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_26/o1_mf_1_29_jhxdgqjh_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_26/o1_mf_1_28_jhx8b6c8_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_26/o1_mf_1_26_jhx752c2_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_26/o1_mf_1_27_jhx754fs_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_26/o1_mf_1_25_jhx750kr_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_26/o1_mf_1_24_jhwmbsk6_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_26/o1_mf_1_30_jhxfy2os_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_25/o1_mf_1_21_jhtrwp8q_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_25/o1_mf_1_10_jhtr7176_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_25/o1_mf_1_11_jhtr72bt_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_25/o1_mf_1_12_jhtr73o7_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_25/o1_mf_1_23_jhts9kwh_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_25/o1_mf_1_9_jhtqv96b_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_25/o1_mf_1_15_jhtrmltg_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_25/o1_mf_1_17_jhtrnorq_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_25/o1_mf_1_19_jhtrwhvo_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_25/o1_mf_1_8_jhtqv8w7_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_25/o1_mf_1_16_jhtrmmh5_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_25/o1_mf_1_18_jhtrpgbr_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_25/o1_mf_1_13_jhtrmj20_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_25/o1_mf_1_20_jhtrwmnk_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_25/o1_mf_1_6_jhtpbq66_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_25/o1_mf_1_14_jhtrmlrt_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_25/o1_mf_1_22_jhts9jpq_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_25/o1_mf_1_7_jhtqjvr0_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_08/o1_mf_1_49_jgg446oc_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_23/o1_mf_1_2_jhokqtln_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_23/o1_mf_1_3_jhop0w97_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_23/o1_mf_1_4_jhop0wt0_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_23/o1_mf_1_5_jhopsvv9_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_19/o1_mf_1_54_jhbqjq1l_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_19/o1_mf_1_64_jhc0njy1_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_19/o1_mf_1_58_jhbw0yf3_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_19/o1_mf_1_61_jhbzy9yf_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_19/o1_mf_1_57_jhbw02oc_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_19/o1_mf_1_63_jhbzyc4y_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_19/o1_mf_1_53_jhbqjqf9_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_19/o1_mf_1_56_jhbvzxjs_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_19/o1_mf_1_60_jhbzy6vz_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_19/o1_mf_1_55_jhbr7mg0_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_19/o1_mf_1_62_jhbzy9xr_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_19/o1_mf_1_59_jhbw495f_.arc

    File Name: /u01/app/oracle/fast_recovery_area/ZABBIXDR/archivelog/2021_07_19/o1_mf_1_1_jhc0tnhq_.arc


    searching for all files that match the pattern /u01/backup


    List of Files Unknown to the Database

    =====================================

    File Name: /u01/backup/standbycontrol_ZABBIX_1d051c76_1_1.ctl


    Do you really want to catalog the above files (enter YES or NO)? yes

    cataloging files...

    cataloging done


    List of Cataloged Files

    =======================

    File Name: /u01/backup/standbycontrol_ZABBIX_1d051c76_1_1.ctl


    RMAN> CONFIGURE DEVICE TYPE DISK PARALLELISM 20 BACKUP TYPE TO BACKUPSET;


    new RMAN configuration parameters:

    CONFIGURE DEVICE TYPE DISK PARALLELISM 20 BACKUP TYPE TO BACKUPSET;

    new RMAN configuration parameters are successfully stored

    released channel: ORA_DISK_1


    RMAN>  RECOVER DATABASE NOREDO;


    RMAN> select process,status,sequence# from v$managed_standby;


    PROCESS   STATUS        SEQUENCE#

    --------- ------------ ----------

    ARCH      CONNECTED             0

    ARCH      CONNECTED             0

    ARCH      CONNECTED             0

    ARCH      CONNECTED             0

    • Apply Media recovery process on standby database 

    RMAN> alter database recover managed standby database disconnect from session;


    Statement processed


    RMAN> select process,status,sequence# from v$managed_standby;


    PROCESS   STATUS        SEQUENCE#

    --------- ------------ ----------

    ARCH      CONNECTED             0

    ARCH      CONNECTED             0

    ARCH      CONNECTED             0

    ARCH      CONNECTED             0

    MRP0      WAIT_FOR_LOG          7




    ORA-38880: Cannot advance compatibility from 12.1.0.2.0 to 19.0.0.0.0 due to guaranteed restore points


    We recently done manual upgradation from 12c to 19c  after post upgrade  after changing compatible parameter to 19c we still  got error. This is due to restore point was not dropped before pre upgrade. so we shut reverted back compatible parameter to 12c and delete restore point  then change it back to 19c compatible parameter and startup the database server. 

    Error

    ORA-38880: Cannot advance compatibility from 12.1.0.2.0 to 19.0.0.0.0 due to guaranteed restore points


     Scenario: 


      
     

    SQL> shut immediate;

    Database closed.

    Database dismounted.

    ORACLE instance shut down.

    SQL> startup

    ORACLE instance started.


    Total System Global Area 1862269648 bytes

    Fixed Size                  9136848 bytes

    Variable Size             637534208 bytes

    Database Buffers         1207959552 bytes

    Redo Buffers                7639040 bytes

    ORA-38880: Cannot advance compatibility from 12.1.0.2.0 to 19.0.0.0.0 due to guaranteed restore points


    SQL> show parameter compatible;


    NAME                                 TYPE                              VALUE

    ------------------------------------ --------------------------------- ------------------------------

    compatible                           string                            19.0.0

    noncdb_compatible                    boolean                           FALSE

    SQL>  select name from v$restore_point;

     select name from v$restore_point

                      *

    ERROR at line 1:

    ORA-01507: database not mounted


    SQL> shutdown immediate;

    ORA-01507: database not mounted


    ORACLE instance shut down.

    SQL> startup force;

    ORACLE instance started.


    Total System Global Area 1862269648 bytes

    Fixed Size                  9136848 bytes

    Variable Size             637534208 bytes

    Database Buffers         1207959552 bytes

    Redo Buffers                7639040 bytes

    ORA-38880: Cannot advance compatibility from 12.1.0.2.0 to 19.0.0.0.0 due to guaranteed restore points


  • Change  compatible parameter to 12.1.0.2.0 then check restore point and delete it.
  • SQL> alter system set compatible="12.1.0.2.0" scope=spfile;


    System altered.


    SQL> startup force;

    ORACLE instance started.


    Total System Global Area 1862269648 bytes

    Fixed Size                  9136848 bytes

    Variable Size             637534208 bytes

    Database Buffers         1207959552 bytes

    Redo Buffers                7639040 bytes

    Database mounted.

    Database opened.


    SQL> select GUARANTEE_FLASHBACK_DATABASE,NAME from v$restore_point;


    GUA NAME

    --- --------------------------------------------------------------------------------------------------------------------------------

    YES BEFORE_UPGRADE_19C


    1 row selected.


    SQL> select flashback_On from v$database;


    FLASHBACK_ON

    ------------------

    RESTORE POINT ONLY


    1 row selected.


    SQL> alter database flashback off;


    Database altered.


    SQL> select flashback_On from v$database;


    FLASHBACK_ON

    ------------------

    RESTORE POINT ONLY


    1 row selected.


    SQL> drop restore point BEFORE_UPGRADE_19C;


    Restore point dropped.


    SQL> select flashback_On from v$database;


    FLASHBACK_ON

    ------------------

    NO


    1 row selected.


    SQL> show parameter compatible


    NAME                                 TYPE        VALUE

    ------------------------------------ ----------- ------------------------------

    compatible                           string      12.1.0.2.0

    noncdb_compatible                    boolean     FALSE


    SQL>  alter system set compatible='19.0.0' scope=spfile;


    System altered.


    SQL> shut immediate;

    Database closed.

    Database dismounted.

    ORACLE instance shut down.

    SQL> startup

    ORACLE instance started.


    Total System Global Area 1862269648 bytes

    Fixed Size                  9136848 bytes

    Variable Size             637534208 bytes

    Database Buffers         1207959552 bytes

    Redo Buffers                7639040 bytes

    Database mounted.


    Database opened.

    SQL> SQL> sho parameter compatible


    NAME                                 TYPE        VALUE

    ------------------------------------ ----------- ------------------------------

    compatible                           string      19.0.0

    noncdb_compatible                    boolean     FALSE

    SQL>