Friday 28 February 2020

RMAN Error RMAN-06026:RMAN-06023


We found rman error RMAN-06026:RMAN-06023  during restoration of physical standby  in  RMAN prompt. This is generally due to incarnation point change, we just disable FRA dest and size in pfile and start db with pfile and perform restoration again.


--- Startup sid in no mount state ---


oracle@8:/u01/rman_smrrac> . oraenv

ORACLE_SID = [smrrac] ?

The Oracle base remains unchanged with value /u01/app/oracle

oracle@8:/u01/rman_smrrac> sqlplus / as sysdba



SQL*Plus: Release 12.1.0.2.0 Production on Wed Feb 26 13:41:22 2020



Copyright (c) 1982, 2014, Oracle.  All rights reserved.



Connected to an idle instance.



SQL> startup nomount;

ORACLE instance started.



Total System Global Area 3942645760 bytes

Fixed Size                  2931616 bytes

Variable Size             939525216 bytes

Database Buffers         2986344448 bytes

Redo Buffers               13844480 bytes

SQL> exit

Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production

With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options


--- Restore standby controlfile ---


oracle@8:/u01/rman_smrrac> rman target /



Recovery Manager: Release 12.1.0.2.0 - Production on Wed Feb 26 13:41:40 2020



Copyright (c) 1982, 2014, Oracle and/or its affiliates.  All rights reserved.



connected to target database: SMRRAC (not mounted)



RMAN> restore standby controlfile from '/u01/rman_smrrac/standbycontrol_SMRRAC_32upfn2e_1_1.ctl';



Starting restore at 26-FEB-2020 13:42:19

using target database control file instead of recovery catalog

allocated channel: ORA_DISK_1

channel ORA_DISK_1: SID=527 device type=DISK



channel ORA_DISK_1: restoring control file

channel ORA_DISK_1: restore complete, elapsed time: 00:00:01

output file name=/u01/app/oracle/oradata/smrrac/control01.ctl

output file name=/u01/app/oracle/fast_recovery_area/smrrac/control02.ctl

Finished restore at 26-FEB-2020 13:42:20

--- Open standby database in mount stage ---

RMAN> alter database mount;



Statement processed

released channel: ORA_DISK_1

--- Catalog standby database ---

RMAN> catalog start with '/u01/rman_smrrac/';



Starting implicit crosscheck backup at 26-FEB-2020 13:43:05

allocated channel: ORA_DISK_1

channel ORA_DISK_1: SID=357 device type=DISK

Crosschecked 34 objects

Finished implicit crosscheck backup at 26-FEB-2020 13:43:06



Starting implicit crosscheck copy at 26-FEB-2020 13:43:06

using channel ORA_DISK_1

Crosschecked 4 objects

Finished implicit crosscheck copy at 26-FEB-2020 13:43:06



searching for all files in the recovery area

cataloging files...

cataloging done



List of Cataloged Files

=======================

File Name: /u01/app/oracle/fast_recovery_area/SMRRAC/archivelog/2020_02_26/o1_mf_2_1_h5drk9hq_.arc



searching for all files that match the pattern /u01/rman_smrrac/



List of Files Unknown to the Database

=====================================

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2nupfmsr_2135_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2mupfmsr_2134_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2kupfmsr_2132_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2lupfmsr_2133_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2oupfmtk_2136_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2rupfmu4_2139_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2tupfmu7_2141_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2uupfmu9_2142_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2qupfmu4_2138_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2supfmu4_2140_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2pupfmu4_2137_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2vupfn2a_2143_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_30upfn2a_2144_1_20200226

File Name: /u01/rman_smrrac/control_SMRRAC_20200226.bak

File Name: /u01/rman_smrrac/standbycontrol_SMRRAC_32upfn2e_1_1.ctl



Do you really want to catalog the above files (enter YES or NO)? yes

cataloging files...

cataloging done



List of Cataloged Files

=======================

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2nupfmsr_2135_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2mupfmsr_2134_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2kupfmsr_2132_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2lupfmsr_2133_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2oupfmtk_2136_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2rupfmu4_2139_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2tupfmu7_2141_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2uupfmu9_2142_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2qupfmu4_2138_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2supfmu4_2140_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2pupfmu4_2137_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2vupfn2a_2143_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_30upfn2a_2144_1_20200226

File Name: /u01/rman_smrrac/control_SMRRAC_20200226.bak

File Name: /u01/rman_smrrac/standbycontrol_SMRRAC_32upfn2e_1_1.ctl


--- Change datafile,undo file, tempfile  location  and restore database from backup --- 

RMAN> run



{

set newname for datafile 1 to "/u01/app/oracle/oradata/smrrac/system01.dbf";

2> 3> 4>

set newname for datafile 2 to "/u02/oradata/smrrac/idata01.dbf";



set newname for datafile 3 to "/u01/app/oracle/oradata/smrrac/sysaux01.dbf";



5> 6> 7> 8> 9> 10> set newname for datafile 4 to "/u01/app/oracle/oradata/smrrac/undotbs01.dbf";



set newname for datafile 5 to "/u01/app/oracle/oradata/smrrac/undotbs02.dbf";



set newname for datafile 6 to "/u02/oradata/smrrac/users01.dbf";



set newname for datafile 7 to "/u02/oradata/smrrac/ilog01.dbf";



set newname for datafile 8 to "/u02/oradata/smrrac/indx01.dbf";



set newname for tempfile 1 to "/u01/app/oracle/oradata/smrrac/temp01.dbf";



RESTORE DATABASE;



SWITCH DATAFILE ALL;



SWITCH TEMPFILE ALL;



}11> 12> 13> 14> 15> 16> 17> 18> 19> 20> 21> 22> 23> 24> 25> 26> 27> 28>



executing command: SET NEWNAME



executing command: SET NEWNAME



executing command: SET NEWNAME



executing command: SET NEWNAME



executing command: SET NEWNAME



executing command: SET NEWNAME



executing command: SET NEWNAME



executing command: SET NEWNAME



executing command: SET NEWNAME



Starting restore at 26-FEB-2020 13:43:44

using channel ORA_DISK_1



RMAN-00571: ===========================================================

RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============

RMAN-00571: ===========================================================

RMAN-03002: failure of restore command at 02/26/2020 13:43:44

RMAN-06026: some targets not found - aborting restore

RMAN-06023: no backup or copy of datafile 6 found to restore

RMAN-06023: no backup or copy of datafile 4 found to restore

RMAN-06023: no backup or copy of datafile 3 found to restore

RMAN-06023: no backup or copy of datafile 1 found to restore


RMAN>


--- We found rman error RMAN-06026, RMAN-06023 , Due to incarnation point issue ---

So we disable db_recovery_file_dest and db_recovery_file_dest_size  and start stndby with pfile  initsmrrac.ora


oracle@8:/u01/app/oracle/product/12.1.0/dbhome_1/dbs> cat initsmrrac.ora
smrrac.__data_transfer_cache_size=0
smrrac.__db_cache_size=2952790016
smrrac.__java_pool_size=16777216
smrrac.__large_pool_size=33554432
smrrac.__oracle_base='/u01/app/oracle'#ORACLE_BASE set from environment
smrrac.__pga_aggregate_target=1325400064
smrrac.__sga_target=3942645760
smrrac.__shared_io_pool_size=201326592
smrrac.__shared_pool_size=721420288
smrrac.__streams_pool_size=0
*.audit_file_dest='/u01/app/oracle/admin/smrrac/adump'
*.audit_trail='db'
*.compatible='12.1.0.2.0'
*.control_files='/u01/app/oracle/oradata/smrrac/control01.ctl','/u01/app/oracle/fast_recovery_area/smrrac/control02.ctl'
*.db_block_size=8192
*.db_domain=''
*.db_name='smrrac'
#*.db_recovery_file_dest='/u01/app/oracle/fast_recovery_area'
#*.db_recovery_file_dest_size=4560m
*.diagnostic_dest='/u01/app/oracle'
*.dispatchers='(PROTOCOL=TCP) (SERVICE=smrracXDB)'
*.local_listener='LISTENER_SMRRAC'
*.log_archive_format='%t_%s_%r.arc'
*.STANDBY_FILE_MANAGEMENT='AUTO'
LOG_ARCHIVE_DEST_1='LOCATION=/u01/app/oracle/fast_recovery_area/SMRRAC/archivelog'
*.open_cursors=300
*.pga_aggregate_target=1250m
*.processes=450
*.remote_login_passwordfile='EXCLUSIVE'
*.sga_target=3750m
*.undo_tablespace='UNDOTBS1'
oracle@8:/u01/app/oracle/product/12.1.0/dbhome_1/dbs>

oracle@8:/u01/app/oracle/product/12.1.0/dbhome_1/dbs> vi initrac.ora


--- Start again physical standby database with pfile initsmrrac.ora ---

oracle@8:/u01/app/oracle/product/12.1.0/dbhome_1/dbs> sqlplus / as sysdba



SQL*Plus: Release 12.1.0.2.0 Production on Wed Feb 26 14:50:25 2020



Copyright (c) 1982, 2014, Oracle.  All rights reserved.



Connected to an idle instance.



SQL> startup nomount pfile='/u01/app/oracle/product/12.1.0/dbhome_1/dbs/initsmrrac.ora';

ORACLE instance started.



Total System Global Area 3942645760 bytes

Fixed Size                  2931616 bytes

Variable Size             939525216 bytes

Database Buffers         2986344448 bytes

Redo Buffers               13844480 bytes

SQL> exit

Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production

With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

oracle@8:/u01/app/oracle/product/12.1.0/dbhome_1/dbs> rman target /



Recovery Manager: Release 12.1.0.2.0 - Production on Wed Feb 26 14:51:11 2020



Copyright (c) 1982, 2014, Oracle and/or its affiliates.  All rights reserved.



connected to target database: SMRRAC (not mounted)

---- Restore Standby database Controlfile Again (optional) ----

RMAN> restore standby controlfile from '/u01/rman_smrrac/standbycontrol_SMRRAC_32upfn2e_1_1.ctl';



Starting restore at 26-FEB-2020 14:51:34

using target database control file instead of recovery catalog

allocated channel: ORA_DISK_1

channel ORA_DISK_1: SID=527 device type=DISK



channel ORA_DISK_1: restoring control file

channel ORA_DISK_1: restore complete, elapsed time: 00:00:01

output file name=/u01/app/oracle/oradata/smrrac/control01.ctl

output file name=/u01/app/oracle/fast_recovery_area/smrrac/control02.ctl

Finished restore at 26-FEB-2020 14:51:36

--- Mount database physical standby  ---

RMAN> alter database mount standby;

RMAN> ALTER DATABASE MOUNT STANDBY DATABASE;

Statement processed

released channel: ORA_DISK_1

--- Catalog Start with ---

RMAN> catalog start with  '/u01/rman_smrrac/';

searching for all files that match the pattern /u01/rman_smrrac/

List of Files Unknown to the Database

=====================================

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2nupfmsr_2135_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2mupfmsr_2134_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2kupfmsr_2132_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2lupfmsr_2133_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2oupfmtk_2136_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2rupfmu4_2139_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2tupfmu7_2141_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2uupfmu9_2142_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2qupfmu4_2138_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2supfmu4_2140_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2pupfmu4_2137_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2vupfn2a_2143_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_30upfn2a_2144_1_20200226

File Name: /u01/rman_smrrac/control_SMRRAC_20200226.bak

File Name: /u01/rman_smrrac/standbycontrol_SMRRAC_32upfn2e_1_1.ctl

Do you really want to catalog the above files (enter YES or NO)? yes

cataloging files...

cataloging done

List of Cataloged Files

=======================

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2nupfmsr_2135_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2mupfmsr_2134_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2kupfmsr_2132_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2lupfmsr_2133_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2oupfmtk_2136_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2rupfmu4_2139_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2tupfmu7_2141_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2uupfmu9_2142_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2qupfmu4_2138_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2supfmu4_2140_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2pupfmu4_2137_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_2vupfn2a_2143_1_20200226

File Name: /u01/rman_smrrac/Backup_SMRRAC_DB_30upfn2a_2144_1_20200226

File Name: /u01/rman_smrrac/control_SMRRAC_20200226.bak

File Name: /u01/rman_smrrac/standbycontrol_SMRRAC_32upfn2e_1_1.ctl

--- Restore standby database ---

RMAN> run

{

set newname for datafile 1 to "/u01/app/oracle/oradata/smrrac/system01.dbf";

set newname for datafile 2 to "/u02/oradata/smrrac/idata01.dbf";

set newname for datafile 3 to "/u01/app/oracle/oradata/smrrac/sysaux01.dbf";

set newname for datafile 4 to "/u01/app/oracle/oradata/smrrac/undotbs01.dbf";

2> 3> set newname for datafile 5 to "/u01/app/oracle/oradata/smrrac/undotbs02.dbf";

set newname for datafile 6 to "/u02/oradata/smrrac/users01.dbf";

set newname for datafile 7 to "/u02/oradata/smrrac/ilog01.dbf";

set newname for datafile 8 to "/u02/oradata/smrrac/indx01.dbf";

4> 5> 6> 7> 8> 9> 10> 11> set newname for tempfile 1 to "/u01/app/oracle/oradata/smrrac/temp01.dbf";

RESTORE DATABASE;

12> 13> SWITCH DATAFILE ALL;

SWITCH TEMPFILE ALL;

}14> 15>

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME



executing command: SET NEWNAME



executing command: SET NEWNAME



executing command: SET NEWNAME



executing command: SET NEWNAME



Starting restore at 26-FEB-2020 14:54:45

allocated channel: ORA_DISK_1

channel ORA_DISK_1: SID=357 device type=DISK



channel ORA_DISK_1: starting datafile backup set restore

channel ORA_DISK_1: specifying datafile(s) to restore from backup set

channel ORA_DISK_1: restoring datafile 00004 to /u01/app/oracle/oradata/smrrac/undotbs01.dbf

channel ORA_DISK_1: restoring datafile 00005 to /u01/app/oracle/oradata/smrrac/undotbs02.dbf

channel ORA_DISK_1: reading from backup piece /u04/MasterDB/smrrac_backup_64/Backup_SMRRAC_DB_2rupfmu4_2139_1_20200226

channel ORA_DISK_1: errors found reading piece handle=/u04/MasterDB/smrrac_backup_64/Backup_SMRRAC_DB_2rupfmu4_2139_1_20200226

channel ORA_DISK_1: failover to piece handle=/u01/rman_smrrac/Backup_SMRRAC_DB_2rupfmu4_2139_1_20200226 tag=TAG20200226T050611

channel ORA_DISK_1: restored backup piece 1

channel ORA_DISK_1: restore complete, elapsed time: 00:00:15

channel ORA_DISK_1: starting datafile backup set restore

channel ORA_DISK_1: specifying datafile(s) to restore from backup set

channel ORA_DISK_1: restoring datafile 00001 to /u01/app/oracle/oradata/smrrac/system01.dbf

channel ORA_DISK_1: restoring datafile 00008 to /u02/oradata/smrrac/indx01.dbf

channel ORA_DISK_1: reading from backup piece /u04/MasterDB/smrrac_backup_64/Backup_SMRRAC_DB_2qupfmu4_2138_1_20200226

channel ORA_DISK_1: errors found reading piece handle=/u04/MasterDB/smrrac_backup_64/Backup_SMRRAC_DB_2qupfmu4_2138_1_20200226

channel ORA_DISK_1: failover to piece handle=/u01/rman_smrrac/Backup_SMRRAC_DB_2qupfmu4_2138_1_20200226 tag=TAG20200226T050611

channel ORA_DISK_1: restored backup piece 1

channel ORA_DISK_1: restore complete, elapsed time: 00:00:35

channel ORA_DISK_1: starting datafile backup set restore

channel ORA_DISK_1: specifying datafile(s) to restore from backup set

channel ORA_DISK_1: restoring datafile 00003 to /u01/app/oracle/oradata/smrrac/sysaux01.dbf

channel ORA_DISK_1: restoring datafile 00007 to /u02/oradata/smrrac/ilog01.dbf

channel ORA_DISK_1: reading from backup piece /u04/MasterDB/smrrac_backup_64/Backup_SMRRAC_DB_2supfmu4_2140_1_20200226

channel ORA_DISK_1: errors found reading piece handle=/u04/MasterDB/smrrac_backup_64/Backup_SMRRAC_DB_2supfmu4_2140_1_20200226

channel ORA_DISK_1: failover to piece handle=/u01/rman_smrrac/Backup_SMRRAC_DB_2supfmu4_2140_1_20200226 tag=TAG20200226T050611

channel ORA_DISK_1: restored backup piece 1

channel ORA_DISK_1: restore complete, elapsed time: 00:00:55

channel ORA_DISK_1: starting datafile backup set restore

channel ORA_DISK_1: specifying datafile(s) to restore from backup set

channel ORA_DISK_1: restoring datafile 00002 to /u02/oradata/smrrac/idata01.dbf

channel ORA_DISK_1: restoring datafile 00006 to /u02/oradata/smrrac/users01.dbf

channel ORA_DISK_1: reading from backup piece /u04/MasterDB/smrrac_backup_64/Backup_SMRRAC_DB_2pupfmu4_2137_1_20200226

channel ORA_DISK_1: errors found reading piece handle=/u04/MasterDB/smrrac_backup_64/Backup_SMRRAC_DB_2pupfmu4_2137_1_20200226

channel ORA_DISK_1: failover to piece handle=/u01/rman_smrrac/Backup_SMRRAC_DB_2pupfmu4_2137_1_20200226 tag=TAG20200226T050611

channel ORA_DISK_1: restored backup piece 1

channel ORA_DISK_1: restore complete, elapsed time: 00:03:05

Finished restore at 26-FEB-2020 14:59:35



datafile 1 switched to datafile copy

input datafile copy RECID=150 STAMP=1033397976 file name=/u01/app/oracle/oradata/smrrac/system01.dbf

datafile 2 switched to datafile copy

input datafile copy RECID=151 STAMP=1033397976 file name=/u02/oradata/smrrac/idata01.dbf

datafile 3 switched to datafile copy

input datafile copy RECID=152 STAMP=1033397976 file name=/u01/app/oracle/oradata/smrrac/sysaux01.dbf

datafile 4 switched to datafile copy

input datafile copy RECID=153 STAMP=1033397976 file name=/u01/app/oracle/oradata/smrrac/undotbs01.dbf

datafile 5 switched to datafile copy

input datafile copy RECID=154 STAMP=1033397976 file name=/u01/app/oracle/oradata/smrrac/undotbs02.dbf

datafile 6 switched to datafile copy

input datafile copy RECID=155 STAMP=1033397976 file name=/u02/oradata/smrrac/users01.dbf

datafile 7 switched to datafile copy

input datafile copy RECID=156 STAMP=1033397976 file name=/u02/oradata/smrrac/ilog01.dbf

datafile 8 switched to datafile copy

input datafile copy RECID=157 STAMP=1033397976 file name=/u02/oradata/smrrac/indx01.dbf



renamed tempfile 1 to /u01/app/oracle/oradata/smrrac/temp01.dbf in control file

-----  Check MRP0 using v$managed_standby -----

RMAN> select process,status,sequence# from v$managed_standby;



PROCESS   STATUS        SEQUENCE#

--------- ------------ ----------

ARCH      CONNECTED             0

ARCH      CONNECTED             0

ARCH      CONNECTED             0

ARCH      CONNECTED             0



RMAN>



alter database recover managed standby database disconnect from session;

RMAN>

RMAN> select process,status,sequence# from v$managed_standby;



Statement processed



PROCESS   STATUS        SEQUENCE#

--------- ------------ ----------

ARCH      CONNECTED             0

ARCH      CONNECTED             0

ARCH      CONNECTED             0

ARCH      CONNECTED             0

MRP0      WAIT_FOR_GAP       1066



RMAN>