Wednesday 29 April 2020

Exception [type: SIGSEGV, Address not mapped to object] [ADDR:0x10] [PC:0xCE518CE, qerixGetKey()+782] [flags: 0x0, count: 1]

Scenario Preview:

We recently hit error in our production node and test environment . So as per suggestion by support. we implemented the same first one is change on parameter on database and second one is apply patch to resolve bug.

ERRORS:

Error at application level and in sqlplus prompt:

ERROR at line 1:

ORA-03113: end-of-file on communication channel

Process ID: 3388

Session ID: 324 Serial number: 55071


Error in Alert.log 

Fri Apr 24 09:41:23 2020

Exception [type: SIGSEGV, Address not mapped to object] [ADDR:0x10] [PC:0xCE518CE, qerixGetKey()+782] [flags: 0x0, count: 1]

Errors in file /u01/app/oracle/diag/rdbms/TESTENV/TESTENV/trace/TESTENV_ora_2543.trc  (incident=615):

ORA-07445: exception encountered: core dump [qerixGetKey()+782] [SIGSEGV] [ADDR:0x10] [PC:0xCE518CE] [Address not mapped to object] []

Incident details in: /u01/app/oracle/diag/rdbms/TESTENV/TESTENV/incident/incdir_615/TESTENV_ora_2543_i615.trc

Use ADRCI or Support Workbench to package the incident.

See Note 411.1 at My Oracle Support for error and packaging details.

Fri Apr 24 09:41:25 2020

Errors in file /u01/app/oracle/diag/rdbms/TESTENV/TESTENV/incident/incdir_615/TESTENV_ora_2543_i615.trc:

ORA-10173: Dynamic Sampling time-out error

ORA-07445: exception encountered: core dump [qerixGetKey()+782] [SIGSEGV] [ADDR:0x10] [PC:0xCE518CE] [Address not mapped to object] []

Fri Apr 24 09:41:25 2020

Errors in file /u01/app/oracle/diag/rdbms/TESTENV/TESTENV/incident/incdir_615/TESTENV_ora_2543_i615.trc:

ORA-10173: Dynamic Sampling time-out error

ORA-07445: exception encountered: core dump [qerixGetKey()+782] [SIGSEGV] [ADDR:0x10] [PC:0xCE518CE] [Address not mapped to object] []


Solution:


1) The recommended temporary workaround is to set one of the following

       OPTIMIZER_ADAPTIVE_FEATURES=false

     alter session set "_projection_pushdown"=false;

    alter session set optimizer_adaptive_features=false;


  2) The permanent solution is for Teamcenter customers who have downloaded

    and installed Oracle 12.1.0.2 to download and install the following

    patch bundle, which includes the correction for Oracle Bug 18430870.


    This patch is available on the GTAC download server for each

    supported Teamcenter platform in the Oracle / Product Updates / Oracle12

    folder.

     p22809813_121020_MSWIN-x86-64.zip

    p22899531_121020_Linux-x86-64.zip

    p22899531_121020_Solaris86-64.zip



Practice Solution Scenario:


NAME      INSTANCE_NAME    DB_UNIQUE_NAME                 OPEN_MODE            DATABASE_ROLE    CURRENT_SCN        FAILOVER_SCN

--------- ---------------- ------------------------------ -------------------- ---------------- ------------------ ----------------------------------------

xxxTESTE  TESTENV       TESTENV                     READ WRITE           PRIMARY          NO                 0



SQL> SELECT /*+ DYNAMIC_SAMPLING(t_02 4)*/ ;

SELECT /*+ DYNAMIC_SAMPLING(t_02 4)*/) ) ) ) )

*

ERROR at line 1:

ORA-03113: end-of-file on communication channel

Process ID: 3388

Session ID: 324 Serial number: 55071



SQL>



SQL> ALTER SYSTEM SET OPTIMIZER_ADAPTIVE_FEATURES = FALSE SCOPE=BOTH;



System altered.



SQL> ALTER SYSTEM SET "_projection_pushdown"=false SCOPE=BOTH;



System altered.



SQL> @d



NAME      INSTANCE_NAME    DB_UNIQUE_NAME                 OPEN_MODE            DATABASE_ROLE    CURRENT_SCN        FAILOVER_SCN

--------- ---------------- ------------------------------ -------------------- ---------------- ------------------ ----------------------------------------

xxxPATCH  TESTENV        TESTENV                      READ WRITE           PRIMARY          NO                 0



SQL> SELECT /*+ DYNAMIC_SAMPLING(t_02 4)*xxxxx -------------------------------;



COLUMN_ENTRY
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
xxx

xxxxx



SQL> sho parameter OPTIMIZER_ADAPTIVE_FEATURES



NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

optimizer_adaptive_features          boolean     FALSE

SQL> sho parameter _projection_pushdown



NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

_projection_pushdown                 boolean     FALSE

SQL>





 




Thursday 23 April 2020

RAC - ORA-15025: could not open disk |WARNING: FAILED to load library: /opt/oracle/extapi/64/asm/orcl/1/libasm.so


Scenario : 

Recently isntalling 2 node RAC 12c R1 , We face error during running ./dbca , Although we successfully installed grid setup ,asmca disk configuration  and oracle rdbms binary. but while installing instance we face error. I have done implementation several times but some silly mistake by me.. As was just test so i was taking it seriously fed up installing RAC setup , But today i learn something new for this error. following  (Doc ID 1378747.1)

Error:

ORA-15025: could not open disk "/dev/oracleasm/disks/FRA"

WARNING: FAILED to load library: /opt/oracle/extapi/64/asm/orcl/1/libasm.so


alert log error on rac1.log


ORA-15025: could not open disk "/dev/oracleasm/disks/FRA"

WARNING: FAILED to load library: /opt/oracle/extapi/64/asm/orcl/1/libasm.so
Wed Apr 22 14:38:18 2020
NOTE: ASMB connected to ASM instance +ASM1 osid: 15845 (Flex mode; client id 0xffffffffffffffff)
NOTE: initiating MARK startup
Starting background process MARK
Wed Apr 22 14:38:18 2020
MARK started with pid=35, OS id=15849
Wed Apr 22 14:38:19 2020
NOTE: MARK has subscribed
Wed Apr 22 14:38:26 2020
NOTE: ASMB mounting group 3 (FRA)
Wed Apr 22 14:38:28 2020
ORA-15025: could not open disk "/dev/oracleasm/disks/FRA"
ORA-27041: unable to open file
Linux-x86_64 Error: 13: Permission denied
Additional information: 9
ORA-15183: ASMLIB initialization error [driver/agent not installed]
NOTE: Disk 0 in group 3 could not be opened.
WARNING: Failed to complete group 3
WARNING: group 3 is being dismounted.
WARNING: ASMB force dismounting group 3 (FRA) due to failed mount
SUCCESS: diskgroup FRA was dismounted
NOTE: ASMB mounting group 2 (DATA)
Wed Apr 22 14:38:32 2020
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA"
ORA-27041: unable to open file
Linux-x86_64 Error: 13: Permission denied
Additional information: 9
ORA-15040: diskgroup is incomplete
NOTE: Disk 0 in group 2 could not be opened.
WARNING: Failed to complete group 2
WARNING: group 2 is being dismounted.
WARNING: ASMB force dismounting group 2 (DATA) due to failed mount
SUCCESS: diskgroup DATA was dismounted
NOTE: ASMB mounting group 2 (DATA)
Wed Apr 22 14:38:33 2020
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA"
ORA-27041: unable to open file
Linux-x86_64 Error: 13: Permission denied
Additional information: 9
ORA-15040: diskgroup is incomplete
NOTE: Disk 0 in group 2 could not be opened.
WARNING: Failed to complete group 2
WARNING: group 2 is being dismounted.

############################################################################

Solution:

After troubleshooting i checked  asmdba group was not provided to oracle and grid user. Although created but some how missed to proved group on user.


grid@sgdcplm01:~> id
uid=54323(grid) gid=54321(oinstall) groups=54321(oinstall),54329(asmadmin)
grid@sgdcplm01:~> su oracle
Password:
oracle@sgdcplm01:/home/grid> id
uid=502(oracle) gid=54321(oinstall) groups=54321(oinstall),503(dba),504(oper),1000(beoper) ,54329(asmadmin)
oracle@sgdcplm01:/home/grid>

( Adding group to oracle and grid user by executing from root user )

 # usermod -a -G asmdba oracle

# usermod -a -G asmdba grid


login as: grid
Keyboard-interactive authentication prompts from server:
| Password:
End of keyboard-interactive prompts from server
Last login: Wed Apr 22 09:54:57 2020 from 10.4.6.143
grid@sgdcplm01:~> id
uid=54323(grid) gid=54321(oinstall) groups=54321(oinstall),54327(asmdba),54329(asmadmin)
grid@sgdcplm01:~> su oracle
Password:
oracle@sgdcplm01:/home/grid> id
uid=502(oracle) gid=54321(oinstall) groups=54321(oinstall),503(dba),504(oper),1000(beoper),54327(asmdba),54329(asmadmin)
oracle@sgdcplm01:/home/grid>


grid@srv2:/u01/app/12.1.0/grid/bin> . oraenv
ORACLE_SID = [grid] ? +ASM2
The Oracle base has been set to /u01/app/grid

grid@srv2:/u01/app/12.1.0/grid/bin> ./setasmgidwrapo=/u01/app/oracle/product/12.1.0/db_1/bin/oracle


grid@srv2:/u01/app/12.1.0/grid/bin>

(NOTE: Following doc id below: Database Will Not Mount: ORA-15025, ORA-27041, 'Permission denied', ORA-15081 (Doc ID 1378747.1)


Alert log :

Wed Apr 22 15:54:45 2020
NOTE: ASMB registering with ASM instance as Standard client 0xffffffffffffffff (reg:2301527851) (new connection)
Starting background process RBAL
Wed Apr 22 15:54:45 2020
RBAL started with pid=34, OS id=11971
Wed Apr 22 15:54:45 2020
NOTE: Loaded library: /opt/oracle/extapi/64/asm/orcl/1/libasm.so
Wed Apr 22 15:54:45 2020
NOTE: ASMB connected to ASM instance +ASM1 osid: 11969 (Flex mode; client id 0xffffffffffffffff)
NOTE: initiating MARK startup
Starting background process MARK
Wed Apr 22 15:54:45 2020
MARK started with pid=35, OS id=11973
Wed Apr 22 15:54:45 2020
NOTE: MARK has subscribed
Wed Apr 22 15:54:50 2020
NOTE: ASMB mounting group 3 (FRA)
NOTE: Assigning number (3,0) to disk (/dev/oracleasm/disks/FRA)
SUCCESS: mounted group 3 (FRA)
NOTE: grp 3 disk 0: FRA_0000 path:/dev/oracleasm/disks/FRA
NOTE: ASMB mounting group 2 (DATA)
NOTE: Assigning number (2,0) to disk (/dev/oracleasm/disks/DATA)
SUCCESS: mounted group 2 (DATA)
NOTE: grp 2 disk 0: DATA_0000 path:/dev/oracleasm/disks/DATA



Tuesday 21 April 2020

RAC : [INS-13001] Environment does not meet minimum requirements.





















Unzip grid software and go to directory /u01/grid/stage/cvu/cv/admin andmake changes on cvu_config file
srv1:/u01/grid # ll
total 24
drwxrwxr-x 1 grid oinstall  298 Apr 21 20:53 install
drwxrwxr-x 1 grid oinstall   32 Jul  7  2014 response
drwxrwxr-x 1 grid oinstall   40 Jul  7  2014 rpm
-rwxrwxr-x 1 grid oinstall 8534 Jul  7  2014 runInstaller
-rwxrwxr-x 1 grid oinstall 5085 Dec 20  2013 runcluvfy.sh
drwxrwxr-x 1 grid oinstall   30 Jul  7  2014 sshsetup
drwxrwxr-x 1 grid oinstall  910 Jul  7  2014 stage
-rwxrwxr-x 1 grid oinstall  500 Feb  7  2013 welcome.html
srv1:/u01/grid #

srv1:/u01/grid/stage/cvu/cv/admin # ll
total 62472
-rwxrwxr-x 1 grid oinstall     1263 Jul  1  2014 cvu_config
-rwxrwxr-x 1 grid oinstall 63964254 Jul  1  2014 cvunetquery
srv1:/u01/grid/stage/cvu/cv/admin #

# Enable value 

CV_ASSUME_DISTID=SUSE11

zypper install libstdc++46-devel libstdc++33 libstdc++33-devel libelf-devel gcc-c++ libaio-devel sysstat unixODBC unixODBC-devel xdpyinfo