ASM Error - CRS-4124: Oracle High Availability Services startup failed on Linux 6

Problem

Once Installed the ASM Instance, Failed to start OHASD services

CRS-4124: Oracle High Availability Services startup failed.
CRS-4000: Command Start failed, or completed with errors.
ohasd failed to start: Inappropriate ioctl for device
ohasd failed to start at /grid/app/11.2.0/grid/crs/install/rootcrs.pl line 443.

Above error because of 11gR2 Database not support with Linux 6


Solution

1. Login to racnode1
2. Open the file $GRID_HOME/crs/install/s_crsconfig_lib.pm
3. Add the following lines before the # Start OHASD

## Added by Mohamed ##

my $UPSTART_OHASD_SERVICE = "oracle-ohasd";
my $INITCTL = "/sbin/initctl";

($status, @output) = system_cmd_capture ("$INITCTL start $UPSTART_OHASD_SERVICE");
if (0 != $status)
{
error ("Failed to start $UPSTART_OHASD_SERVICE, error: $!");
return $FAILED;
}

        # Start OHASD

4. Create a file /etc/init/oracle-ohasd.conf with below content

# Oracle OHASD startup
start on runlevel [35]
stop on runlevel [!35]
respawn
exec /etc/init.d/init.ohasd run >/dev/null 2>&1

5. De-config root.sh on racnode1

[root@astrac1 ~]# cd /u01/app/11.2.0/grid/crs/install/
[root@astrac1 install]# ./roothas.pl -deconfig -force -verbose
2016-03-16 15:20:48: Checking for super user privileges
2016-03-16 15:20:48: User has super user privileges
2016-03-16 15:20:48: Parsing the host name
Using configuration parameter file: ./crsconfig_params
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Stop failed, or completed with errors.
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Delete failed, or completed with errors.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'astrac1'
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'astrac1'
CRS-2673: Attempting to stop 'ora.evmd' on 'astrac1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'astrac1'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'astrac1'
CRS-2677: Stop of 'ora.cssdmonitor' on 'astrac1' succeeded
CRS-2677: Stop of 'ora.evmd' on 'astrac1' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'astrac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'astrac1'
CRS-2677: Stop of 'ora.mdnsd' on 'astrac1' succeeded
CRS-2677: Stop of 'ora.gipcd' on 'astrac1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'astrac1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
ADVM/ACFS is not supported on oraclelinux-release-6Server-7.0.5.x86_64

ACFS-9201: Not Supported
Successfully deconfigured Oracle Restart stack
[root@astrac1 install]#

5. Re-run root.sh on racnode1

[root@astrac1 ~]# cd /u01/app/11.2.0/grid/
[root@astrac1 grid]# ./root.sh
Running Oracle 11g root.sh script...

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The file "dbhome" already exists in /usr/local/bin.  Overwrite it? (y/n)
[n]: y
   Copying dbhome to /usr/local/bin ...
The file "oraenv" already exists in /usr/local/bin.  Overwrite it? (y/n)
[n]: y
   Copying oraenv to /usr/local/bin ...
The file "coraenv" already exists in /usr/local/bin.  Overwrite it? (y/n)
[n]: y
   Copying coraenv to /usr/local/bin ...

Now racnode1 is configured successfully and OHASD service started. Do the same steps to start the OHASD services on rest of the nodes.

To find forgotten Apps and Sysadmin password in R12

To find forgotten Apps Password

1. Connect sqlplus via sysdba

sqlplus / as sysdba

2. Create a function as follows

create or replace FUNCTION apps.decrypt_pin_func(in_chr_key IN VARCHAR2,in_chr_encrypt_pin IN VARCHAR2) RETURN VARCHAR2
AS
LANGUAGE JAVA NAME 'oracle.apps.fnd.security.WebSessionManagerProc.decrypt(java.lang.String,java.lang.String) return java.lang.String';
/

3. Get the Encrypted password for GUEST user

select ENCRYPTED_FOUNDATION_PASSWORD from apps.fnd_user where USER_NAME='GUEST';

4. Get the apps password using encrypted password return in step 3

SELECT apps.decrypt_pin_func('GUEST/ORACLE','<ENCRYPTED_FOUNDATION_PASSWORD>') from dual;

To find forgotten Sysadmin Password

1. Connect sqlplus via sysdba

sqlplus / as sysdba

2. Create a package an package body as follows

CREATE OR REPLACE PACKAGE XXARTO_GET_PWD AS
FUNCTION decrypt (KEY IN VARCHAR2, VALUE IN VARCHAR2)
RETURN VARCHAR2;
END XXARTO_GET_PWD;

CREATE OR REPLACE PACKAGE BODY XXARTO_GET_PWD AS
FUNCTION decrypt (KEY IN VARCHAR2, VALUE IN VARCHAR2)
RETURN VARCHAR2 AS
LANGUAGE JAVA NAME 'oracle.apps.fnd.security.WebSessionManagerProc.decrypt
(java.lang.String,java.lang.String) return java.lang.String';
END XXARTO_GET_PWD;

4. Get the password for sysadmin using below script

SELECT Usr.User_Name,
Usr.Description,
XXARTO_GET_PWD.Decrypt (
(SELECT (SELECT XXARTO_GET_PWD.Decrypt (
Fnd_Web_Sec.Get_Guest_Username_Pwd,
Usertable.Encrypted_Foundation_Password)
FROM DUAL)
AS Apps_Password
FROM applsys.Fnd_User Usertable
WHERE Usertable.User_Name =
(SELECT SUBSTR (
Fnd_Web_Sec.Get_Guest_Username_Pwd,
1,
INSTR (Fnd_Web_Sec.Get_Guest_Username_Pwd,
'/')
- 1)
FROM DUAL)),
Usr.Encrypted_User_Password)
Password
FROM applsys.Fnd_User Usr
WHERE Usr.User_Name = 'SYSADMIN';