ASM Error - CRS-4124: Oracle High Availability Services startup failed on Linux 6

Problem

Once Installed the ASM Instance, Failed to start OHASD services

CRS-4124: Oracle High Availability Services startup failed.
CRS-4000: Command Start failed, or completed with errors.
ohasd failed to start: Inappropriate ioctl for device
ohasd failed to start at /grid/app/11.2.0/grid/crs/install/rootcrs.pl line 443.

Above error because of 11gR2 Database not support with Linux 6


Solution

1. Login to racnode1
2. Open the file $GRID_HOME/crs/install/s_crsconfig_lib.pm
3. Add the following lines before the # Start OHASD

## Added by Mohamed ##

my $UPSTART_OHASD_SERVICE = "oracle-ohasd";
my $INITCTL = "/sbin/initctl";

($status, @output) = system_cmd_capture ("$INITCTL start $UPSTART_OHASD_SERVICE");
if (0 != $status)
{
error ("Failed to start $UPSTART_OHASD_SERVICE, error: $!");
return $FAILED;
}

        # Start OHASD

4. Create a file /etc/init/oracle-ohasd.conf with below content

# Oracle OHASD startup
start on runlevel [35]
stop on runlevel [!35]
respawn
exec /etc/init.d/init.ohasd run >/dev/null 2>&1

5. De-config root.sh on racnode1

[root@astrac1 ~]# cd /u01/app/11.2.0/grid/crs/install/
[root@astrac1 install]# ./roothas.pl -deconfig -force -verbose
2016-03-16 15:20:48: Checking for super user privileges
2016-03-16 15:20:48: User has super user privileges
2016-03-16 15:20:48: Parsing the host name
Using configuration parameter file: ./crsconfig_params
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Stop failed, or completed with errors.
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Delete failed, or completed with errors.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'astrac1'
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'astrac1'
CRS-2673: Attempting to stop 'ora.evmd' on 'astrac1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'astrac1'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'astrac1'
CRS-2677: Stop of 'ora.cssdmonitor' on 'astrac1' succeeded
CRS-2677: Stop of 'ora.evmd' on 'astrac1' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'astrac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'astrac1'
CRS-2677: Stop of 'ora.mdnsd' on 'astrac1' succeeded
CRS-2677: Stop of 'ora.gipcd' on 'astrac1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'astrac1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
ADVM/ACFS is not supported on oraclelinux-release-6Server-7.0.5.x86_64

ACFS-9201: Not Supported
Successfully deconfigured Oracle Restart stack
[root@astrac1 install]#

5. Re-run root.sh on racnode1

[root@astrac1 ~]# cd /u01/app/11.2.0/grid/
[root@astrac1 grid]# ./root.sh
Running Oracle 11g root.sh script...

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The file "dbhome" already exists in /usr/local/bin.  Overwrite it? (y/n)
[n]: y
   Copying dbhome to /usr/local/bin ...
The file "oraenv" already exists in /usr/local/bin.  Overwrite it? (y/n)
[n]: y
   Copying oraenv to /usr/local/bin ...
The file "coraenv" already exists in /usr/local/bin.  Overwrite it? (y/n)
[n]: y
   Copying coraenv to /usr/local/bin ...

Now racnode1 is configured successfully and OHASD service started. Do the same steps to start the OHASD services on rest of the nodes.

To find forgotten Apps and Sysadmin password in R12

To find forgotten Apps Password

1. Connect sqlplus via sysdba

sqlplus / as sysdba

2. Create a function as follows

create or replace FUNCTION apps.decrypt_pin_func(in_chr_key IN VARCHAR2,in_chr_encrypt_pin IN VARCHAR2) RETURN VARCHAR2
AS
LANGUAGE JAVA NAME 'oracle.apps.fnd.security.WebSessionManagerProc.decrypt(java.lang.String,java.lang.String) return java.lang.String';
/

3. Get the Encrypted password for GUEST user

select ENCRYPTED_FOUNDATION_PASSWORD from apps.fnd_user where USER_NAME='GUEST';

4. Get the apps password using encrypted password return in step 3

SELECT apps.decrypt_pin_func('GUEST/ORACLE','<ENCRYPTED_FOUNDATION_PASSWORD>') from dual;

To find forgotten Sysadmin Password

1. Connect sqlplus via sysdba

sqlplus / as sysdba

2. Create a package an package body as follows

CREATE OR REPLACE PACKAGE XXARTO_GET_PWD AS
FUNCTION decrypt (KEY IN VARCHAR2, VALUE IN VARCHAR2)
RETURN VARCHAR2;
END XXARTO_GET_PWD;

CREATE OR REPLACE PACKAGE BODY XXARTO_GET_PWD AS
FUNCTION decrypt (KEY IN VARCHAR2, VALUE IN VARCHAR2)
RETURN VARCHAR2 AS
LANGUAGE JAVA NAME 'oracle.apps.fnd.security.WebSessionManagerProc.decrypt
(java.lang.String,java.lang.String) return java.lang.String';
END XXARTO_GET_PWD;

4. Get the password for sysadmin using below script

SELECT Usr.User_Name,
Usr.Description,
XXARTO_GET_PWD.Decrypt (
(SELECT (SELECT XXARTO_GET_PWD.Decrypt (
Fnd_Web_Sec.Get_Guest_Username_Pwd,
Usertable.Encrypted_Foundation_Password)
FROM DUAL)
AS Apps_Password
FROM applsys.Fnd_User Usertable
WHERE Usertable.User_Name =
(SELECT SUBSTR (
Fnd_Web_Sec.Get_Guest_Username_Pwd,
1,
INSTR (Fnd_Web_Sec.Get_Guest_Username_Pwd,
'/')
- 1)
FROM DUAL)),
Usr.Encrypted_User_Password)
Password
FROM applsys.Fnd_User Usr
WHERE Usr.User_Name = 'SYSADMIN';

Tape Backup / Restoration commands

Display the content of the tape

tar tvf /dev/st0 

To backup directory to tape with compression

tar cvzf /dev/st0 /u01/finsys

To restore entire data from the tape

tar -xvf /dev/st0 -C /u01/TEST

To restore specific file from the tape

tar -xvf /dev/st0 u01/PROD/finsys.tar -C /u01/TEST

Reference
http://www.cyberciti.biz/faq/linux-tape-backup-with-mt-and-tar-command-howto/

Usage of FNDCPASS utility

Prerequisite

Oracle Applications system should be shut down before changing any schema passwords. Also FND_USER and FND_ORACLE_USERID tables should be backed up before changing any passwords.

FNDCPASS <apps username>/<apps password> 0 Y <system username>/<system password> <token> <username> <new_password>  

=>The SYSTEM token is used when changing the APPLSYS password.
=>The ORACLE token is used when changing a Single Applications schema password.
=>The ALLORACLE token is used when changing All Applications schema passwords.
=>The USER token is used when changing an Applications User password.


Following are the different usage of FNDCPASS utility

1. Change Apps and Applsys schema password
Token=SYSTEM

FNDCPASS apps/<APPS password> 0 Y system/<SYSTEM password> SYSTEM APPLSYS <new_password>

2. Change an Oracle Registered Application schema password (Other than Apps and Applsys)
Token=ORACLE

FNDCPASS apps/<APPS password> 0 Y system/<SYSTEM password> ORACLE GL <new_password>

3. Change All Oracle Registered Application schema password (Other than Apps and Applsys)
Token=ALLORACLE

FNDCPASS apps/<APPS password> 0 Y system/<SYSTEM password> ALLORACLE <new_password>

4. Change an All Oracle Applications user password
Token=USER

FNDCPASS apps/<APPS password> 0 Y system/<SYSTEM password> USER SYSADMIN <new_password>

Reference
How to Change Applications Passwords using Applications Schema Password Change Utility (FNDCPASS or AFPASSWD) (Doc ID 437260.1)

ORA-20008 ORA-6512 ADOP patching issue on 12.2

Problem

When prepare phase following error occur

adop phase=prepare

ERROR at line 1:

ORA-20008: No Concurrent Manager is defined that can run concurrent program
ADZDPATCH
ORA-6512: at "APPS.AD_ZD_ADOP", line 240

Solution

Execute below command to overcome the issue

FNDLOAD apps/<apps_pw> 0 Y UPLOAD $FND_TOP/patch/115/import/afcpprog.lct $AD_TOP/patch/115/import/US/adzdpatch.ldt - CUSTOM_MODE=FORCE

SQL Script for find Application Short Name of Oracle R12 Modules

The Following Query List all the EBS Application name with short name

SELECT fat.application_name        "Application Name",
       fa.application_short_name   "Short Name"
 FROM fnd_application     fa,
       fnd_application_tl  fat
 WHERE fa.application_id = fat.application_id
   AND fat.language      = USERENV('LANG')
 ORDER BY fa.application_short_name;

Application Name	Short Name
Activity Based Management (Obsolete)	ABM
Applications DBA	AD
Applications Demonstration Services	ADS
ADS Development	ADS_DEV
Complex Maintenance Repair and Overhaul	AHL
Hosting Manager(Obsolete)	AHM
Common Modules-AK	AK
Alert	ALR
Approvals Management	AME
Fulfillment Services (Obsolete)	AMF
Marketing	AMS
Marketing Encyclopedia System	AMV
Internal Controls Manager	AMW
Sales Analysis	AN
Receivables	AR
Sales Foundation	AS
Sales Online	ASF
CRM Gateway for Mobile Devices	ASG
Sales Offline	ASL
Sales	ASN
Order Capture	ASO
Oracle Sales for Handhelds	ASP
TeleSales	AST
Application Utilities	AU
Global Accounting Engine	AX
Application Implementation	AZ
Banking	BANKING
Advanced Benefits	BEN
Customer Intelligence (obsolete)	BIC
eCommerce Intelligence	BIE
Sales Intelligence	BIL
Marketing Intelligence	BIM
Communications Intelligence	BIN
Applications BIS	BIS
Service Intelligence	BIV
Interaction Center Intelligence	BIX
Systems Intelligence	BIY
Utility Billing	BLC
Web Applications Desktop Integrator	BNE
Bills of Material	BOM
Balanced Scorecard	BSC
Telephony Manager	CCT
Oracle Clinical Data Repository	CDR
Cash Management	CE
Supplier Scheduling	CHV
APAC Consulting Localizations	CLA
EMEA Consulting Localizations	CLE
Japan Consulting Localizations	CLJ
LAD Consulting Localizations	CLL
Supply Chain Trading Connector for RosettaNet	CLN
Culinary Application	CLR
Incentive Compensation	CN
Core Banking	COREBANK
CPG – CDOA	CPGC
Capacity	CRP
Service	CS
Customer Care	CSC
Depot Repair	CSD
Asset Tracking	CSE
Field Service	CSF
Install Base	CSI
Field Service/Laptop	CSL
Field Service/Palm	CSM
Call Center	CSN
Spares Management	CSP
Scheduler	CSR
Support (obsolete)	CSS
Cost Management	CST
Clinical Transaction Base	CTB
Capital Resource Logistics – Assets	CUA
Revenue Accounting	CUC
Billing Connect (obsolete)	CUE
Capital Resource Logistics – Financials	CUF
Citizen Interaction Center	CUG
Network Logistics – Inventory	CUI
Network Logistics – NATS (obsolete)	CUN
Network Logistics – Purchasing	CUP
Mass Market Receivables for Comms	CUR
Network Logistics	CUS
Custom Development	CUSTOM
Configurator	CZ
CADView-3D	DDD
Demand Signal Repository	DDR
Demo Order Entry (AOL Class)	DEM
Development	DNA
Document Managment and Collaboration	DOM
Oracle Price Protection	DPP
DateTrack	DT
Obsolete Process Operations	DUMMY_GMO
SEM Exchange (obsolete)	EAA
Enterprise Asset Management	EAM
e-Commerce Gateway	EC
XML Gateway	ECX
E-Records	EDR
Advanced Product Catalog	EGO
Environment Management System	EMS
Engineering	ENG
Product Intelligence	ENI
Value Based Management	EVM
Enterprise Performance Foundation	FEM
FastFormula	FF
Financial Intelligence	FII
Flow Manufacturing	FLM
Application Object Library	FND
Project Portfolio Analysis	FPA
Banking Center (obsolete)	FPT
Report Manager	FRM
Transportation Execution	FTE
Transfer Pricing	FTP
Financials Common Modules	FUN
Federal Financials	FV
Financial Consolidation Hub	GCS
US Federal Human Resources	GHR
Process Manufacturing Systems	GMA
Process Manufacturing Product Development	GMD
Process Manufacturing Process Execution	GME
Process Manufacturing Financials	GMF
Process Manufacturing Inventory	GMI
Process Manufacturing Logistics	GML
Manufacturing Execution System for Process Manufacturing	GMO
Process Manufacturing Process Planning	GMP
Grants Accounting	GMS
Process Manufacturing Portal	GMW
Genealogy Intelligence	GNI
Process Manufacturing Regulatory Management	GR
Healthcare	HCA
iHCConnect	HCC
iHCIntegrate	HCN
Healthcare Intelligence	HCP
Healthcare Terminology Server	HCT
Human Resources Intelligence	HRI
Time and Labor Engine	HXC
Time and Labor	HXT
iAssets	IA
Digital Asset Management	IAM
iMarketing (Obsolete)	IBA
Content Manager	IBC
iStore	IBE
Bill Presentment & Payment	IBP
iAuction	IBT
iSupport	IBU
Oracle Web Analytics	IBW
Payments	IBY
Oracle iProcurement	ICX
Interaction Blending	IEB
Advanced Outbound Telephony	IEC
Email Center	IEM
Interaction Center Technology	IEO
Predictive	IEP
Scripting	IES
Call Center Connectors	IET
Universal Work Queue	IEU
IVR Integrator	IEV
Collections	IEX
Contract Commitment	IGC
Financial Aid	IGF
Public Sector Financials International	IGI
Student System	IGS
Grants Proposal	IGW
Customers Online	IMC
iMeeting (obsolete)	IMT
Oracle Landed Cost Management	INL
Institutional Banking	INSTITBANK
Inventory	INV
Capital Resource Logistics – Projects	IPA
Product Development (obsolete)	IPD
Oracle Imaging Process Management	IPM
iRecruitment	IRC
Supply Chain Intelligence	ISC
iSettlement	ISX
Information Technology Audit	ITA
Internet Procurement Enterprise Connector	ITG
Oracle E-Business Suite Diagnostics	IZU
Asia/Pacific Localizations	JA
European Localizations	JE
Regional Localizations	JG
Latin America Localizations	JL
Supply Chain Localizations	JMF
CRM Foundation	JTF
Mobile Application Foundation	JTM
CRM Self Service Administration	JTS
Loans	LNS
Controlled Availability Product(Obsolete)	ME
Manufacturing	MFG
Mobile Applications for Inventory Management	MIA
Media Interactive	MIV
Mobile Quality Applications	MQA
Master Scheduling/MRP	MRP
Advanced Supply Chain Planning	MSC
Demand Planning	MSD
Constraint Based Optimization	MSO
Inventory Optimization	MSR
Transportation Planning	MST
Oracle Manufacturing Operations Center	MTH
Mobile Applications	MWA
Oracle Applications Manager	OAM
Data Query	ODQ
Order Entry	OE
Assets	OFA
Contracts for Subscriptions (Obsolete)	OKB
Contracts Core	OKC
Project Contracts	OKE
Contracts Intelligence	OKI
Lease and Finance Management	OKL
Contracts for Sales (Obsolete)	OKO
Contracts for Procurement (Obsolete)	OKP
Contracts for Rights (Obsolete)	OKR
Service Contracts	OKS
Royalty Management	OKT
Contracts Integration	OKX
Order Management	ONT
Operations Intelligence	OPI
Learning Management	OTA
University Curriculum	OUC
Trade Management	OZF
Trade Planning (Obsolete)	OZP
iClaims (Obsolete)	OZS
Projects	PA
Payroll	PAY
Budgeting and Planning	PBR
Human Resources	PER
Oracle Profitability Manager	PFT
Project Intelligence	PJI
Project Manufacturing	PJM
Process Manufacturing Intelligence	PMI
Property Manager	PN
Purchasing	PO
Purchasing Intelligence	POA
Exchange	POM
Sourcing	PON
iSupplier Portal	POS
Public Sector HR	PQH
Public Sector Payroll	PQP
Progress Custom	PRGC
Proposals	PRP
Public Sector Financials	PSA
Public Sector Budgeting	PSB
Labor Distribution	PSP
Public Sector Receivables	PSR
Patch Tracking System	PTX
Partner Management	PV
Quality	QA
Quoting	QOT
Advanced Pricing	QP
Oracle Deal Management	QPR
Risk Management	QRM
Regulatory Capital Manager (obsolete)	RCM
Application Report Generator	RG
Advanced Planning Foundation(obsolete)	RHX
Release Management Integration Kit (Obsolete)	RLA
Release Management	RLM
Risk Manager	RMG
Retail Core	RRC
Site Management	RRS
Applications Shared Technology	SHT
Payables	SQLAP
General Ledger	SQLGL
SSP	SSP
System Administration	SYSADMIN
test	TEST
Automotive	VEA
Automotive Integration Kit (Obsolete)	VEH
Work in Process	WIP
Manufacturing Mobile Applications	WMA
Warehouse Management	WMS
Manufacturing Scheduling	WPS
Shipping Execution	WSH
Shop Floor Management	WSM
XML Publisher	XDO
Provisioning	XDP
Subledger Accounting	XLA
Legal Entity Configurator	XLE
Service Assurance for Communications	XNA
Oracle Telecommunications Billing Integrator	XNB
Sales for Communications (Obsolete)	XNC
Install Base Intelligence (Obsolete)	XNI
Marketing for Communications (Obsolete)	XNM
Number Portability	XNP
Service for Communications (obsolete)	XNS
TeleBusiness for Telecom/Utilities	XNT
Treasury	XTR
Financial Analyzer	ZFA
Enterprise Planning and Budgeting	ZPB
Sales Analyzer	ZSA
E-Business Tax	ZX

Start and stop Mobile Web Application (MWA) services in R12

Start MWA Services
Login from Application binary owner

$ . <APPL_TOP>/APPS<CONTEXT_NAME>.env
$ cd $ADMIN_SCRIPT_HOME

$ ./mwactl.sh -login apps/<apps_pwd> start
OR
$ ./mwactlwrpr.sh start apps/<apps_pwd>

Stop MWA Services
Login from Application binary owner

$ . <APPL_TOP>/APPS<CONTEXT_NAME>.env
$ cd $ADMIN_SCRIPT_HOME

$ ./mwactl.sh -login apps/<apps_pwd> stop
OR
$ ./mwactlwrpr.sh stop apps/<apps_pwd>

Check Whether MWA Services are running

$ ps -ef |grep mwa

Find out port number of MWA services
Login from Application binary owner

$ . <APPL_TOP>/APPS<CONTEXT_NAME>.env
$ grep mwa $CONTEXT_FILE