Cloning Oracle Applications Release 12 with Rapid Clone

Prerequisite Tasks

Before cloning, prepare the source system by applying any required patches and running AutoConfig.

1. Verify OS requirements on target system
   Before cloning to a new server, ensure the target system meets all the requirements for Oracle Applications Release 12 stated on the Oracle Applications Release Notes, and on the Oracle Applications Installation and Upgrade Notes for each Platform.
2. Apply the latest AD patch
   Apply patch 6510214 (R12.AD.A.DELTA.4) or higher.
3. Apply the latest AutoConfig template patch
   Update the Oracle Applications file system with the latest AutoConfig template files by applying the TXK AutoConfig Template rollup patch to all application tier server nodes.
4. Apply the latest Rapid Clone patches
   Update the Oracle Applications file system with the latest Rapid Clone files by applying the following patches to all Applications nodes. For Release 12.0: 5484000 Oracle E-Business Suite 12.0.2 Release Update Pack (RUP2) or higher and 9171651:R12.OAM.A - 12.0 RAPIDCLONE CONSOLIDATED FIXES JUL/2010
   For Release 12.1: 9171651:R12.OAM.B -12.1 RAPIDCLONE CONSOLIDATED FIXES JUL/2010
   Other Patches (All releases): 8246709 - Required for Microsoft Windows if using OracleAS 10.1.3.4. This patch must be re-applied to the OracleAS 10.1.3.4 ORACLE_HOME before every cloning operation.
5. Run AutoConfig on the application tiers
6. Synchronize appsutil on the database tier nodes
7. Run AutoConfig on the database tier
8. Maintain snapshot information

Log in to each application tier node as the APPLMGR user, and run "Maintain Snapshot information" in AD Administration.  

Cloning Tasks

1. Prepare the source system
   Execute the following commands to prepare the source system for cloning:
1. Prepare the source system database tier for cloning
   Log on to the source system as the ORACLE user, and run the following commands:

$ cd [RDBMS ORACLE_HOME]/appsutil/scripts/[CONTEXT_NAME]
$ perl adpreclone.pl dbTier

2. Prepare the source system application tier for cloning
   Log on to the source system as the APPLMGR user, and run the following commands on each node that contains an APPL_TOP:

$ cd [INST_TOP]/admin/scripts
$ perl adpreclone.pl appsTier

2. Copy the source system to the target system
   Copy the application tier file system from the source Applications system to the target node by executing the following steps in the order listed. Ensure the application tier files copied to the target system are owned by the target APPLMGR user, and that the database node files are owned by the target ORACLE user.For example: cd /target_dest_dir/db cp -RH /source_dir/db/*. Alternatively, the tar command can be used to compress the directories into a temporary staging area. 

1. Copy the application tier file system
   Log on to the source system application tier nodes as the APPLMGR user and shut down the application tier server processes. Copy the following application tier directories from the source node to the target application tier node:
• [APPL_TOP]
• [COMMON_TOP]
• Applications Technology Stack:
• [OracleAS Tools ORACLE_HOME]
• [OracleAS Web IAS_ORACLE_HOME]
2. Copy the database node file system
   Log on to the source system database node as the ORACLE user, and then:

1.     Perform a normal shutdown of the source system database

2.     Copy the database (.dbf) files from the source system to the target system

3.     Copy the source database ORACLE_HOME to the target system

4.     Start the source Applications system database and application tier processes

Configure the target system

Run the following commands to configure the target system. You will be prompted for specific target system values such as SID, paths, and ports.

Configure the target system database server
Log on to the target system as the ORACLE user and enter the following commands:

$ cd [RDBMS ORACLE_HOME]/appsutil/clone/bin
$ perl adcfgclone.pl dbTier

Configure the target system application tier server nodes
Log on to the target system as the APPLMGR user and enter the following commands:

$ cd [COMMON_TOP]/clone/bin
$ perl adcfgclone.pl appsTier

Finishing Tasks

This section lists tasks that may be necessary, depending on your implementation and the intended use of the cloned system.

1. Update profile options
   Rapid Clone updates only site level profile options. If any other profile options are set to instance specific values, you must update them manually.
2. Update printer settings
   If the new cloned system needs to utilize different printers, update the target system with the new printer settings now.
3. Update Workflow configuration settings
   Cloning an Oracle Applications instance will not update the host and instance-specific information used by Oracle Workflow. Review the tables and columns listed below to check for any instance-specific data in the Workflow configuration on the target system.

Workflow configuration settings

Table Name	Column Name	Column Value Details
WF_NOTIFICATION_ATTRIBUTES	TEXT_VALUE	Value starts with http://[old web host]: Update to new web host.
WF_ITEM_ATTRIBUTE_VALUES	TEXT_VALUE	Value starts with "http://[old web host]: Update to new web host.
WF_SYSTEMS	GUID	Using the Workflow Administrator Web Applications responsibility, create a new system defined as the new global database name.
WF_SYSTEMS	NAME	Replace value with the database global name.
WF_AGENTS	ADDRESS	Update database link with the new database global name.
FND_FORM_FUNCTIONS	WEB_HOST_NAME	Update with the new web host name.
FND_FORM_FUNCTIONS	WEB_AGENT_NAME	Update to point at the new PL/SQL listener name.
FND_CONCURRENT_REQUESTS	LOGFILE_NAME	Update with the correct path to the logfile directory.
FND_CONCURRENT_REQUESTS	OUTFILE_NAME	Update with the new directory path on the target system.

4. Verify the APPLCSF variable setting
   Source the APPS environment and review that the variable APPLCSF (identifying the top-level directory for concurrent manager log and output files) points to a suitable directory. To modify it, change the value of the s_applcsf variable in the context file and then run AutoConfig.
5. Update the SESSION_COOKIE_DOMAIN value in ICX_PARAMETERS
   If the target system is in a different domain name than the source system and SESSION_COOKIE_DOMAIN was not null in the source system, update that value to reflect the new domain name.
6. Re-Implement SSL and SSO configuration
   If the Source System was SSL or SSO enabled, reconfigure the Target by following the SSL/SSO documentation.

Advanced Cloning Options

Refreshing a Target System

You may need to refresh the target system periodically to synchronize it with changes to the source system.

1. Prepare the source system.
2. Copy the source system to the target system.
• If the application tier file system if the APPL_TOP, COMMON_TOP, or technology stack needs to be refreshed, copy the portion of the application tier file system that has been updated
• If the RDBMS ORACLE_HOME or the database needs to be refreshed, copy the database node file system. If refreshing the database, the ORACLE_HOME should be refreshed at the same time.
3. Configure the target system.
   Specify the existing target system context file when running adcfgclone.pl commands.
1. Configure the target system database server by logging on to the target system as the ORACLE user and entering the following commands to configure and start the database:

$cd [RDBMS ORACLE_HOME]/appsutil/clone/bin
perl adcfgclone.pl dbTier [Database target context file]

Where Database target context file is [RDBMS ORACLE_HOME]/appsutil/[Target CONTEXT_NAME].xml

2. Configure the target system application tier server nodes by logging on to the target system as the APPLMGR user and entering the following commands:

$ cd [COMMON_TOP]/clone/bin
$ perl adcfgclone.pl appsTier [APPL_TOP target context file]

Where APPL_TOP target context file is [INST_TOP]/appl/admin/[Target CONTEXT_NAME].xml

4. Perform the standard finishing tasks mentioned above.

Cloning a Multi-Node System

This procedure allows the source system or target system to be a multi-node system. As of Release 12, all APPL_TOPs are unified APPL_TOPs. This means that all files required for all application tier services are installed on every application tier node. Thus, only one copy of the applications tier node files needs to be copied to the target system, regardless of whether a shared file system is being used on the source or target system. Multiple application tier nodes are distinguished from each other by the services running.

1. Perform the standard prerequisite tasks.
   Carry out these steps on all source and target nodes.
2. Carry out the cloning tasks mentioned above
   Prepare, copy and configure the cloned Applications system. When creating more than one application tier node on the target system, follow these steps:
• Perform a full clone (Prepare, copy and configure steps) of the database node and primary application tier node.
• To add shared application tier nodes on the target system, follow the instructions in My Oracle Support Knowledge Document 384248.1, Section 4: Adding a node to a Shared Application Tier File System.
• To add non-shared application tier nodes, execute the copy and configure steps as on the primary node.
• Specify the services to start on each target Applications tier node when responding to the prompts during the configuration step.
3. Perform the required finishing tasks mentioned above.

Adding a New Node to an Existing System

You can use Rapid Clone to clone a node and add it to the existing Applications system, a process also known as scale up or scale out. The new node can run the same services as the source node, or different services. Follow the instructions in the Application tier part of Cloning Tasks.

1. Prepare the source system, copy it to the new node and configure it.
2. After adcfgclone.pl completes, source the Applications environment and run the following commands on the target system:

$ cd [COMMON_TOP]/clone/bin
$ perl adaddnode.pl

Note: After adding new nodes, do the set up for load balancing.

Cloning the Database Separately

Some situations require the database to be recreated separately, without using Rapid Clone. Typical scenarios are when system downtime is not feasible, or advanced database replication tools like RMAN are being used to copy the database in hot backup mode.

1. Log on to the target system as the ORACLE user
2. Configure the [RDBMS ORACLE_HOME]

$ cd [RDBMS ORACLE_HOME]/appsutil/clone/bin
$ perl adcfgclone.pl dbTechStack

3. Create the target database control files manually
   In this step, you copy and recreate the database using your preferred method, such as RMAN restore, Flash Copy, Snap View, or Mirror View.
4. Start the target database in open mode
5. Run the library update script against the database

$ cd [RDBMS ORACLE_HOME]/appsutil/install/[CONTEXT NAME]
$ sqlplus "/ as sysdba" @adupdlib.sql [libext]

Where [libext] should be set to 'sl' for HP-UX, 'so' for any other UNIX platform, or 'dll' for Windows.

6. Configure the target database
    The database must be running and open before performing this step.

$ cd [RDBMS ORACLE_HOME]/appsutil/clone/bin
$ perl adcfgclone.pl dbconfig [Database target context file]

Where Database target context file is: [RDBMS ORACLE_HOME]/appsutil/[Target CONTEXT_NAME].xml.

Note: The dbconfig option will configure the database with the required settings for the new target, but it will not recreate the control files.

Cheers!!!

RC-50013: FATAL: Instantiate driver did not complete successfully on DBTier

After upgradation from DB 10.2.0.4 to 11.2.0 in OEL 4.6 of Release 12.1.2 application, did the autoconfig and preclone process successfully in both db and apps tier.

While cloning to the target instance getting error from DbTier.

$perl adcfgclone.pl dbTier

[AutoConfig Error Report]
The following report lists errors AutoConfig encountered during each
phase of its execution.  Errors are grouped by directory and phase.
The report format is:
[filename]  [phase]  [return code where appropriate]


[APPLY PHASE]
AutoConfig could not successfully execute the following scripts:
Directory: /u01/oracle/AJI/db/tech_st/11.2.0/perl/bin/perl -I /u01/oracle/AJI/db/tech_st/11.2.0/perl/lib/5.8.3 -I /u01/oracle/AJI/db/tech_st/11.2.0/perl/lib/site_perl/5.8.3 -I /u01/oracle/AJI/db/tech_st/11.2.0/appsutil/perl /u01/oracle/AJI/db/tech_st/11.2.0/appsutil/clone
ouicli.pl   INSTE8_APPLY 255


AutoConfig is exiting with status 1


RC-50013: Fatal: Instantiate driver did not complete successfully.
/u01/oracle/AJI/db/tech_st/11.2.0/appsutil/driver/regclone.drv

I have resolved this issue by copying java folder to the appsutil folder. I have already installed the latest Java 6.0. The issue is caused by [DB oracle home]/appsutil not containing the jre directory

$cp -R /usr/java/jdk1.6.0_16/  $ORACLE_HOME/appsutil/jre

Again run the cloning process,
perl adcfgclone.pl dbTier

ApplyDatabase Completed Successfully.


Cheers!!!

Autoconfig Error on Appstier Tier: Context Value Management Failed

After clone, while running adautocfg.sh getting the following error.

Context Value Management will now update the Context file


Updating Context file...COMPLETED


Attempting upload of Context file and templates to database...Exception 
in thread "main" java.lang.AbstractMethodError: oracle.sql.LnxLibThin.lnxnuc([BILjava/lang/String;)
at oracle.sql.NUMBER.toInt(NUMBER.java:413) 
at oracle.jdbc.dbaccess.DBConversion.NumberBytesToInt(DBConversion.java: 2959) 
at oracle.jdbc.driver.OracleStatement.getIntValue(OracleStatement.java:4583) 
at oracle.jdbc.driver.OracleCallableStatement.getInt(OracleCallableStatement.java:635) 
at oracle.apps.ad.autoconfig.oam.InDbCtxFile.uploadCtx(InDbCtxFile.java: 216) 
at oracle.apps.ad.autoconfig.oam.CtxSynchronizer.uploadToDb(CtxSynchronizer.java:328) 
at oracle.apps.ad.tools.configuration.FileSysDBCtxMerge.updateDBCtx(FileSysDBCtxMerge.java:678) 
at oracle.apps.ad.tools.configuration.FileSysDBCtxMerge.updateDBFiles(FileSysDBCtxMerge.java:222) 
at oracle.apps.ad.context.CtxValueMgt.processCtxFile(CtxValueMgt.java:1586) 
at oracle.apps.ad.context.CtxValueMgt.main(CtxValueMgt.java:665) 
Error returned from java. Context Value Management was aborted 
ERROR: Context Value Management Failed. 
Terminate. 


The logfile for this session is located at: 
%APPL_TOP%\admin\%CONTEXT_NAME%\log\\adconfig.log 

The reason for this error is that JAVA_TOP/oracle/sql contain the following two files
LnxLibThin.class, 
LdxLibThin.class 
Need to remove and re-run autoconfig again.

As part of the 11.5.10 Maintenance Pack, these files should have been removed by adpatch
After successfully removed these two files, need to re-run the autoconfig from Applmgr user

Cheers!!!

What happening when executing adcfgclone.pl in DB Tier as well as Apps Tier?

adcfgclone.pl - This script will performs the clone configuration in the target machine.

During this configuration phase will reconfigure the copied instance after gathering the required information.

adchkutl.sh
The first step performed is the OS utility check (adchkutl.sh).
This ensures make, cc, ld and ar executables are available in the PATH.

CloneContext.class
This will create the Target Context file from the values entered on the screen.

All the questions, values entered, results of the port availability check etc will be logged in a log file located in same location as the adcfgclone.pl script:-
dbTier : $ORACLE_HOME/appsutil/clone/bin
appsTier(s) : $COMMON_TOP/clone/bin

Log files for this phase:-
dbTier: $TMP/CloneContext_xxxxxx.log

appsTier(s): $TMP/CloneContext_xxxxxx.log

Both Tiers: $TMP/adclonectx.err
Executing adcfgclone.pl will generate logs as follows:-

     dbTier: $ORACLE_HOME/appsutil/clone/bin/CloneContext_xxxxxx.log
     appsTier(s): $COMMON_TOP/clone/bin/CloneContext_xxxxxx.log
     Both Tiers: $TMP/adclonectx.err

If any of the variables in the Target system Context file appear to be incorrect, then check this log file to see how the incorrect value arose.

ApplyDBTier.class/ApplyAppsTier.class

Once the XML is created, or passed as a parameter on the command line, then the configuration phase begins.

This part will instantiate all the templates created during the preparation phase under:
$ORACLE_HOME/appsutil/template.

Templates to instantiate are located in driver file
$ORACLE_HOME/appsutil/driver/instconf.drv

Log files for this phase:-
dbTier: $ORACLE_HOME/appsutil/log/[Target Context Name]/ApplyDBTier_xxxxxx.log
appsTier(s): $APPL_TOP/admin/[Target Context Name]/log/ApplyAppsTier_xxxxxx.log

Relink the ORACLE_HOME(s)
The executables in the ORACLE_HOME(s) will be relinked during the configuration phase.
The commands used, and results of the relinking will displayed in log file :
$ORACLE_HOME/install/make.log

Create the Control files (DB Tier only)
Once the templates are instantiated, and the ORACLE_HOME is relinked, then the control files can be recreated.

The templates used here are located in
$ORACLE_HOME/appsutil/clone/data/stage/adcrdb.zip
The files in this zip file are used to create a SQL file called
adcrdbclone.sql

This above SQL script is called from a shell script
adcrdb.sh.

Both of the above scripts can be found under
$ORACLE_HOME/appsutil/install/$CONTEXT_NAME

If there are any problems creating the control files, then check the log files :
$ORACLE_HOME/appsutil/log/$CONTEXT_NAME/adcrdb_SID.txt
$ORACLE_HOME/appsutil/log/$CONTEXT_NAME/ApplyDBTier_xxxxxx.log

AutoConfig
The next phase will run AutoConfig using use the usual templates and drivers associated with AutoConfig on db or appsTier.

The following log files will contain the information on the results of running AutoConfig
$ORACLE_HOME/appsutil/log/$CONTEXT_NAME/ApplyDBTier_xxxxxx.log
$APPL_TOP/admin/$CONTEXT_NAME/log/ApplyAppsTier_xxxxxx.log

Update the oraInventory (OracleHomeCloner.class)
Once the Target instance is configured, it will update the Global Inventory will the locations of the new ORACLE_HOMES

Any problems with this step will be detailed in the log files:-
[path to oraInventory]/logs/OracleHomeCloner_MMDDHHMM.log
[path to oraInventory]/logs/apps[Year]-[Mon]-[Day]-[Hour]-[Min]-[Sec]-[AM/PM].log

What happening when executing adpreclone.pl in DB and Apps Tiers

What happening when executing adpreclone.pl in DB and Apps Tier?

adpreclone.pl - This is the preparation phase, will collects information about the source system, creates a cloning stage area, and generates templates and drivers. All of these are to reconfigure the instance on a Target machine.
Preclone will do the following:

Convert Symbolic links
All symbolic links pointing to a static path will be converted into relative paths

Create templates
Any files under the $ORACLE_HOME that contain system specific information, will be replicated and converted into a template. These templates are placed into the $ORACLE_HOME/appsutil/template directory.

Create driver(s)
A driver file, relating to these new templates is created called instconf.drv. This contains a list of all the templates and their locations, and the destination configuration files that these templates will create.
This driver file is called instconf.drv and is placed into directory
$ORACLE_HOME/appsutil/driver

Create Stage area
A clone stage is created containing the required java code and scripts to reconfigure the instace on the Target machine

Rapid Clone stage area:
dbTier : $ORACLE_HOME/appsutil/clone
appsTier(s) - $COMMON_TOP/clone

The stage area(s) consist of the following directories:-
 jre used to run the java code on the Target machine.
 bin contains the RapidClone scripts that can be run on the Target machine:-

•    adclone.pl is the main cloning script
•    adcfgclone.pl is used to configure the Target system, this calls adclone.pl
•    adclonectx.pl is used to clone a Source XML file manually
•    adaddnode.pl is used to add a new node to the Patch History tables
•    adchkutl.sh checks for existence of require O/S utils, cc, make, ar and ld

 jlib contains all the Rapid Clone java code, jdbc libraries etc
 context contains templates used for a Target XML file
 data (Database Tier only) contains the driver file, and templates used to generate the control file SQL script
 adcrdb.zip contains the template and list of datafiles on the Source
 addbhomsrc.xml contains information on the datafile mount points of the Source
 appl (Applications Tier only) this is used when merging appltops, i.e Multi-node to Single node cloning

Executing adpreclone.pl will create a log file:-
Rapid Clone:
dbTier : $ORACLE_HOME/appsutil/log/$CONTEXT_NAME/StageDBTier_xxxxxx.log
appsTier : $APPL_TOP/admin/$CONTEXT_NAME/log/StageAppsTier_xxxxxx.log

Once this adpreclone.pl step has been completed successfully, all the java .class files under the following directories should be identical to those under $JAVA_TOP/oracle :

RDBMS $ORACLE_HOME/appsutil/java/oracle
RDBMS $ORACLE_HOME/appsutil/clone/jlib/java/oracle
$COMMON_TOP/clone/jlib/java/oracle

Next post, What happening when executing adcfgclone.pl in DB Tier as well as Apps Tier?

After doing a clone all the concurrent managers do not start

After Doing a Clone All the Concurrent Managers Do Not Start
Sometimes after cloning, all the concurrent managers are not coming up. Finally I found that its unusual but in Oracle Application Release 11i and 12  also have the same problem.
The symptom of the issue is that, after doing a clone when attempting to start the concurrent manager using the below :

$COMMON_TOP/admin/scripts/
. adcmctl.sh stop apps/apps

Errors found in ICM logfile as
Could not initialize the Service Manager FNDSM__. Verify that has been registered for concurrent processing. 

The reason for this issue is that, FND_NODES table having incorrect node entries (old node from the Source Instance) and that no Service Manager was found for the existing node.

To Solve this issue,
1.  Stop all middle tier services including the concurrent managers.
2.  Stop the database.
3.  Start the database.
4.  Connect SQLPLUS as APPS user and run the following :
EXEC FND_CONC_CLONE.SETUP_CLEAN;
COMMIT;
EXIT;

5.  Run AutoConfig on all tiers, firstly on the DB tier and then the APPS tiers and webtiers to repopulate the required system tables.

6.  Connect to SQLPLUS as APPS user and run the following statement :

select CONCURRENT_QUEUE_NAME from FND_CONCURRENT_QUEUES where CONCURRENT_QUEUE_NAME like 'FNDSM%';

If the above SQL does not returning any value then please do the following:

Go to $FND_TOP/patch/115/sql

Connect SQLPLUS as APPS user and run the following script :

SQL> @afdcm037.sql;

(Note :  The same "afdcm037.sql" is used for Release 12 also ).
This script will create libraries for FNDSM and create managers for preexisting nodes.
Check again that FNDSM entries now exist:

select CONCURRENT_QUEUE_NAME from FND_CONCURRENT_QUEUES where CONCURRENT_QUEUE_NAME like 'FNDSM%';

7.  Go to cd $FND_TOP/bin

• $ adrelink.sh force=y "fnd FNDLIBR" 
• $ adrelink.sh force=y "fnd FNDSM" 
• $ adrelink.sh force=y "fnd FNDFS" 
• $ adrelink.sh force=y "fnd FNDCRM" 

8.  Run the CMCLEAN.SQL script from the referenced note below (don't forget to commit).
9.  Start the middle tier services including your concurrent manager.
10.  Retest the issue

Also see the document from Metalink
NOTE:218893.1 - How to Create The Service Manager 'FNDSM' on Oracle Applications
NOTE:260887.1 - Steps to Clean Nonexistent Nodes or IP Addresses from FND_NODES
NOTE:134007.1 - CMCLEAN.SQL - Non Destructive Script to Clean Concurrent Manager Tables

Unable to connect to Oracle Applications Database 'afscpgcs' after cloning an APPS instance and changing APPS passwords

After successfully cloning an Apps instance and changing Apps passwords, getting error below in the cloned instance when trying to connect using Discoverer.

"Error: A connection error has occurred.
- OracleBi Discoverer was unable to authenticate using the password provided. This can happen due
to an invalid password or because the password was lost while using back, forward, or refresh in your browser. Please enter the password again to continue.
- Failed to connect to database - Unable to connect to Oracle Applications database (afscpgcs)".

Solution to this issue is that
1. Set the Site-Level value for Profile option "Signon Password Case" temporarily to "insensitive".
2. Stop Applications tier services and concurrent tier services.
3. Use FNDCPASS to change APPS/APPLSYS password.
4. Run autoconfig on Apps tiers so that new apps password is propagated correctly.
5. Restart Applications tier and concurrent tier services.
6. Reset "Signon Password Case" back to "Sensitive" again.

Also please see the NOTE:399449.1 - After Changing the Applications or SSO User Password, Login to Discoverer Fails with Invalid Username/Password Error

Running Perl adcfgclone.pl Database Results In -Rmanstage Usage Error - R12.0.6

While attempting to clone to the 12.0.6 Test instance machine, the following error occurs.

Run the adupdlib.sql script against target database.

When running the "perl adcfgclone.pl dbconfig $ORACLE_HOME/appsutil/dba12_ltcsd141.xml",  getting Error related to -rmanstage:

/ appsutil/clone -rmanstage -rmantgtloc -srcdbname -showProgress -e
/d25/app/oracle/dba12/10.2.0/appsutil/dba12_ltcsd141.xml
-stage
/d25/app/oracle/dba12/10.2.0/appsutil/clone
-rmanstage
-rmantgtloc
-srcdbname
-showProgress
Option -rmanstage can only appear once"

### File/script version or Patch level and number? ###
adcfgclone Version 120.20.12000000.12

The reason for this problem

The variable rmanstage is called twice.

Bug 7364547 fixed in AD bin adclone.pl 120.26.12000000.16

To implement the solution, please execute the following steps:

Download and apply The Patch 7699109 to source system.
Patch will provide adclone.pl 120.26.12000000.16
Redo the cloning steps, including The preclone.

How to Configure OEM 10g DB Control after Clone

In this post, I would like to share the way of bringing up the database Enterprise Manager control 10g from cloned Oracle Apps test instance.
[oratest@testdb ~]$ emctl start dbconsole
TZ set to Asia/Dubai
OC4J Configuration issue. /oracle/testdb/10.2.0/oc4j/j2ee/OC4J_DBConsole_test.apps.local_TEST not found.
[oratest@testdb ~]$
In this stage, I know DB Console wasn't configured to this instance yet.
Then using emca I have tried to recreate the DB Control configuration files as follows:
[oratest@testdb ~]$ emca -config dbcontrol db
STARTED EMCA at May 11, 2010 7:16:31 AM
EM Configuration Assistant, Version 10.2.0.1.0 Production
Copyright (c) 2003, 2005, Oracle.  All rights reserved.
Enter the following information:
Database SID: TEST
Listener port number: 1521
Password for SYS user:
Password for DBSNMP user:
Password for SYSMAN user:
Email address for notifications (optional):
Outgoing Mail (SMTP) server for notifications (optional):
---------------------------
You have specified the following settings
Database ORACLE_HOME ................ /oracle/testdb/10.2.0
Database hostname ................ test.apps.local
Listener port number ................ 1521
Database SID ................ TEST
Email address for notifications ...............
Outgoing Mail (SMTP) server for notifications ...............
------------------------------
Do you wish to continue? [yes(Y)/no(N)]: y
May 11, 2010 7:18:34 AM oracle.sysman.emcp.EMConfig perform
INFO: This operation is being logged at /oracle/testdb/10.2.0/cfgtoollogs/emca/TEST/emca_2010-05-11_07-16-31-AM.log.
May 11, 2010 7:18:41 AM oracle.sysman.emcp.util.DBControlUtil startOMS
INFO: Starting Database Control (this may take a while) ...
May 11, 2010 7:18:50 AM oracle.sysman.emcp.EMDBPostConfig performConfiguration
INFO: Database Control started successfully
May 11, 2010 7:18:50 AM oracle.sysman.emcp.EMDBPostConfig
performConfiguration
INFO: >>>>>>>>>>> The Database Control URL is http://test.apps.local:5500/em <<<<<<<<<<<
Enterprise Manager configuration completed successfully
FINISHED EMCA at May 11, 2010 7:18:50 AM
[oratest@testdb ~]$
When I have tried to opened the DB Console using http://test.apps.local:5500/em, getting the following error message
error:
java.lang.Exception: Exception in sending Request :: null

This will not solve this issue, the right method of configuration of Enterprise Manager Control 10g is as follows. After the cloning procedure completed in db tier, do these steps one bye one.
There are two issues need to fix in the env_sysman.mk file, then need to relink manually for Agent component



1.Stop DB Console: 

emctl stop dbconsole 

2. cd $ORACLE_HOME/sysman/lib 

  

3. Edit env_sysman.mk and look for "JRE_LIB_DIR" 
In my instance JRE_LIB_DIR was populated with the old environment name.. 
I changed it from: 
JRE_LIB_DIR=/oracle/prod/db/proddb/10.2/jdk/jre/lib/i386 
To: 
JRE_LIB_DIR=/oracle/test/db/testdb/10.2/jdk/jre/lib/i386


4. One more issue for env_sysman.mk, look for "$(LIBNMEM)  $(LIBNMER)" 
And change from: 

$(LIBNMEM)  $(LIBNMER) \
To: 
$(LIBNMEM)  $(LIBNMER) $(LIBNMO) \


5. Now relink the Agent component by issuing this command: 
make -f ins_sysman.mk agent


6. Recreate the DB Control configuration files once again, if already done it 
emca -config dbcontrol db 


Now Enterprise Manager control is configured perfectly and coming up as usual from cloned instance and I can able to see everything fine in that screen.

How to Change IP, Hostname, Domain and Ports - Oracle E-Business Suite

This post will help you to change of IP, hostname, domain and ports setting from single node instance of existing oracle e-business suite configuration. Most of them are aware of how to reconfigure an E-Business 11i system using AutoConfig, (i will explain the steps for autoconfig in next post) however certain configuration changes are not reflected in the configuration files. The Net Services Topology Data Model stores the entire topological information about a single Oracle Application instance. The data model stores information about each node in the Oracle Applications instance which is then used to generate the Net Service configuration files (for example tnsnames.ora). AutoConfig seeds the data model with relevant data.

The Net Services Topology Data Model stores the following information:
On the database tier: Hostname, Database SID, Database Name, Instance Name, TNS Descriptors.....
On the application tier: Hostname, FNDFS and FNDSM alias descriptors.........

This functionality is causing confusion for those who are trying to reconfigure their existing systems using AutoConfig. Changes such as hostname or Database port are not reflected in the configuration files after AutoConfig is run.

In Oracle E-Business Suite IP address is not dependent, so one can easily change the IP address of nodes without any effect on E-Business Suite.

But Hostname, Domain and Ports are not like that, Oracle E-Business Suite is dependent to these entries. So, we couldn’t change these values without co-ordination of E-Business Suite itself.
There are two methods available for changing Hostname, Domain and Ports of Oracle E-Business Suite. The first method, that will be explaining here, is using autoconfig and another method is by cloning. It is very easy to use autoconfig method to change hostname, domain and ports and use cloning method for changing other configurations of Oracle E-Business Suite.

In this post, assume that our E-Business Suite has just one node so that all Tiers and Servers reside on this node. For changing hostname, domain and ports of a multinode Oracle E-Business Suite, please refere metalink note: 341322.1 - How to change the hostname of an Applications Tier using AutoConfig, metalink note: 338003.1 - How to change the hostname and/or port of the Database Tier using AutoConfig  and Note: 391406.1 - How to get a clean Autoconfig Environment.


a. DB Tier: Deregister the current database server (Required)

As the database hostname and domain will be changed, the current database server node needs to be de-registered. To deregister the current database server node, run the following command as the owner of the Oracle RDBMS file system and current database instance :

SQL> select NAME, SERVER_TYPE from FND_APP_SERVERS, FND_NODES
where FND_APP_SERVERS.NODE_ID = FND_NODES.NODE_ID and
SERVER_TYPE='DB' and FND_NODES.NODE_NAME=UPPER('oldhost');

NAME            SERVER_TYPE
--------------- -----------
oldhost_DEVT_DB  DB

# cd $ORACLE_HOME/appsutil
# perl ./bin/adgentns.pl appspass= contextfile=./DEVT_oldhost.xml -removeserver

SQL> select NAME, SERVER_TYPE from FND_APP_SERVERS, FND_NODES
where FND_APP_SERVERS.NODE_ID = FND_NODES.NODE_ID and
SERVER_TYPE='DB' and FND_NODES.NODE_NAME=UPPER('oldhost');
no rows selected

b. DB Tier: Create a new Context file

You can create the new context file using whichever of the following methods:
a. Manuall Method:
# cd $ORACLE_HOME/appsutil
# cp PROD_oldhost.xml DEVT_newhost.xml
Edit DEVT_newhost.xml manually:
Replace all oldhost with newhost
Replace all olddomain with newdomain


b. Script Method:
# cd $ORACLE_HOME/appsutil
# perl ./bin/adclonectx.pl contextfile=./DEVT_oldhost.xml

Provide the values required for creation of the new Database Context file.
Do you want to use a virtual hostname for the target node (y/n) [n] ?:
Target hostname [oldhost]:newhost
Target System database name [DEVT]:
Target instance is a Real Application Cluster (RAC) instance (y/n) [n]:
Target system domain name [olddomain]:newdomain
Oracle OS User [oradevt]:
Oracle OS Group [dba]:
Target system RDBMS ORACLE_HOME directory [/oracle/devtdb/9.2.0]:
Target system utl_file accessible directories list [/usr/tmp]:
Number of DATA_TOP's on the target system [4]:1
Target system DATA_TOP 1:/oracle/devtdata
Do you want to preserve the Display set to oldhost:0.0 (y/n) [y] ?:n
Target system Display [newhost:0.0]:
Target system JAVA_TOP location [null]:/oracle/devtcomn/java
Do you want to preserve the port values from the source system on the target system (y/n) [y] ?:
Database port is 1521
New context path and file name [DEVT_newhost.xml]:
Creating the new Database Context file from :
/oracle/devtdb/9.2.0/appsutil/template/adxdbctx.tmp
The new database context file has been created :
/oracle/devtdb/9.2.0/appsutil/DEVT_newhost.xml

Note: The command above will create a new Context file of the format DEVT_newhost.xml in the current working directory.

c. APPS Tier: Deregister the current Applications server

As the Applications hostname and domain will be changed, the current Applications server node needs to be de-registered.
To deregister the current Applications server node, run the following command as the owner of the Oracle Applications file system and current database instance:

SQL> select NAME, SERVER_TYPE from FND_APP_SERVERS, FND_NODES
where FND_APP_SERVERS.NODE_ID = FND_NODES.NODE_ID and
SERVER_TYPE='APPS' and FND_NODES.NODE_NAME=UPPER('oldhost');
NAME              SERVER_TYP
----------------- ----------
oldhost_DEVT_APPS  APPS

# cd $APPL_TOP/admin
# perl $AD_TOP/bin/adgentns.pl appspass= contextfile=./DEVT_oldhost.xml -removeserver

SQL> select NAME, SERVER_TYPE from FND_APP_SERVERS, FND_NODES
where FND_APP_SERVERS.NODE_ID = FND_NODES.NODE_ID and
SERVER_TYPE='APPS' and FND_NODES.NODE_NAME=UPPER('oldhost');

no rows selected

d. APPS Tier: Create a new Context file

You can create the new context file using whichever of the following methods:


a. Manuall Method:
# cd $APPL_TOP/admin
# cp DEVT_oldhost.xml DEVT_newhost.xml
Edit DEVT_newhost.xml manually:
Replace all oldhost with newhost
Replace all olddomain with newdomain


b. Script Method:
# cd $APPL_TOP/admin
# perl $AD_TOP/bin/adclonectx.pl contextfile=./DEVT_oldhost.xml

Provide the values required for creation of the new APPL_TOP Context file.
Do you want to use a virtual hostname for the target node (y/n) [n] ?:
Target hostname [oldhost]:newhost
Target system database SID [DEVT]:
Target system domain name [olddomain]:newdomain
Username for the applications file system owner [appldevt]:
Group for the applications file system owner [dba]:
Target system database server node [oldhost]:newhost
Does the target system have more than one application tier server node (y/n) [n] ?:
Is the target system APPL_TOP divided into multiple mount points(y/n)[n]?:
Target system APPL_TOP mount point [/apps/devtappl]:
Target system COMMON_TOP directory [/apps/devtcomn]:
Target system 8.0.6 ORACLE_HOME directory [/apps/devtora/8.0.6]:
Target system iAS ORACLE_HOME directory [/apps/devtora/iAS]:
Do you want to preserve the Display set to oldhost:0.0 (y/n) [y] ?:n
Target system Display [newhost:0.0]:
Location of JDK 1.3.1 on the target system [/usr/java14]:
Do you want to preserve the port values from the source system on the target system (y/n) [y] ?:
Web Listener port is 8000
Complete port information available at /apps/devtappl/admin/out/DEVT_newhost/portpool.lst
New context path and file name [DEVT_newhost.xml]:
Creating the new APPL_TOP Context file from :
/apps/devtappl/ad/11.5.0/admin/template/adxmlctx.tmp
The new APPL_TOP context file has been created :
/apps/devtappl/admin/DEVT_newhost.xml

Note: The command above will create a new Context file of the format DEVT_newhost.xml in the current working directory.

e. APPS Tier: Shutdown the Application Tier Services

# $COMMON_TOP/admin/scripts/DEVT_oldhost/adstpall.sh apps/

f. Change the hostname and domain

Change the hostname and domain at OS level.
# hostname -s newhost
# edit /etc/hosts as follow:
OLD: IP-Address     oldhost.olddomain     oldhost
NEW: IP-Address     newhost.newdomain     newhost

Verify that the /etc/sysconfig/network file contains an entry that is similar to the following:
HOSTNAME=.
Check to see if the /etc/sysconfig/networking/profiles/default/network file exists. If it does, remove it.
If you changed any files in the previous steps, reboot the system.

g. DB Tier: Reseed the Net Services Topology Model

# cd $ORACLE_HOME/appsutil
# ./bin/adconfig.sh contextfile=./DEVT_newhost.xml

h. APPS Tier: Reseed the Net Services Topology Model

The Net Services Topology Model is automatically updated by running AutoConfig.
# $AD_TOP/bin/adconfig.sh contextfile=$APPL_TOP/admin/DEVT_newhost.xml appspass=

i. DB Tier: Shutdown the database and listener

# $ORACLE_HOME/appsutil/scripts/DEVT_oldhost/addbctl.sh stop
# $ORACLE_HOME/appsutil/scripts/DEVT_oldhost/addlnctl.sh stop DEVT

j. DB Tier: Start the listener and database
# $ORACLE_HOME/appsutil/scripts/DEVT_newhost/addlnctl.sh start DEVT
# $ORACLE_HOME/appsutil/scripts/DEVT_newhost/addbctl.sh start

k. APPS Tier: Start the Application Tier Services

# $COMMON_TOP/admin/scripts/DEVT_newhost/adstrtal.sh apps/

Sometimes you will encounter the following errors when you start the listener?
Error listening on: (ADDRESS=(PROTOCOL=TCP)(Host=newhost)(Port=1521))
TNS-12532: TNS:invalid argument
TNS-12560: TNS:protocol adapter error
TNS-00502: Invalid argument
IBM/AIX RISC System/6000 Error: 515: Error 515 occurred.

Please check $ORACLE_HOME/network/admin/DEVT_newhost/sqlnet.ora to insure it is using new values:
OLD: tcp.invited_nodes=(oldhost.olddomain, newhost.newdomain)
NEW: tcp.invited_nodes=(newhost.newdomain)

Another possible error is that in Jinitiator console when you try to start form applications?
java.lang.NoClassDefFoundError: org/apache/regexp/RESyntaxException
at oracle.ewt.lwAWT.lwText.LWTextField.getText(LWTextField.java:196)
at oracle.ewt.lwAWT.lwText.LWTextField._getDisplayString(LWTextField.java:787)

Please change $OA_HTML/bin/appsweb_DEVT_newhost.cfg for the value of ‘archive2′. It should be as follow:
archive2=,/OA_JAVA/regexp.jar

May be this also will encounter you when start Jinitiator console form applications?
java.lang.Error: RE internal error: Corrupt program
at org.apache.regexp.RE.internalError(RE.java:820)
at org.apache.regexp.RE.matchNodes(RE.java:1414)

Do make the following changes in JAVA_TOP
# cd $JAVA_TOP
# mv org org.back

Please send your comments....

11i Cloning (Multi Node to Single Node)

Merging from Multinode to Single node environments

In this post, what am explaining is that merging from multi-node to single node. Before cloning, prepare the Source System by applying patches and running AutoConfig. For a better understanding the concepts of cloning in Oracle application e-business suite 11i  first need to refer metalink note 230672.1 then apply all the pre-requisites patches and also refer metalink note: 233428.1 Section 4 for further steps.

You can use Rapid Clone to clone a Multi-Node System to a Single-Node System (or a Multi-Node System with fewer Nodes) by merging the APPL_TOPs file system together. For that follow this procedure carefully...

Source: DEVELOPMENT, this is a multi-node environment  with two servers
NODE A  192.168.1.101 (Forms and Web server TIER (Primary node))
NODE B  192.168.1.102 (Concurrent  + Database + Admin TIER)

Directory structure of application tier
/apps/devtappl
/apps/devtcomn
/apps/devtora

Directory structure of database tier looks like
/oracle/devtdb
/oracle/devtdata

Merging from mult-node to single node, always start merging from Form/WEB Tier node and then go to remaining Admin/Concurrent tier nodes

Target: TEST - Single node
Merge NODE 192.168.1.201  - Concurrent  + Database + Admin + Forms and Web server TIER
Here in this example, I am using the same directory structure mentioned above for merged tier too...

So the servers details of this example are
NODE A : 192.168.1.101 (Forms and Web server TIER) PRIMARY NODE Source
NODE B : 192.168.1.102 (Concurrent  + Database + Admin TIER) Source
Merge-NODE : 192.168.1.201 Concurrent  + Database + Admin + Forms and Web server TIER

I. Prepare the Source System

Execute the following commands to prepare the Source System for cloning.
a. Prepare the Source System Database Tier for cloning

Log on to the Source System (192.168.1.102 - NODE B) as the ORACLE user and run the following commands:

$cd [RDBMS ORACLE_HOME]/appsutil/scripts/[CONTEXT_NAME]
perl adpreclone.pl dbTier

b. Prepare the Source System Application Tier for cloning
Log on to the Source System (192.168.1.101 - NODE A) as the APPLMGR user and run the following commands on each Node that contains an APPL_TOP:

$cd [COMMON_TOP]/admin/scripts/[CONTEXT_NAME]
$perl adpreclone.pl appsTier merge

It will create clone directort in common_top/clone/[context_name]

Do the preclone procedure for other node also
Login as APPLMGR user in (192.168.1.102) NODE B

$cd [COMMON_TOP]/admin/scripts/[CONTEXT_NAME]
$perl adpreclone.pl appltop merge

It will create APPL_TOPs direcory in common_top/clone/appl/[context_name]

Next, copy following from Node A (Primary Forms/Web Tier) to Merge-NODE (Target Node) using linux tar command

On NODE A (192.168.1.101)
$cd /apps
$tar -cvf devtappl.tar devtappl  
$tar cvf devtcomn.tar devtcomn  
$tar -cvf devtora.tar devtora

Copy the Tar files to Target location (as an APPLMGR user)

$cd /apps 
$scp -rC devtappl.tar applmgr@192.168.1.201:/apps 
$scp -rC devtcomn.tar applmgr@192.168.1.201:/apps
$scp -rC devtora.tar applmgr@192.168.1.201:/apps

And then extract these tar files using following command

$tar -xvf devtappl.tar 
$tar -xvf devtcomn.tar 
$tar -xvf devtora.tar

Copy the required files from NODE B (192.168.1.102) to Merge-NODE (TARGET 192.168.1.201)
$COMMON_TOP/clone/appl/[context_name] (NODE B)
to
$COMMON_TOP/clone/appl  (Merge-NODE)

Login as APPLMGR user in 192.168.1.102 (NODE B)

$cd /apps/devcomn/clone/appl
$scp -rC appl applscm@192.168.1.201:/apps/devtcomn/clone

From NODE B (Source)  to Merge-NODE (TARGET)
Copy the database tier file system
Copy the database (DBF) files from the source to the target system
Copy the source database ORACLE_HOME(9.2.0) to the target system

II. Prepare the Target System

a. Configure the database tier server node first
Login as ORACLE user in Merge-NODE (192.168.1.201) and use the following method if database was backup-ed using COLD BACKUP methology

$cd /appsutil/clone/bin 
$perl adcfgclone.pl dbTier

Use this method if database was backup-ed using HOT BACKUP methology

$cd /appsutil/clone/bin 
$perl adcfgclone.pl dbTechStack

Create the target database control files manually
Start the target system database in open mode

Run the library update script against the database

$cd /appsutil/install/ 
$sqlplus "/ as sysdba" @adupdlib.sql 

where is "sl" for HP-UX, "so" for any other UNIX/LINUX platform
Configure the target database (the database must be open)

$cd /appsutil/clone/bin 
$perl adcfgclone.pl dbconfig

Database and DB Listener should be up and running

b. Configure the application tier server nodes

Login as appl user 192.168.1.201 (Merge-NODE)

$ cd /clone/bin
$ perl adcfgclone.pl appsTier

c. Finishing tasks

Log in to the target system application tier node as the APPLMGR user.
Run the following tasks in adadmin for all products:

1. Generate JAR files 
2. Generate message files 
3. Relink executables 
4. Copy files to destination 

Depending on which tier you chose as the primary node, certain files may be missing. Run adadmin to verify files required at runtime. If any files are listed as missing files, you must manually copy them to the merged APPL_TOP from other nodes' APPL_TOP.

Remove the temporary directory /clone/appl to reduce disk space usage

Please leave your comments....