scsi_id returns nothing on OEL6 running on VMware

Today while I was trying to setup UDEV for an Oracle 12cR1 RAC environment, I found scsi_id returns nothing. After googling around, the solution is as below:

1. shutdown the VM
2. right click the VM, then left click 'Edit Settings'
3. click 'Options' tab
4. click on 'General', then 'Configuration Parameters'
5. click 'Add Row'
6. add a parameter 'disk.EnableUUID' and set it to 'True', click 'OK'

boot up the server and now scsi_id -gud /dev/sdx returns values.

Virus Scan of Documents Uploaded to PeopleSoft

Lately I was assigned a task to configure PeopleSoft Web Server with McAfee VirusScan Enterprise for Linux (which we already owned). It turned out that McAfee VSEL doesn't support ICAP, therefore it won't work with PeopleSoft. Then I was able to download a "Symantec Protection Engine for Cloud Services" trial version and get it to work with weblogic web server. The sample VirusScan.xml file that PeopleSoft provided with PIA installation worked like a charm in a tools 8.52 environment.

PeopleTools 8.52 and libpsio_dir.so

I recently upgraded our SA and HRMS's PeopleTools from 8.50.20 to 8.52.10, the upgrade itself was pretty straight forward and went smoothly. However, it broke the LDAP authentication. When you go to "PeopleTools -> Security -> Directory -> Configure Directory", click the "search" button, click the Directory ID, then click the "Test Connectivity" tab, it just hangs there. Under the "Directory Setup" tab, all settings were confirmed correct. After some research, it turns out the problem was PS_HOME/bin/libpsio_dir.so. It appears PeopleTools 8.52 now delivers libpsio_dir.so under PS_HOME/bin/interfacedrivers, if the old  8.50.20 PS_HOME/bin/ libpsio_dir.so still exists, it will get called instead of the new 8.52.10 PS_HOME/bin/interfacedrivers/libpsio_dir.so, thus causing the "hang" issue. So remember to remove the old libpsio_dir.so after the upgrade is done.

Setup PeopleTools 8.52 PIA

Yesterday I was setting up a PIA for an CRM9.1 Demo environment and ran into below warning:


This version of PeopleSoft PeopleTools requires a 64-bit Oracle WebLogic
installation. The selected Oracle WebLogic is a 32-bit installation. Are you
sure you want to continue?

Since I didn't install the weblogic on the server, I had no idea if the weblogic that was pointed to is 32 bit or 64 bit. After tweaking a few things and still the same issue, I decided to reinstall weblogic. To ensure weblogic is 64 bit, the key is to use 64 bit java to install weblogic. Below is how to check if your java is 32 bit or 64 bit:

./java -d64 -version
Running a 64-bit JVM is not supported on this platform.

This means this java is not 64 bit.

 ./java -d64 -version
java version "1.6.0_20"
Java(TM) SE Runtime Environment (build 1.6.0_20-b02)
Oracle JRockit(R) (build R28.1.0-123-138454-1.6.0_20-20101014-1350-linux-x86_64, compiled mode)

This means this java is 64 bit.

Once I reinstalled weblogic with 64 bit java, the warning during setting up 8.52 PIA disappeared.

RMAN Duplication from Active RAC database to non-RAC database

I ran into below error when trying to RMAN duplicate from an active RAC database to a non-RAC database, the issue turned out to be oracle version mismatch between Target and Auxiliary. In my case, both Target and Auxiliary are 11.2.0.2.3, however, PSU 11.2.0.2.3 has 2 portions, one for GI_HOME and one for RDBMS_HOME. On my RAC cluster, both portions of PSU 11.2.0.2.3 were applied, while on my non-RAC host, only database portion of PSU 11.2.0.2.3 was applied. I ended up installing a new RDBMS_HOME and applying both portions of PSU 11.2.0.2.3 to this new RDBMS_HOME, the issue resolved by using this new RDBMS_HOME.

copying current control file
Oracle instance started
Segmentation fault

Another issue I ran into was as below, I was puzzled for a little bit because I know my Auxiliary's SYS password was correct and I was able to use it to startup/shutdown the instance. I had to change my Auxiliary's SYS password to be identical to my Target's SYS password. Once this was done, all went well.

contents of Memory Script:
{
   sql clone "alter system set  db_name =
 ''SA9TOLD'' comment=
 ''Reset to original value by RMAN'' scope=spfile";
   sql clone "alter system reset  db_unique_name scope=spfile";
   shutdown clone immediate;
}
executing Memory Script

sql statement: alter system set  db_name =  ''SA9TOLD'' comment= ''Reset to original value by RMAN'' scope=spfile

sql statement: alter system reset  db_unique_name scope=spfile

Oracle instance shut down
released channel: ch1
released channel: ch2
released channel: ch3
released channel: ch4
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of Duplicate Db command at 11/09/2012 13:27:49
RMAN-05501: aborting duplication of target database
RMAN-03015: error occurred in stored script Memory Script
RMAN-03009: failure of backup command on ch1 channel at 11/09/2012 13:27:39
ORA-17629: Cannot connect to the remote database server
ORA-17627: ORA-01017: invalid username/password; logon denied
ORA-17629: Cannot connect to the remote database server

Log Corruption on Standby Database.

Oracle 11.2.0.2 on Oracle Enterprise Linux 6.1, kernel bug caused log corruption on standby database. It was not a certified configuration by Oracle (historical reason beyond my control). Upgraded to OEL 6.2, log corruption disappeared. I had to use below note to roll forward standby database at times before the kernel was upgraded.

Steps to perform for Rolling forward a standby database using RMAN incremental backup when primary and standby are in ASM filesystem [ID 836986.1]

This worked well in my Non-ASM filesystem with OMF.

Run 10.2.0.3 and 11.1.0.7 RDBMS on 11.2.0.2 Grid Infrastructure

Recently I've been tasked to a project that needs to run 10.2.0.3 and 11.1.0.7 databases on the latest 11.2.0.2 Grid Infrastructure, I thought that would be easy, as the latest Grid Infrastructure should support the older RDBMS well, but it turns out it's not as easy as I thought. Here are the notes that describe what needs to be done to make it work:

1. Install 10.2.0.3 RDBMS: I got an error stating that it's not a correct oracle clusterware version, I ignored this error and the installation went just fine.

2. Create 10.2.0.3 database using dbca: I ran into "ORA-29702: error occurred in Cluster Group Service operation", had to "crsctl pin css -n node1 node2" to get past this, then I got "DBCA could not startup the ASM instance configured on this node. To processd with database creation using ASM you need the ASM instance to be up and running. Do you want to recreate the ASM instance on this node?" I applied patch 8288940 to get past this. Then I got "Encountered file error when copying listeners from home=/opt/app/11.2.0/grid", had to create a symbolic link "listener.ora" under the RDBMS $ORACLE_HOME/network/admin to point to the endpoints_listener.ora under $GI_HOME/network/admin, and setup TNS_ADMIN variable to point to $ORACLE_HOME/network/admin. Once all these are done, I was finally able to create databases using dbca.

3. To use srvctl to start/stop 10.2.0.3 database, I ran into "/opt/app/oracle/product/10.2.0/db_1/jdk/jre/bin/java: error while loading shared libraries: libpthread.so.0: cannot open shared object file: No such file or directory". I had to comment out "LD_ASSUME_KERNEL" in $ORACLE_HOME/bin/srvctl to fix this issue.

4. Install 11.1.0.7 RDBMS: This worked flawlessly.

5. Create 11.1.0.7 database using dbca: Same issues with creating 10.2.0.3 database, apply the same fixes, I was able to create the database successfully, but at the end it fails to start the database with error PRKP-1001 and CRS-0215. However you can start and stop the database in SQL/Plus.

6. To use srvctl to start/stop 11.1.0.7 database, I ran into the same error PRKP-1001 and CRS-0215. I had to apply patch 9294495 to fix it.