Run 10.2.0.3 and 11.1.0.7 RDBMS on 11.2.0.2 Grid Infrastructure

Recently I've been tasked to a project that needs to run 10.2.0.3 and 11.1.0.7 databases on the latest 11.2.0.2 Grid Infrastructure, I thought that would be easy, as the latest Grid Infrastructure should support the older RDBMS well, but it turns out it's not as easy as I thought. Here are the notes that describe what needs to be done to make it work:

1. Install 10.2.0.3 RDBMS: I got an error stating that it's not a correct oracle clusterware version, I ignored this error and the installation went just fine.

2. Create 10.2.0.3 database using dbca: I ran into "ORA-29702: error occurred in Cluster Group Service operation", had to "crsctl pin css -n node1 node2" to get past this, then I got "DBCA could not startup the ASM instance configured on this node. To processd with database creation using ASM you need the ASM instance to be up and running. Do you want to recreate the ASM instance on this node?" I applied patch 8288940 to get past this. Then I got "Encountered file error when copying listeners from home=/opt/app/11.2.0/grid", had to create a symbolic link "listener.ora" under the RDBMS $ORACLE_HOME/network/admin to point to the endpoints_listener.ora under $GI_HOME/network/admin, and setup TNS_ADMIN variable to point to $ORACLE_HOME/network/admin. Once all these are done, I was finally able to create databases using dbca.

3. To use srvctl to start/stop 10.2.0.3 database, I ran into "/opt/app/oracle/product/10.2.0/db_1/jdk/jre/bin/java: error while loading shared libraries: libpthread.so.0: cannot open shared object file: No such file or directory". I had to comment out "LD_ASSUME_KERNEL" in $ORACLE_HOME/bin/srvctl to fix this issue.

4. Install 11.1.0.7 RDBMS: This worked flawlessly.

5. Create 11.1.0.7 database using dbca: Same issues with creating 10.2.0.3 database, apply the same fixes, I was able to create the database successfully, but at the end it fails to start the database with error PRKP-1001 and CRS-0215. However you can start and stop the database in SQL/Plus.

6. To use srvctl to start/stop 11.1.0.7 database, I ran into the same error PRKP-1001 and CRS-0215. I had to apply patch 9294495 to fix it.