Friday, August 19, 2011

Weblogic not start after creating the Domain.

After creating a domain using a RDBMS Security Store and starting WebLogic Server, we have reports that it cannot be started up with errors like the following:

psfmw_domain/servers/AdminServer/logs/AdminServer.log00006. Log messages will continue to be logged in /u100/app/IPSFMW/oripsfmw/oracle/Oracle/Middleware/user_projects/domains/ipsfmw_domain/servers/AdminServer/logs/AdminServer.log.>
<Aug 19, 2011 3:12:17 AM CDT> <Notice> <Log Management> <BEA-170019> <The server log file /u100/app/IPSFMW/oripsfmw/oracle/Oracle/Middleware/user_projects/domains/ipsfmw_domain/servers/AdminServer/logs/AdminServer.log is opened. All server side log events will be written to this file.>
<Aug 19, 2011 3:12:23 AM CDT> <Error> <Security> <BEA-090870> <The realm "myrealm" failed to be loaded: weblogic.security.service.SecurityServiceException: com.bea.common.engine.ServiceInitializationException: weblogic.security.spi.ProviderInitializationException: A failure occurred attempting to load LDIF for provider Authorizer from file /u100/app/IPSFMW/oripsfmw/oracle/Oracle/Middleware/wlserver_10.3/server/lib/XACMLAuthorizerInit.ldift..
weblogic.security.service.SecurityServiceException: com.bea.common.engine.ServiceInitializationException: weblogic.security.spi.ProviderInitializationException: A failure occurred attempting to load LDIF for provider Authorizer from file /u100/app/IPSFMW/oripsfmw/oracle/Oracle/Middleware/wlserver_10.3/server/lib/XACMLAuthorizerInit.ldift.
        at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initializeRealm(CommonSecurityServiceManagerDelegateImpl.java:465)
        at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.loadRealm(CommonSecurityServiceManagerDelegateImpl.java:840)
        at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initializeRealms(CommonSecurityServiceManagerDelegateImpl.java:869)
        at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initialize(CommonSecurityServiceManagerDelegateImpl.java:1028)
        at weblogic.security.service.SecurityServiceManager.initialize(SecurityServiceManager.java:873)
        Truncated. see log file for complete stacktrace
Caused By: com.bea.common.engine.ServiceInitializationException: weblogic.security.spi.ProviderInitializationException: A failure occurred attempting to load LDIF for provider Authorizer from file /u100/app/IPSFMW/oripsfmw/oracle/Oracle/Middleware/wlserver_10.3/server/lib/XACMLAuthorizerInit.ldift.
        at com.bea.common.engine.internal.ServiceEngineImpl.findOrStartService(ServiceEngineImpl.java:365)
        at com.bea.common.engine.internal.ServiceEngineImpl.findOrStartService(ServiceEngineImpl.java:315)
        at com.bea.common.engine.internal.ServiceEngineImpl.lookupService(ServiceEngineImpl.java:257)
        at com.bea.common.engine.internal.ServicesImpl.getService(ServicesImpl.java:72)
        at weblogic.security.service.CSSWLSDelegateImpl.getService(CSSWLSDelegateImpl.java:155)
        Truncated. see log file for complete stacktrace
Caused By: weblogic.security.spi.ProviderInitializationException: A failure occurred attempting to load LDIF for provider Authorizer from file /u100/app/IPSFMW/oripsfmw/oracle/Oracle/Middleware/wlserver_10.3/server/lib/XACMLAuthorizerInit.ldift.
        at com.bea.common.store.bootstrap.internal.BootStrapServiceImpl.loadFullLDIFTemplate(BootStrapServiceImpl.java:910)
        at com.bea.common.store.bootstrap.internal.BootStrapServiceImpl.loadLDIFTemplate(BootStrapServiceImpl.java:688)
        at com.bea.common.store.bootstrap.internal.BootStrapServiceImpl.loadLDIFXACMLAuthorizerTemplate(BootStrapServiceImpl.java:176)
        at com.bea.common.store.bootstrap.internal.BootStrapServiceImpl.loadLDIFXACMLAuthorizerTemplate(BootStrapServiceImpl.java:160)
        at com.bea.common.security.internal.service.BootStrapServiceImpl.loadLDIFXACMLAuthorizerTemplate(BootStrapServiceImpl.java:106)
        Truncated. see log file for complete stacktrace
Caused By: <openjpa-1.1.1-SNAPSHOT-r422266:965591 fatal store error> kodo.jdo.FatalDataStoreException: The transaction has been rolled back.  See the nested exceptions for details on the errors that occurred.
        at org.apache.openjpa.kernel.BrokerImpl.newFlushException(BrokerImpl.java:2170)
        at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerImpl.java:2017)
        at org.apache.openjpa.kernel.BrokerImpl.flushSafe(BrokerImpl.java:1915)
        at org.apache.openjpa.kernel.BrokerImpl.beforeCompletion(BrokerImpl.java:1833)
        at org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime.java:81)
        Truncated. see log file for complete stacktrace
Caused By: <openjpa-1.1.1-SNAPSHOT-r422266:965591 nonfatal store error> kodo.jdo.DataStoreException: ORA-00942: table or view does not exist
 {prepstmnt 15960898 INSERT INTO BEAXACMLAP (DOMN, REALMN, TYPEN, CN, XVER, CTS, WCN, WCI, WXF, XD, XS, MTS) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) [params=(String) ipsfmw_domain, (String) myrealm, (String) Policies, (String) urn:bea:xacml:2.0:entitlement:resource:type@E@Fmbean@G, (String) 1.0, (null) null, (null) null, (null) null, (Blob) oracle.sql.BLOB@908f05, (InputStream) java.io.ByteArrayInputStream@9dc8f2, (String) 3, (Timestamp) 2011-08-19 03:12:23.296] [reused=0]} [code=942, state=42000]
FailedObject: com.bea.common.security.store.data.XACMLAuthorizationPolicy@1486b51
        at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:4207)
        at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4171)
        at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:102)
        at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:72)
        at kodo.jdbc.kernel.BatchingPreparedStatementManager.flushInternal(BatchingPreparedStatementManager.java:214)
        Truncated. see log file for complete stacktrace
Caused By: org.apache.openjpa.lib.jdbc.ReportingSQLException: ORA-00942: table or view does not exist
 {prepstmnt 15960898 INSERT INTO BEAXACMLAP (DOMN, REALMN, TYPEN, CN, XVER, CTS, WCN, WCI, WXF, XD, XS, MTS) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) [params=(String) ipsfmw_domain, (String) myrealm, (String) Policies, (String) urn:bea:xacml:2.0:entitlement:resource:type@E@Fmbean@G, (String) 1.0, (null) null, (null) null, (null) null, (Blob) oracle.sql.BLOB@908f05, (InputStream) java.io.ByteArrayInputStream@9dc8f2, (String) 3, (Timestamp) 2011-08-19 03:12:23.296] [reused=0]} [code=942, state=42000]
        at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConnectionDecorator.java:192)
        at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.access$700(LoggingConnectionDecorator.java:57)
        at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$LoggingConnection$LoggingPreparedStatement.executeUpdate(LoggingConnectionDecorator.java:866)
        at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:269)
        at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement.executeUpdate(JDBCStoreManager.java:1421)
        Truncated. see log file for complete stacktrace
>
<Aug 19, 2011 3:12:23 AM CDT> <Notice> <Security> <BEA-090082> <Security initializing using security realm myrealm.>
<Aug 19, 2011 3:12:23 AM CDT> <Critical> <WebLogicServer> <BEA-000362> <Server failed. Reason:

There are 1 nested errors:

weblogic.security.service.SecurityServiceRuntimeException: [Security:090399]Security Services Unavailable
        at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.doBootAuthorization(CommonSecurityServiceManagerDelegateImpl.java:916)
        at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initialize(CommonSecurityServiceManagerDelegateImpl.java:1050)
        at weblogic.security.service.SecurityServiceManager.initialize(SecurityServiceManager.java:873)
        at weblogic.security.SecurityService.start(SecurityService.java:141)
        at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
        at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
        at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
        at weblogic.work.ExecuteThread.run(ExecuteThread.java:178)

>
<Aug 19, 2011 3:12:23 AM CDT> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>
<Aug 19, 2011 3:12:23 AM CDT> <Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down>
<Aug 19, 2011 3:12:23 AM CDT> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FORCE_SHUTTING_DOWN>
-bash-3.2$

Cause:

The root cause is that the RDBMS Tables are not created in the Security Datastore.

Solution:

Before booting the domain, the RDBMS tables need to be created in the database:
Specify the same connection properties, including the credentials of the user who has access, the database URL, etc., as specified for that RDBMS during domain creation.
    Run the appropriate script to create RDBMS tables. There are a set of SQL scripts for creating/removing RDBMS tables under WL_HOME/server/lib: e.g., for Oracle DB, rdbms_security_store_oracle.sql is to create RRDBMS tables and rdbms_security_store_oracle_remove.sql is to remove these tables.

For details, please refer to "Create RDBMS Tables in the Security Datastore" part in http://www.oracle.com/pls/as1111/lookup?id=SECMG346

Ref Doc:

WebLogic Server Cannot Start Up with RDBMS Security Store (Doc ID 1327167.1)
WebLogic Server Does Not Start With Error: BEA-090870 The realm "myrealm" failed to be loaded (Doc ID 1328521.1)

Wednesday, August 3, 2011

Running AutoConfig On DB Node Reports: UnsatisfiedLinkError Exception Loading Native Library: njni11 while DB upgraded to 11.2.0.2

Recently While DB Upgraded from 11.1.0.7 to 11.2.0.2 , while running autoconfig below is the error


Context Value Management will now update the Context file
UnsatisfiedLinkError exception loading native library: njni11
java.lang.UnsatisfiedLinkError: njni11 (No such file or directory)

        Updating Context file...COMPLETED

        Attempting upload of Context file and templates to database...COMPLETED

Updating rdbms version in Context file to db112
Updating rdbms type in Context file to 64 bits
Configuring templates from ORACLE_HOME ...

AutoConfig completed successfully.

This issue because of Database 11.2 it only comes with 64bit libraries, but we are still trying to use 32bit java as defined by context variables s_jdktop and s_jretop which points to ORACLE_HOME/appsutil/jre

So changed the s_jdktop and s_jretop in $CONTEXT_FILE  to point to $ORACLE_HOME/jdk/jre and ran the autoconfig and if went successful.

Context Value Management will now update the Context file

        Updating Context file...COMPLETED

        Attempting upload of Context file and templates to database...COMPLETED

Updating rdbms version in Context file to db112
Updating rdbms type in Context file to 64 bits
Configuring templates from ORACLE_HOME ...

AutoConfig completed successfully.

Reference DOC: 1087973.1