All glassfish login modules stopped working after container reboot

Question

This is expanded version of another login module problem I had. I diagnosed some more important details which encouraged me to start new thread. It occurres that after Glassfish reboot all login modules stopped working. What is even funnier, Glassfish reports that modules where loaded correctly.

[#|2012-09-05T16:40:59.698+0200|FINE|glassfish3.1.1|javax.enterprise.system.core.security.com.sun.enterprise.security.auth.realm|_ThreadID=1;_ThreadName=Thread-2;ClassName=com.sun.enterprise.security.auth.realm.RealmsManager;MethodName=createRealms;|Initializing configured realms from SecurityService in Domain.xml....|#]

[#|2012-09-05T16:40:59.710+0200|FINE|glassfish3.1.1|javax.enterprise.system.core.security.com.sun.enterprise.security.auth.realm|_ThreadID=1;_ThreadName=Thread-2;ClassName=com.sun.enterprise.security.auth.realm.file.FileRealm;MethodName=init;|FileRealm : file={glass_home}/glassfish/domains/domain1/config/admin-keyfile|#]

[#|2012-09-05T16:40:59.710+0200|FINE|glassfish3.1.1|javax.enterprise.system.core.security.com.sun.enterprise.security.auth.realm|_ThreadID=1;_ThreadName=Thread-2;ClassName=com.sun.enterprise.security.auth.realm.file.FileRealm;MethodName=init;|FileRealm : jaas-context=fileRealm|#]

[#|2012-09-05T16:40:59.711+0200|FINE|glassfish3.1.1|javax.enterprise.system.core.security.com.sun.enterprise.security.auth.realm|_ThreadID=1;_ThreadName=Thread-2;ClassName=com.sun.enterprise.security.auth.realm.file.FileRealm;MethodName=loadKeyFile;|Reading file realm: {glass_home}/glassfish/domains/domain1/config/admin-keyfile|#]

[#|2012-09-05T16:40:59.713+0200|INFO|glassfish3.1.1|javax.enterprise.system.core.security.com.sun.enterprise.security.auth.realm|_ThreadID=1;_ThreadName=Thread-2;|SEC1115: Realm [admin-realm] of classtype [com.sun.enterprise.security.auth.realm.file.FileRealm] successfully created.|#]

[#|2012-09-05T16:40:59.713+0200|FINE|glassfish3.1.1|javax.enterprise.system.core.security.com.sun.enterprise.security.auth.realm|_ThreadID=1;_ThreadName=Thread-2;ClassName=com.sun.enterprise.security.auth.realm.RealmConfig;MethodName=createRealms;|Configured realm: admin-realm|#]

[#|2012-09-05T16:40:59.714+0200|FINE|glassfish3.1.1|javax.enterprise.system.core.security.com.sun.enterprise.security.auth.realm|_ThreadID=1;_ThreadName=Thread-2;ClassName=com.sun.enterprise.security.auth.realm.file.FileRealm;MethodName=init;|FileRealm : file={glass_home}/glassfish/domains/domain1/config/keyfile|#]

[#|2012-09-05T16:40:59.714+0200|FINE|glassfish3.1.1|javax.enterprise.system.core.security.com.sun.enterprise.security.auth.realm|_ThreadID=1;_ThreadName=Thread-2;ClassName=com.sun.enterprise.security.auth.realm.file.FileRealm;MethodName=init;|FileRealm : jaas-context=fileRealm|#]

[#|2012-09-05T16:40:59.715+0200|FINE|glassfish3.1.1|javax.enterprise.system.core.security.com.sun.enterprise.security.auth.realm|_ThreadID=1;_ThreadName=Thread-2;ClassName=com.sun.enterprise.security.auth.realm.file.FileRealm;MethodName=loadKeyFile;|Reading file realm: {glass_home}/glassfish/domains/domain1/config/keyfile|#]

[#|2012-09-05T16:40:59.715+0200|INFO|glassfish3.1.1|javax.enterprise.system.core.security.com.sun.enterprise.security.auth.realm|_ThreadID=1;_ThreadName=Thread-2;|SEC1115: Realm [file] of classtype [com.sun.enterprise.security.auth.realm.file.FileRealm] successfully created.|#]

[#|2012-09-05T16:40:59.715+0200|FINE|glassfish3.1.1|javax.enterprise.system.core.security.com.sun.enterprise.security.auth.realm|_ThreadID=1;_ThreadName=Thread-2;ClassName=com.sun.enterprise.security.auth.realm.RealmConfig;MethodName=createRealms;|Configured realm: file|#]

[#|2012-09-05T16:40:59.717+0200|INFO|glassfish3.1.1|javax.enterprise.system.core.security.com.sun.enterprise.security.auth.realm|_ThreadID=1;_ThreadName=Thread-2;|SEC1115: Realm [certificate] of classtype [com.sun.enterprise.security.auth.realm.certificate.CertificateRealm] successfully created.|#]

[#|2012-09-05T16:40:59.718+0200|FINE|glassfish3.1.1|javax.enterprise.system.core.security.com.sun.enterprise.security.auth.realm|_ThreadID=1;_ThreadName=Thread-2;ClassName=com.sun.enterprise.security.auth.realm.RealmConfig;MethodName=createRealms;|Configured realm: certificate|#]

[#|2012-09-05T16:40:59.757+0200|FINEST|glassfish3.1.1|javax.enterprise.system.core.security.com.sun.enterprise.security.auth.realm|_ThreadID=1;_ThreadName=Thread-2;ClassName=com.sun.enterprise.security.auth.realm.jdbc.JDBCRealm;MethodName=init;|JDBCRealm : jaas-context= jdbcRealm, datasource-jndi = jdbc/mysql-zus, db-user = null, digest-algorithm = MD5, encoding = Hex, charset = UTF-8|#]

[#|2012-09-05T16:40:59.758+0200|INFO|glassfish3.1.1|javax.enterprise.system.core.security.com.sun.enterprise.security.auth.realm|_ThreadID=1;_ThreadName=Thread-2;|SEC1115: Realm [ShibUserPassAuth] of classtype [com.sun.enterprise.security.auth.realm.jdbc.JDBCRealm] successfully created.|#]

[#|2012-09-05T16:40:59.758+0200|FINE|glassfish3.1.1|javax.enterprise.system.core.security.com.sun.enterprise.security.auth.realm|_ThreadID=1;_ThreadName=Thread-2;ClassName=com.sun.enterprise.security.auth.realm.RealmConfig;MethodName=createRealms;|Configured realm: ShibUserPassAuth|#]

[#|2012-09-05T16:40:59.758+0200|FINE|glassfish3.1.1|javax.enterprise.system.core.security.com.sun.enterprise.security.auth.realm|_ThreadID=1;_ThreadName=Thread-2;ClassName=com.sun.enterprise.security.auth.realm.RealmConfig;MethodName=createRealms;|Default realm is set to: ShibUserPassAuth|#]

[#|2012-09-05T16:40:59.762+0200|INFO|glassfish3.1.1|javax.enterprise.system.core.security.com.sun.enterprise.security|_ThreadID=1;_ThreadName=Thread-2;|SEC1011: Security Service(s) Started Successfully|#]

But I am unable to use any of secure realms, during basic authentication. I tried both custom jdbcRealm and preconfigured fileRealm with new user It ends up with this:

javax.security.auth.login.LoginException: No LoginModules configured for jdbcRealm
at javax.security.auth.login.LoginContext.init(LoginContext.java:273)

and this

javax.security.auth.login.LoginException: No LoginModules configured for fileRealm
at javax.security.auth.login.LoginContext.init(LoginContext.java:273)

More over I can't even use more sophisticated asadmin commands, because I received "unauthorised wrong login or password" and in server.log

No LoginModule configured for fileRealm

I checked LoginContext implementation and it looks like that

AppConfigurationEntry[] entries = config.getAppConfigurationEntry (name);

is constantly going wrong way, and null is returned.

I don't manipulate config files by hand. Can I somehow corrupt on of them in a way that block LoginModules?

Which configuration file is being read by ConfigurationEntry above?

Before this unfortunate reboot problem with the MySQL database occurred. There were more available for the Glassfish connection pool, than database permits. After killing connections from database, changing connection numbers for appropriate I rebooted container and everything collapsed.

login.conf exist in domain dir and looks ok.

Hope for help

Answer 1

Paraphrasing Catherine Zeta Jones and Liza Minnelli before her i will say

"and All that JAAS...".

It occurs that one of third party aplications was loading it's own login.conf file for JAAS module, which destroyed all Glassfish login modules. This situation occurs after container reboot when secondary configuration file was being loaded. Funny thing, Glassfish was able to load it's own file properly, show off in logs... and then overwrite it malicious configuration. Watch your configuration files!