KeyManagementException: FIPS mode: only SunJSSE TrustManagers may be used

Question

i use custom DummySocketFactory and DummyTrustMAnager to connect to smtp over TLS. DummySocketFactory:

package XMailMessenger;

public class DummySSLSocketFactory extends SSLSocketFactory {
private SSLSocketFactory factory;

public DummySSLSocketFactory() {
try {


    SSLContext sslcontext = SSLContext.getInstance("TLS");
    //Security.removeProvider("SunJSSE");
    sslcontext.init(null,
             new TrustManager[] { new DummyTrustManager()},
            null );
    factory = (SSLSocketFactory)sslcontext.getSocketFactory();

} catch(Exception ex) {
    System.out.println(ex.toString());
}
}

public static SocketFactory getDefault() {
    SocketFactory a = new DummySSLSocketFactory();
    if ( a == null ) { System.out.println("1"); }
    return a;
}
 ...

DummyTrustManager:

public class DummyTrustManager implements X509TrustManager{

public void checkClientTrusted(X509Certificate[] cert, String authType) {
// everything is trusted


}

public void checkServerTrusted(X509Certificate[] cert, String authType) {
// everything is trusted
}

public X509Certificate[] getAcceptedIssuers() {
return new X509Certificate[0];
    //return null;
}
}

in sending e-mail i receive exception as in subject, this exception goes from function sslcontext.init in DummySSLSocketFactory. I debug it and noticed , that in code:

    private X509TrustManager chooseTrustManager(TrustManager[] tm)
        throws KeyManagementException {
    // We only use the first instance of X509TrustManager passed to us.
    for (int i = 0; tm != null && i < tm.length; i++) {
        if (tm[i] instanceof X509TrustManager) {
            if (SunJSSE.isFIPS() &&
                    !(tm[i] instanceof X509TrustManagerImpl)) {
                throw new KeyManagementException
                    ("FIPS mode: only SunJSSE TrustManagers may be used");
            }

            if (tm[i] instanceof X509ExtendedTrustManager) {
                return (X509TrustManager)tm[i];
            } else {
                return new AbstractTrustManagerWrapper(
                                    (X509TrustManager)tm[i]);
            }
        }
    }

    // nothing found, return a dummy X509TrustManager.
    return DummyX509TrustManager.INSTANCE;
}

exception occures in if (SunJSSE.isFIPS() && !(tm[i] instanceof X509TrustManagerImpl)) expression.

I suppose that tm[i] contains my DummyTrustManager , it can not be extended from X509TrustManagerImpl so my question is : How to disable Fips in SunJSSE ?

Answer 1

SunJSSE can be configured to run on FIPS-140 compliant mode as long as it uses a FIPS-140 certified cryptographic hardware or software provider that implements all cryptographic algorithms required by JSSE (ex. Network Security Services – NSS, Sun Cryptographic Accelerator 6000, nCipher, etc).

To enable FIPS mode, edit the file ${java.home}/lib/security/java.security and modify the line that lists com.sun.net.ssl.internal.ssl.Provider and associate the name of the FIPS-140 cryptographic provider (ex. SunPKCS11-NSS). The name of the provider is a string that concatenates the prefix SunPKCS11- with the name of the specified PKCS#11 provider in its configuration file.

security.provider.4=com.sun.net.ssl.internal.ssl.Provider SunPKCS11-NSS

In case of using NSS as cryptographic software token (Make use of NSS 3.1.1. or above), assuming the libraries are located under the /opt/nss/lib directory and its key database files (with the suffix .db) are under the /opt/nss/fipsdb directory, the sample configuration for representing NSS will be as follows:

                       # Use NSS as a FIPS-140 compliant cryptographic token 
                       # SunPKCS11-NSS
                      name = NSS
                      nssLibraryDirectory = /opt/nss/lib
                      nssSecmodDirectory = /opt/nss/fipsdb
                      nssModule = fips

In FIPS mode, SunJSSE will perform SSL/TLS 1.0 based communication and cryptographic operations including symmetric and asymmetric encryption, signature generation and verification, message digests and message authentication codes, key generation and key derivation, random number generation, etc.

Answer 2

To anyone having a giant headache when you need to install a tomcat webapp on a third party server, I lost 1 hour trying to bypass this damn thing...

I solved in this way, without touching anything in the webapp.

1. Add this java parameter:

-Djava.security.disableSystemPropertiesFile=true

Source: https://access.redhat.com/documentation/en-us/openjdk/8/pdf/configuring_openjdk_8_on_rhel_with_fips/OpenJDK-8-Configuring_OpenJDK_8_on_RHEL_with_FIPS-en-US.pdf

2. Also, if the app needs to connect to a Windows Server, you might want to disable FIPS there too:

• In Control Panel, click Administrative Tools -> Local Security Policy.
• In Security Settings -> Local Policies -> Security Options.
• Under Policy in the right pane, double-click System cryptography: Use FIPS compliant algorithms for encryption, hashing, and signing, and then click Disabled.
• Reboot the server

---

(bonus) If you want to uninstall FIPS from the server, follow this giude (I didn't test it):

https://www.bggofurther.com/2021/02/disable-fips-mode-on-centos-7/