SSL_NULL_WITH_NULL_NULL cipher suite in in Jetty logs

Question

I'm using Jetty with HTTPS and a valid certificate, and I'm not sure to get it right because cipher suite appears to be SSL_NULL_WITH_NULL_NULL in server logs. Client logs look good, however.

The long story: I'm attaching a Java sample expecting Jetty-7.6.10 and two scripts to create both keystore and truststore.

JettyHttpsForStackOverflow runs client and server, together, or separately to deintricate the logs.

The create-chains.sh script creates the keystore and the truststore. The keystore contains a chain ending by a root certificate authority generated from a transient keystore. It replicates a real-world case with a certification authority and intermediate certificates.

The create-single-autosigned.sh script creates the keystore and the truststore, too, but with a self-signed certificate.

Please note that SSL_NULL_WITH_NULL_NULL appears as the server's cipher suite with both certificate chains.

I think there is no problem with the server domain name. I get the same problem with a server running on a machine with a domain name matching the distinguished name in a properly-signed certificate. SSLLab confirmed that SSL on my server works fine (grade B), and Google Chrome connects happily.

I think there is no problem with Jetty client. As I'm using it, it just calls the SSLContextFactory I'm setting up to create an SSLSocket. Amazingly, in Jetty client logs, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA appears to be the cipher suite in use.

Is it normal to get SSL_NULL_WITH_NULL_NULL in Jetty server logs? If not, how to get that thing right?

create-single-autosigned.sh

#!/bin/bash

rm  their-keystore.jks 2> /dev/null
rm  my-keystore.jks    2> /dev/null
rm  my-truststore.jks  2> /dev/null

echo "===================================================="
echo "Creating fake third-party chain ca2 -> ca1 -> ca ..."
echo "===================================================="

keytool -genkeypair -alias ca  -dname cn=ca                           \
  -validity 10000 -keyalg RSA -keysize 2048                           \
  -ext BasicConstraints:critical=ca:true,pathlen:10000                \
  -keystore their-keystore.jks -keypass Keypass -storepass Storepass

keytool -genkeypair -alias ca1 -dname cn=ca1                          \
  -validity 10000 -keyalg RSA -keysize 2048                           \
  -keystore their-keystore.jks -keypass Keypass -storepass Storepass

keytool -genkeypair -alias ca2 -dname cn=ca2                          \
  -validity 10000 -keyalg RSA -keysize 2048                           \
  -keystore their-keystore.jks -keypass Keypass -storepass Storepass


  keytool -certreq -alias ca1                                            \
    -keystore their-keystore.jks -keypass Keypass -storepass Storepass   \
| keytool -gencert -alias ca                                             \
    -ext KeyUsage:critical=keyCertSign                                   \
    -ext SubjectAlternativeName=dns:ca1                                  \
    -keystore their-keystore.jks -keypass Keypass -storepass Storepass   \
| keytool -importcert -alias ca1                                         \
    -keystore   their-keystore.jks -keypass Keypass -storepass Storepass

#echo "Debug exit" ; exit 0

  keytool -certreq -alias ca2                                           \
    -keystore their-keystore.jks -keypass Keypass -storepass Storepass  \
| keytool -gencert -alias ca1                                           \
    -ext KeyUsage:critical=keyCertSign                                  \
    -ext SubjectAlternativeName=dns:ca2                                 \
    -keystore their-keystore.jks -keypass Keypass -storepass Storepass  \
| keytool -importcert -alias ca2                                        \
    -keystore their-keystore.jks -keypass Keypass -storepass Storepass

keytool -list -v -storepass Storepass -keystore their-keystore.jks


echo  "===================================================================="
echo  "Fake third-party chain generated. Now generating my-keystore.jks ..."
echo  "===================================================================="
read -p "Press a key to continue."

# Import authority's certificate chain

  keytool -exportcert -alias ca                                         \
    -keystore their-keystore.jks -keypass Keypass -storepass Storepass  \
| keytool -importcert -trustcacerts -noprompt -alias ca                 \
    -keystore  my-keystore.jks -keypass Keypass -storepass Storepass

  keytool -exportcert -alias ca1                                        \
    -keystore their-keystore.jks -keypass Keypass -storepass Storepass  \
| keytool -importcert -noprompt -alias ca1                              \
    -keystore  my-keystore.jks -keypass Keypass -storepass Storepass

  keytool -exportcert -alias ca2                                        \
    -keystore their-keystore.jks -keypass Keypass -storepass Storepass  \
| keytool -importcert -noprompt -alias ca2                              \
    -keystore  my-keystore.jks -keypass Keypass -storepass Storepass

# Create our own certificate, the authority signs it.

keytool -genkeypair -alias e1  -dname cn=e1                        \
  -validity 10000 -keyalg RSA -keysize 2048                        \
  -keystore my-keystore.jks -keypass Keypass -storepass Storepass

  keytool -certreq -alias e1                                            \
    -keystore my-keystore.jks -keypass Keypass -storepass Storepass     \
| keytool -gencert -alias ca2                                           \
    -ext SubjectAlternativeName=dns:localhost,ip:127.0.0.1              \
    -ext KeyUsage:critical=keyEncipherment,digitalSignature             \
    -ext ExtendedKeyUsage=serverAuth,clientAuth                         \
    -keystore their-keystore.jks -keypass Keypass -storepass Storepass  \
| keytool -importcert -alias e1                                         \
    -keystore my-keystore.jks -keypass Keypass -storepass Storepass

keytool -list -v  -storepass Storepass -keystore  my-keystore.jks

echo "================================================="
echo "Keystore generated. Now generating truststore ..."
echo "================================================="
read -p "Press a key to continue."

  keytool -exportcert -alias ca                                        \
    -keystore my-keystore.jks -keypass Keypass -storepass Storepass    \
| keytool -importcert -trustcacerts -noprompt -alias ca                \
    -keystore my-truststore.jks -keypass Keypass -storepass Storepass

  keytool -exportcert -alias ca1                                       \
    -keystore my-keystore.jks -keypass Keypass -storepass Storepass    \
| keytool -importcert -noprompt -alias ca1                             \
    -keystore my-truststore.jks -keypass Keypass -storepass Storepass

  keytool -exportcert -alias ca2                                       \
    -keystore my-keystore.jks -keypass Keypass -storepass Storepass    \
| keytool -importcert -noprompt -alias ca2                             \
    -keystore my-truststore.jks -keypass Keypass -storepass Storepass

  keytool -exportcert -alias e1                                        \
    -keystore my-keystore.jks -keypass Keypass -storepass Storepass    \
| keytool -importcert -noprompt -alias e1                              \
    -keystore my-truststore.jks -keypass Keypass -storepass Storepass

keytool -list -v  -storepass Storepass -keystore  my-truststore.jks

rm  their-keystore.jks 2> /dev/null

create-single-autosigned.sh

#!/bin/bash

rm  my-keystore.jks    2> /dev/null
rm  my-truststore.jks  2> /dev/null

keytool -genkeypair -alias e1  -dname cn=e1                        \
  -validity 10000 -keyalg RSA -keysize 2048                        \
  -keystore my-keystore.jks -keypass Keypass -storepass Storepass



keytool -list -v  -storepass Storepass -keystore  my-keystore.jks

echo "================================================="
echo "Keystore generated. Now generating truststore ..."
echo "================================================="
read -p "Press a key to continue."

  keytool -exportcert -alias e1                                        \
    -keystore my-keystore.jks -keypass Keypass -storepass Storepass    \
| keytool -importcert -noprompt -alias e1                              \
    -keystore my-truststore.jks -keypass Keypass -storepass Storepass

keytool -list -v  -storepass Storepass -keystore  my-truststore.jks

JettyHttpsForStackOverflow.java

import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.security.KeyManagementException;
import java.security.KeyStore;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.security.UnrecoverableKeyException;
import java.security.cert.Certificate;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;
import javax.net.ssl.KeyManager;
import javax.net.ssl.KeyManagerFactory;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLEngine;
import javax.net.ssl.TrustManager;
import javax.net.ssl.TrustManagerFactory;
import javax.net.ssl.X509TrustManager;

import org.eclipse.jetty.client.ContentExchange;
import org.eclipse.jetty.client.HttpClient;
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.server.ssl.SslSelectChannelConnector;
import org.eclipse.jetty.util.ssl.SslContextFactory;


/**
 * Code sample for Jetty {@link HttpClient} with HTTPS, in a completely standalone fashion.
 * Use create-chains.sh and create-empty.sh to generate completely standalone certificates.
 */
public class JettyHttpsForStackOverflow {

  public static void main( final String... arguments ) throws Exception {
    System.setProperty( "javax.net.debug", "all" ) ;

    try {
      if( arguments.length == 0 || "server".equals( arguments[ 0 ] ) ) {
        runServer() ;
      }
      if( arguments.length == 0 || "client".equals( arguments[ 0 ] ) ) {
        runClient() ;
      }
    } catch( Exception e ) {
      e.printStackTrace() ;
      System.exit( 1 ) ; // Avoids keeping the port open.
    }

  }

  private static void runServer() throws Exception {
    final KeyStore keyStore = loadKeystore() ;
    final SSLContext sslContext = createSslContext(
        keyStore,
        KEYPASS,
        newTrustManagers( keyStore, CERTIFICATE_ALIAS )
    ) ;

    final SslContextFactory sslContextFactory = new SslContextFactory() {
      @Override
      public SSLEngine newSslEngine() {
        return sslContext.createSSLEngine() ;
      }
      @Override
      public SSLEngine newSslEngine( final String host, final int port ) {
        return sslContext.createSSLEngine( host, port ) ;
      }
    } ;
    sslContextFactory.setAllowRenegotiate( true ) ;
    sslContextFactory.setNeedClientAuth( false ) ;
    sslContextFactory.setWantClientAuth( false ) ;
    sslContextFactory.setKeyStorePath( keyStore.toString() ) ; // Better logging.
    sslContextFactory.setKeyStore( keyStore ) ;
    sslContextFactory.setCertAlias( CERTIFICATE_ALIAS ) ;
    sslContextFactory.setKeyManagerPassword( KEYPASS ) ;

    final SslSelectChannelConnector sslConnector =
        new SslSelectChannelConnector( sslContextFactory ) ;
    sslConnector.setPort( PORT ) ;
    sslConnector.open() ;

    final Server jettyServer = new Server() ;
    jettyServer.addConnector( sslConnector ) ;

    jettyServer.start() ;
  }

  public static void runClient() throws Exception {
    final KeyStore keyStore = loadTruststore() ;

    final HttpClient httpClient = new HttpClient() ;
    httpClient.getSslContextFactory().setKeyStore( keyStore ) ; // Better logging.
    httpClient.getSslContextFactory().setKeyStorePassword( "storepwd" ) ;
    httpClient.getSslContextFactory().setKeyManagerPassword( KEYPASS ) ;
    httpClient.setConnectorType( HttpClient.CONNECTOR_SELECT_CHANNEL ) ;
    httpClient.setConnectorType(HttpClient.CONNECTOR_SOCKET);


    // Don't need that because shipping our own certificate in the truststore.
    // Anyways, it blows when set to true.
//    httpClient.getSslContextFactory().setValidateCerts( false ) ;

    httpClient.start() ;

    final ContentExchange contentExchange = new ContentExchange() ;
    contentExchange.setURI( new URL( "https://localhost:" + PORT ).toURI() ) ;
    contentExchange.setTimeout( 36_000_000 ) ; // Leave time for debugging.
    httpClient.send( contentExchange ) ;
    contentExchange.waitForDone() ;
    assert( contentExchange.getStatus() == ContentExchange.STATUS_COMPLETED ) ;
  }

  private static SSLContext createSslContext(
      final KeyStore keyStore,
      final String keypass,
      final TrustManager[] trustManagers
  ) {
    try {
      final KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance( "SunX509" ) ;
      keyManagerFactory.init( keyStore, keypass == null ? null : keypass.toCharArray() ) ;
      final KeyManager[] keyManagers = keyManagerFactory.getKeyManagers() ;
      final SecureRandom secureRandom = new SecureRandom() ;

      final SSLContext sslContext = SSLContext.getInstance( "TLS" ) ;
      sslContext.init(
          keyManagers,
          trustManagers,
          secureRandom
      ) ;
      return sslContext ;
    } catch( NoSuchAlgorithmException | UnrecoverableKeyException | KeyStoreException
        | KeyManagementException e
    ) {
      throw new RuntimeException( e ) ;
    }
  }



  private static TrustManager[] newTrustManagers(
      final KeyStore keyStore,
      final String certificateAlias
  ) {
    try {
      final TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance( "SunX509" ) ;
      trustManagerFactory.init( keyStore ) ;
      final TrustManager[] trustManagers ;
      if( certificateAlias == null ) {
        trustManagers = trustManagerFactory.getTrustManagers() ;
      } else {
        final Certificate certificate = keyStore.getCertificate( certificateAlias ) ;
        final X509Certificate[] x509Certificates ;
        if( certificate == null ) {
          x509Certificates = new X509Certificate[ 0 ] ;
        } else {
          x509Certificates = new X509Certificate[] { ( X509Certificate ) certificate } ;
        }
        trustManagers = new TrustManager[] { newX509TrustManager( x509Certificates ) } ;

      }
      return trustManagers ;
    } catch( KeyStoreException | NoSuchAlgorithmException e ) {
      throw new RuntimeException( e );
    }

  }

  private static final TrustManager newX509TrustManager( final X509Certificate[] certificates ) {
    return new X509TrustManager() {

      public X509Certificate[] getAcceptedIssuers() {
        return certificates ;
      }

      public void checkClientTrusted(
          final X509Certificate[] certs,
          final String authType
      ) { ; }

      public void checkServerTrusted(
          final X509Certificate[] certs,
          final String authType
      ) { ; }
    } ;
  }


  public static KeyStore loadKeystore()
      throws CertificateException, NoSuchAlgorithmException, KeyStoreException, IOException
  {
    return loadKeystore( KEYSTORE_RESOURCE_URL ) ;
  }

  public static KeyStore loadTruststore()
      throws CertificateException, NoSuchAlgorithmException, KeyStoreException, IOException
  {
    return loadKeystore( TRUSTSTORE_RESOURCE_URL ) ;
  }

  public static KeyStore loadKeystore( final URL keystoreResourceUrl )
      throws IOException, KeyStoreException, CertificateException, NoSuchAlgorithmException
  {
    try( final InputStream inputStream = keystoreResourceUrl.openStream() ) {
      final KeyStore keyStore = KeyStore.getInstance( "JKS" ) ;
      // We don't need the storepass for just reading one password-protected certificate
      // of our own, or a trusted entry.
      keyStore.load( inputStream, null ) ;
      return keyStore ;
    }
  }


  private static final int PORT = 8443 ;

  private static final String CERTIFICATE_ALIAS = "e1";

  private static final String KEYPASS = "Keypass";

  private static final URL KEYSTORE_RESOURCE_URL
      = JettyHttpsForStackOverflow.class.getResource( "my-keystore.jks" ) ;

  private static final URL TRUSTSTORE_RESOURCE_URL
      = JettyHttpsForStackOverflow.class.getResource( "my-truststore.jks" ) ;


}

Answer 1

It turns out that SslConnection in Jetty-7.6.10.v20130312 logs incorrectly, while ciphering happens as it should.

Long story: when created, the SslConnection extracts the initial SSLSession object from the SSLEngine and keeps logging with it. Initial SSLSession has a SSL_NULL_WITH_NULL_NULL cipher and that's normal because SSL handshake hasn't happened yet. Activating -Djavax.net.debug=all shows that handshake really occurs, and interactive debugging shows that SSLEngine upgrades to an SSLSession with a real cipher. The problem is just Jetty's SslConnection that still logs with initial SSLSession object. (It also uses values from initial SSLSession to allocate buffers but that's another problem.)

Patching SslConnection for logging with _engine.getSession() gives expected result.

Epilogue: Jetty 9 completely rewrites its SslConnection.

Answer 2

You will see this under at least the following circumstances:

1. You have modified the enabledCipherSuites to include all the supported cipher suites. (Don't!)

2. The SSL handshake is not yet complete.