JavaSSL mutual auth certificate not sent

Question

I am working on web application hosted in Tomcat. This app consumes SOAP WS (hosted on IIS) which require client certificates. I have everything setup and ready, but on production environment, the handshake just dont happen right.

Here are relative parts of output from javax.net.debug=ssl

1) Client certificate and private key found

found key for : authentication service client company2
chain [0] = [
[
  Version: V3
  Subject: EMAILADDRESS=john.smith@EXAMPLE1.cz, CN=EXAMPLE1, OU=Web Service App, O=My Company, ST=Czech Republic, C=CZ
  Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5

  Key:  Sun RSA public key, 2048 bits
  modulus: ...... 
  public exponent: 65537
  Validity: [From: Mon Nov 04 17:55:55 CET 2013,
               To: Sun Nov 04 17:55:55 CET 2018]
  Issuer: EMAILADDRESS=peter.smith@EXAMPLE2.cz, CN=CA, OU=Web Service App, O=My Company, L=Prague, ST=Czech Republic, C=CZ
  SerialNumber: [ ...... ]

Certificate Extensions: 4
[1]: ObjectId: 2.16.840.1.113730.1.13 Criticality=false
Extension unknown: DER encoded OCTET string = ...... 


[2]: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [ ..... ]
]

[3]: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
[EMAILADDRESS=peter.smith@EXAMPLE2.cz, CN=CA, OU=Web Service App, O=My Company, L=Prague, ST=Czech Republic, C=CZ]
SerialNumber: [ .... ]
]

[4]: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
  CA:false
  PathLen: undefined
]

]
  Algorithm: [SHA1withRSA]
  Signature: .......

]

2) Truststore init

trustStore is: ......
trustStore type is : jks
trustStore provider is : 
init truststore
...
adding as trusted cert:
  Subject: EMAILADDRESS=peter.smith@EXAMPLE2.cz, CN=CA, OU=Web Service App, O=My Company, L=Prague, ST=Czech Republic, C=CZ
  Issuer:  EMAILADDRESS=peter.smith@EXAMPLE2.cz, CN=CA, OU=Web Service App, O=My Company, L=Prague, ST=Czech Republic, C=CZ
  Algorithm: RSA; Serial number: 0x......
  Valid from Mon Nov 04 15:35:23 CET 2013 until Sat Nov 04 15:35:23 CET 2023
...

3) Found trusted certificate at WS server endpoint

Found trusted certificate:
[
[
  Version: V3
  Subject: CN=Thawte SSL CA, O="Thawte, Inc.", C=US
  Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5

  Key:  Sun RSA public key, 2048 bits
  modulus: .....
  public exponent: 65537
  Validity: [From: Mon Feb 08 01:00:00 CET 2010,
               To: Sat Feb 08 00:59:59 CET 2020]
  Issuer: CN=thawte Primary Root CA, OU="(c) 2006 thawte, Inc. - For authorized use only", OU=Certification Services Division, O="thawte, Inc.", C=US
  SerialNumber: [ ..... ]

4) Server ask for client cert

CertificateRequest
Cert Types: RSA, DSS, ECDSA
Cert Authorities:
...
<EMAILADDRESS=peter.smith@EXAMPLE2.cz, CN=CA, OU=Web Service App, O=My Company, L=Prague, ST=Czech Republic, C=CZ> 

5) Client provides NOTHING :-(

ServerHelloDone
Certificate chain

ClientKeyExchange, RSA PreMasterSecret, TLSv1
...  

The whole thing ends up by http 403 on the WS server (correct behaviour for not providing the client cert), no exceptions regarding the handshake. The client is javax.xml.ws.Service generated by wsimport.

What confuses me the most, on dev/test environment the exact same certificate is passed just fine (using the exact same trustore and WS client). So the client app and certificate seems to work fine.

Maybe there is some java related glitch on the other environment or am I missing something?

Any help much appreciated. I'am a bit lost here, after several days of figuring it out.

I will append any related input that will be asked for.

Thank you.

Additional info:

• Attemp to access webservice by browser result in certificate selection dialog. After selection is made (using correct certificate), standard page "Metadata publishing for this service is currently disabled" is displayed

Answer 1

I figured it out!

The problem is, that I am using custom KeyManagerFactory to select certificates according to other question here on SO (similar to http://vafer.org/blog/20061010073725/). I think everything works fine on test environment just because the default SSLContext is initialized first time when I use it with my custom KeyManagerFactory.

On production tho, global SSLContext (or something similar, sorry for my JAVA noobiness) is used and initialized before my app can initialize it and insert the desired key. So even I am giving the SSLContext access to the keypair (as seen above, in the logs), it did not use it during the handshake. I can now reproduce this on test env just by invoking other applications first.

To solve this problem, I also set -Djavax.net.keyStore arguments to point to the store with client keypair. So the strore is now properly initialized with the key. Drawback of this workaround is obvious: every other app running on tomcat has now access to the client certificate keypair. I can live with that, I guess.