Upgrading socket to SSLSocket with STARTTLS: recv failed

Question

I am trying to upgrade a socket to an SSLSocket using STARTTLS. On InspIRCd's wiki this is how its supposed to work

>> STARTTLS
<< :test2.chatspike.net 670 nickname :STARTTLS successful, go ahead with TLS handshake
(SSL Handshake)

So in my code I've written (slightly simplified)

else if (code.equals("670")) {
    SSLSocketFactory sslSocketFactory = ((SSLSocketFactory) SSLSocketFactory.getDefault());
    SSLSocket sslSocket = (SSLSocket) sslSocketFactory.createSocket(
                        socket,
                        socket.getInetAddress().getHostAddress(),
                        socket.getPort(),
                        true);
    sslSocket.startHandshake();
    bufferedReader = new BufferedReader(new InputStreamReader(sslSocket.getInputStream(), getEncoding()));
    bufferedWriter = new BufferedWriter(new OutputStreamWriter(sslSocket.getOutputStream(), getEncoding()));
    socket = sslSocket;
}

However, with and without startHandshake() I always get a recv failed (without I just get it when I try to read from the new bufferedReader

java.io.IOException: Can't connect to server
    at org.pircbotx.PircBotX.connect(PircBotX.java:472) [Exception handling]
    at org.pircbotx.PircBotX.connect(PircBotX.java:224) [Overridden constructor]
    at org.pircbotx.impl.PircBotXExample.main(PircBotXExample.java:185)
Caused by: java.net.SocketException: Software caused connection abort: recv failed
    at java.net.SocketInputStream.read(SocketInputStream.java:150)
    at java.net.SocketInputStream.read(SocketInputStream.java:121)
    at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
    at sun.security.ssl.InputRecord.read(InputRecord.java:480)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
    at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1707)
    at sun.security.ssl.HandshakeOutStream.flush(HandshakeOutStream.java:122)
    at sun.security.ssl.Handshaker.kickstart(Handshaker.java:909)
    at sun.security.ssl.SSLSocketImpl.kickstartHandshake(SSLSocketImpl.java:1423)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1288)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
    at org.pircbotx.PircBotX.connect(PircBotX.java:411) [startHandshake line]
    ... 2 more

Full log (with -Djavax.net.debug=all)

1365707641940 <<<:kenny.chatspike.net 670 PircBotX :STARTTLS successful, go ahead with TLS handshake
keyStore is : 
keyStore type is : jks
keyStore provider is : 
init keystore
init keymanager of type SunX509
trustStore is: C:\Program Files\Java\jdk1.7.0_17\jre\lib\security\cacerts
trustStore type is : jks
trustStore provider is : 
init truststore

[*SNIP* A bunch of adding trusted certificates]

trigger seeding of SecureRandom
done seeding SecureRandom
Ignoring unavailable cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
Ignoring unavailable cipher suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA
Ignoring unavailable cipher suite: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA
Ignoring unsupported cipher suite: TLS_DHE_DSS_WITH_AES_128_CBC_SHA256
Ignoring unsupported cipher suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA256
Ignoring unsupported cipher suite: TLS_DHE_RSA_WITH_AES_128_CBC_SHA256
Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256
Ignoring unsupported cipher suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384
Ignoring unsupported cipher suite: TLS_RSA_WITH_AES_256_CBC_SHA256
Ignoring unavailable cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384
Ignoring unavailable cipher suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA
Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256
Ignoring unavailable cipher suite: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA
Ignoring unavailable cipher suite: TLS_RSA_WITH_AES_256_CBC_SHA
Ignoring unsupported cipher suite: TLS_RSA_WITH_AES_128_CBC_SHA256
Allow unsafe renegotiation: false
Allow legacy hello messages: true
Is initial handshake: true
Is secure renegotiation: false
%% No cached client session
*** ClientHello, TLSv1
RandomCookie:  GMT: 1365707642 bytes = { 219, 198, 250, 114, 173, 93, 144, 4, 231, 53, 192, 242, 141, 137, 219, 115, 48, 203, 218, 188, 51, 30, 184, 245, 182, 150, 132, 254 }
Session ID:  {}
Cipher Suites: [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_RC4_128_SHA, SSL_RSA_WITH_RC4_128_SHA, TLS_ECDH_ECDSA_WITH_RC4_128_SHA, TLS_ECDH_RSA_WITH_RC4_128_SHA, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_RC4_128_MD5, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
Compression Methods:  { 0 }
Extension elliptic_curves, curve names: {secp256r1, sect163k1, sect163r2, secp192r1, secp224r1, sect233k1, sect233r1, sect283k1, sect283r1, secp384r1, sect409k1, sect409r1, secp521r1, sect571k1, sect571r1, secp160k1, secp160r1, secp160r2, sect163r1, secp192k1, sect193r1, sect193r2, secp224k1, sect239k1, secp256k1}
Extension ec_point_formats, formats: [uncompressed]
***
[write] MD5 and SHA1 hashes:  len = 149
0000: 01 00 00 91 03 01 51 67   0B 7A DB C6 FA 72 AD 5D  ......Qg.z...r.]
0010: 90 04 E7 35 C0 F2 8D 89   DB 73 30 CB DA BC 33 1E  ...5.....s0...3.
0020: B8 F5 B6 96 84 FE 00 00   2A C0 09 C0 13 00 2F C0  ........*...../.
0030: 04 C0 0E 00 33 00 32 C0   07 C0 11 00 05 C0 02 C0  ....3.2.........
0040: 0C C0 08 C0 12 00 0A C0   03 C0 0D 00 16 00 13 00  ................
0050: 04 00 FF 01 00 00 3E 00   0A 00 34 00 32 00 17 00  ......>...4.2...
0060: 01 00 03 00 13 00 15 00   06 00 07 00 09 00 0A 00  ................
0070: 18 00 0B 00 0C 00 19 00   0D 00 0E 00 0F 00 10 00  ................
0080: 11 00 02 00 12 00 04 00   05 00 14 00 08 00 16 00  ................
0090: 0B 00 02 01 00                                     .....
main, WRITE: TLSv1 Handshake, length = 149
main, waiting for close_notify or alert: state 1
main, Exception while waiting for close java.net.SocketException: Software caused connection abort: recv failed
main, handling exception: java.net.SocketException: Software caused connection abort: recv failed
main, SEND TLSv1 ALERT:  fatal, description = unexpected_message
    at java.net.SocketInputStream.socketRead0(Native Method)
main, WRITE: TLSv1 Alert, length = 2
main, Exception sending alert: java.net.SocketException: Software caused connection abort: socket write error
main, called closeSocket()
1365707642425 *** Disconnected.

What could I be doing wrong that would cause this error? I know this feature works because other clients can do it, but I just can't figure out how to do it from Java

Any suggestions?

Answer 1

I did some testing and got this error when receiving data on the socket while the sslSocket is handshaking. This can happen if you for example register (send NICK/LOGIN) to the socket and not the sslSocket. Try to make the first command you make to the server after connection TLSSTART and wait for the 670/691 code before you send anything else.

The following is not producing any exceptions for me and manages to handshake without any problems:

public static void main(String[] args) throws Exception {
    Socket socket = new Socket("irc.chatspike.net",6667);
    BufferedReader reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
    BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream()));
    String line;
    writer.write("STARTTLS\r\n");
    writer.flush(); 
    while ((line = reader.readLine()) != null) {
        if (line.contains(" 670 ")) {
            SSLContext sc = SSLContext.getInstance("TLS");
            sc.init(null, new TrustManager[]{
                    new X509TrustManager() {
                        public java.security.cert.X509Certificate[] getAcceptedIssuers() {
                            return null;
                        }
                        public void checkClientTrusted(
                            java.security.cert.X509Certificate[] certs, String authType) {
                        }
                        public void checkServerTrusted(
                            java.security.cert.X509Certificate[] certs, String authType) {
                        }
                    }
                }, new java.security.SecureRandom());
            SSLSocketFactory sslSocketFactory = ((SSLSocketFactory) sc.getSocketFactory());
            SSLSocket sslSocket = (SSLSocket) sslSocketFactory.createSocket(
                                socket,
                                socket.getInetAddress().getHostAddress(),
                                socket.getPort(),
                                true);
            sslSocket.startHandshake();
        }
    }

Answer 2

The problem here is that you are sending something other than an SSL handshake after the STARTTLS command, via the plaintext socket. Once you've issued STARTTLS and it has been accepted, use of the plaintext socket must cease.