TLS-Structure of certificate message

Question

I'm implementing RFC 5246(TLS 1.2), and I'm stuck at the certificate message.I'm debugging the server with a combination of Openssl s_client and Browsers.The server hello message is received and interpreted fine, and with -msg option in openssl I can see the message has been interpreted properly as Client_hello, without any errors. When the certificate message is sent, the browser doesn't respond, and openssl s_client with -msg doesn't respond either. Openssl s_client with -debug reads the message but doesn't respond after the server_hello_done message. No errors are logged. I suspect the problem is with the structure of my certificate message, because anything I send after the Record layer length gets received without any errors, even if it's just random text or binary data. Modifying the structure of the record layer to incorrect/inappropriate values throws an error with the appropriate error message, for example setting the version to 9.3 throws the error 5256:error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number as expected. This is the structure I am currently using:

/Record layer/

• One byte for the message type, two bytes for the protocol version(major and minor), and two bytes for the message length.

/Handshake message data/

• Two bytes for the length, two bytes for the certificate(s) length.Finally the certificate(s).

I'm currently working with a self-signed certificate, and neither the browser nor openssl s_client include a signature_algorithms extension in the Client_hello, so I send the certificate as it is, in .PEM format, and in plain text.Below is openssl's hex dump of the handshake thus far:

CONNECTED(00000164)
write to 0x14ad698 [0x13f1ffb] (210 bytes => 210 (0xD2))
0000 - 16 03 01 00 cd 01 00 00-c9 03 01 53 5e 5c d6 a0   ...........S^\..
0010 - 34 27 ea 22 ed 01 dc 36-bb 0b 84 1e 5a 58 3e d5   4'."...6....ZX>.
0020 - 95 4d 5f 81 9f 2a f0 27-75 fb 1f 00 00 5c c0 14   .M_..*.'u....\..
0030 - c0 0a 00 39 00 38 00 88-00 87 c0 0f c0 05 00 35   ...9.8.........5
0040 - 00 84 c0 12 c0 08 00 16-00 13 c0 0d c0 03 00 0a   ................
0050 - c0 13 c0 09 00 33 00 32-00 9a 00 99 00 45 00 44   .....3.2.....E.D
0060 - c0 0e c0 04 00 2f 00 96-00 41 00 07 c0 11 c0 07   ...../...A......
0070 - c0 0c c0 02 00 05 00 04-00 15 00 12 00 09 00 14   ................
0080 - 00 11 00 08 00 06 00 03-00 ff 01 00 00 44 00 0b   .............D..
0090 - 00 04 03 00 01 02 00 0a-00 34 00 32 00 01 00 02   .........4.2....
00a0 - 00 03 00 04 00 05 00 06-00 07 00 08 00 09 00 0a   ................
00b0 - 00 0b 00 0c 00 0d 00 0e-00 0f 00 10 00 11 00 12   ................
00c0 - 00 13 00 14 00 15 00 16-00 17 00 18 00 19 00 23   ...............#
00d2 - <SPACES/NULS>
read from 0x14ad698 [0x13edaab] (5 bytes => 5 (0x5))
0000 - 16 03 01 00 2c                                    ....,
read from 0x14ad698 [0x13edab0] (44 bytes => 44 (0x2C))
0000 - 02 00 00 26 03 01 53 5e-5c d6 4a 5a 5b 4d 63 38   ...&..S^\.JZ[Mc8
0010 - 57 47 53 45 4d 5b 3f 3c-5f 38 23 67 26 32 38 4c   WGSEM[?<_8#g&28L
0020 - 2e 67 47 67 28 56 00 00-2f                        .gGg(V../
002c - <SPACES/NULS>
read from 0x14ad698 [0x13edaab] (5 bytes => 5 (0x5))
0000 - 16 03 01 02 99                                    .....
read from 0x14ad698 [0x13edab0] (665 bytes => 665 (0x299))
0000 - 0b 02 96 02 94 2d 2d 2d-2d 2d 42 45 47 49 4e 20   .....-----BEGIN 
0010 - 43 45 52 54 49 46 49 43-41 54 45 2d 2d 2d 2d 2d   CERTIFICATE-----
0020 - 0a 4d 49 49 42 75 54 43-43 41 53 49 43 43 51 43   .MIIBuTCCASICCQC
0030 - 43 65 67 31 46 6f 4f 76-43 4b 6a 41 4e 42 67 6b   Ceg1FoOvCKjANBgk
0040 - 71 68 6b 69 47 39 77 30-42 41 51 55 46 41 44 41   qhkiG9w0BAQUFADA
0050 - 68 4d 51 73 77 43 51 59-44 56 51 51 47 45 77 4a   hMQswCQYDVQQGEwJ
0060 - 4c 0a 52 54 45 53 4d 42-41 47 41 31 55 45 41 78   L.RTESMBAGA1UEAx
0070 - 4d 4a 62 47 39 6a 59 57-78 6f 62 33 4e 30 4d 42   MJbG9jYWxob3N0MB
0080 - 34 58 44 54 45 30 4d 44-51 79 4d 54 45 32 4e 44   4XDTE0MDQyMTE2ND
0090 - 4d 30 4e 56 6f 58 44 54-45 31 4d 44 51 79 4d 54   M0NVoXDTE1MDQyMT
00a0 - 45 32 0a 4e 44 4d 30 4e-56 6f 77 49 54 45 4c 4d   E2.NDM0NVowITELM
00b0 - 41 6b 47 41 31 55 45 42-68 4d 43 53 30 55 78 45   AkGA1UEBhMCS0UxE
00c0 - 6a 41 51 42 67 4e 56 42-41 4d 54 43 57 78 76 59   jAQBgNVBAMTCWxvY
00d0 - 32 46 73 61 47 39 7a 64-44 43 42 6e 7a 41 4e 42   2FsaG9zdDCBnzANB
00e0 - 67 6b 71 0a 68 6b 69 47-39 77 30 42 41 51 45 46   gkq.hkiG9w0BAQEF
00f0 - 41 41 4f 42 6a 51 41 77-67 59 6b 43 67 59 45 41   AAOBjQAwgYkCgYEA
0100 - 72 51 71 76 50 36 4c 35-41 71 31 31 67 76 38 2b   rQqvP6L5Aq11gv8+
0110 - 2f 59 55 53 62 50 46 4b-34 66 51 71 30 74 42 79   /YUSbPFK4fQq0tBy
0120 - 36 53 39 6c 0a 78 6f 45-6d 50 47 79 52 49 7a 44   6S9l.xoEmPGyRIzD
0130 - 31 46 78 78 52 65 50 79-55 6a 69 78 63 39 66 41   1FxxRePyUjixc9fA
0140 - 59 6f 74 5a 31 53 71 71-2f 2b 37 77 69 52 2b 7a   YotZ1Sqq/+7wiR+z
0150 - 33 46 6f 65 51 58 73 53-64 32 78 32 44 4b 63 62   3FoeQXsSd2x2DKcb
0160 - 73 62 64 62 76 0a 2f 73-49 2b 68 63 57 39 4c 5a   sbdbv./sI+hcW9LZ
0170 - 48 4d 45 75 49 6a 4d 42-73 6f 6f 4d 52 6a 76 35   HMEuIjMBsooMRjv5
0180 - 79 78 79 50 67 38 33 35-34 66 46 6d 51 50 38 4a   yxyPg8354fFmQP8J
0190 - 49 73 54 54 48 31 54 56-5a 4f 47 48 79 49 62 76   IsTTH1TVZOGHyIbv
01a0 - 6e 34 7a 36 35 6b 0a 48-63 62 75 4f 52 38 43 41   n4z65k.HcbuOR8CA
01b0 - 77 45 41 41 54 41 4e 42-67 6b 71 68 6b 69 47 39   wEAATANBgkqhkiG9
01c0 - 77 30 42 41 51 55 46 41-41 4f 42 67 51 41 72 73   w0BAQUFAAOBgQArs
01d0 - 36 62 74 6f 38 6b 76 67-6b 48 70 4d 56 50 42 42   6bto8kvgkHpMVPBB
01e0 - 50 62 4e 71 6d 2f 62 0a-4a 78 4a 34 4a 78 5a 72   PbNqm/b.JxJ4JxZr
01f0 - 51 4c 6b 6a 48 7a 39 34-44 75 57 7a 64 67 41 6a   QLkjHz94DuWzdgAj
0200 - 56 2b 70 39 72 7a 65 76-37 56 65 57 44 49 5a 41   V+p9rzev7VeWDIZA
0210 - 78 39 6a 43 6b 65 39 59-51 4c 68 42 67 62 2b 7a   x9jCke9YQLhBgb+z
0220 - 48 73 6d 30 39 70 50 43-0a 4c 4f 50 37 4b 67 4b   Hsm09pPC.LOP7KgK
0230 - 42 78 6e 68 52 49 33 4f-43 48 41 39 6f 43 78 30   BxnhRI3OCHA9oCx0
0240 - 78 46 35 46 65 34 53 38-34 65 6b 30 5a 37 65 5a   xF5Fe4S84ek0Z7eZ
0250 - 4c 55 63 7a 52 41 6f 55-57 50 42 65 70 32 6e 62   LUczRAoUWPBep2nb
0260 - 44 77 39 78 6c 6e 30 57-49 0a 53 33 72 6d 2f 47   Dw9xln0WI.S3rm/G
0270 - 4d 6d 6e 4e 73 78 75 74-68 69 63 41 3d 3d 0a 2d   MmnNsxuthicA==.-
0280 - 2d 2d 2d 2d 45 4e 44 20-43 45 52 54 49 46 49 43   ----END CERTIFIC
0290 - 41 54 45 2d 2d 2d 2d 2d-0a                        ATE-----.
read from 0x14ad698 [0x13edaab] (5 bytes => 5 (0x5))
0000 - 0e 03 01                                          ...
0005 - <SPACES/NULS>
read from 0x14ad698 [0x13edaab] (5 bytes => -1 (0xFFFFFFFF))
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 724 bytes and written 0 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : 0000
    Session-ID: 
    Session-ID-ctx: 
    Master-Key: 
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    Start Time: 1398693078
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
---

Answer 1

PEM is certainly not the right format to send on the wire. You should send it in binary form (use a base-64 decoded version of this PEM content).

You'll also need to wrap this in other layers, since a certificate_list in the Certificate struct is expected.

If you want to learn a bit more about all this, it's probably worth looking at existing traffic produced by working implementations using Wireshark. Its SSL wiki page even has an existing capture file that you can use.

(You're also mentioning you want to implement TLS 1.2, but you're sending 03 01, which is for TLS 1.0, although this shouldn't matter very much at this stage of your implementation.)