What's the most straightforward way to add an encryption layer to my TCP client/server app?

Question

I wrote a client/server application that runs over TCP. The server runs at a well-known IP address, and clients connect to the server from various locations on the Internet. The TCP connections to the server are persistent and full-duplex; that is, the client and the server are both free to send data 'messages' to each other at any time for as long as the TCP connection remains open. The server is written in C++ and is designed around select() and non-blocking I/O.

This all works fine, but currently all data is sent as plaintext, which means the client<->server TCP traffic can be easily read by anyone along the route who cares to run a packet sniffer. I'd like to add some encryption so that the data sent by my clients and server is less susceptible to snooping, but I'm not sure what is the proper API to use for this purpose.

I've looked at the libssh2 API and the OpenSSL API, and they both look like they might be appropriate... but openssl seems to require the generation and distribution of public keys, which I'm not sure how to do in a secure and user-friendly manner; libssh2, on the other hand, is a bit higher-level, but it looks like it might be oriented towards clients only, and for my system I'll need to add support on the server side as well as the client side.

Am I looking in the right place? Or is what I'm trying to do even possible without going through a formal process of buying certificates from VeriSign and so on? Any pointers will be much appreciated, as I'm very confused at the moment... :)

Answer 1

You could add TLS encryption using the OpenSSL API.

Its rather straight forward to implement, eg. subclass you current socket class and change the Connect(), Disconnect(), Read() and Write().

Depending on the kind af application you have, you'll have to fine tune the certificate checking. Sometimes (like for web servers) only the client can check the certificate of the server, but whenever possible both sides should verify the certificate of the other.

If you can only check the server key, get a certificate signed by a well known CA so all clients can check it. If your server also checks the clients, they they also need to get their certs signed by a public CA, or you configure your own CA and send a certificate to each client along with the user registration for example.