Remote tcp connection in python with zeromq

Question

I have a python client that needs to talk to a remote server I manage. They communicate using zeromq. When I tested the client/server locally everything worked. But now I have the client and server deployed on the cloud, each using a different provider. My question is, what's the simplest way (that is safe) to make the connection? I'm assuming I can't pass the password over, and even if I could I'm guessing there are safer alternatives.

I know how to set an ssh connection without a password using ssh-keygen. Would that work? Would the client need to make an ssh connection with the server before sending the tcp req? If there's a python library that helps with this it'd be a big help.

Thanks!

Update: So more than 24 hours passed and no one replied/answered. I think I'm getting closer to solve this, but not quite there yet. I added my client's key to .ssh/authorized_key on the server, and now I can ssh from the client to the server without a password. Next, I followed this post about "Tunneling PyZMQ Connections with SSH". Here's what I have in my client code:

1    context = zmq.Context()
2    socket = context.socket(zmq.REQ)
3    socket.connect("tcp://localhost:5555")
4    ssh.tunnel_connection(socket, "tcp://locahost:5555", "myuser@remote-server-ip:5555")
5    socket.send_string(some_string)
6    reply = socket.recv()

This doesn't work. I don't really understand lines 3 & 4 and I assume I do something wrong there. Also, my server (hosted on linode) has a "Default Gateway" IP and a "Public IP" -- in the tunnel connection I only specify the public ip, which is also the ip I use to ssh to the machine.

Answer 1

Indeed, ZMQ way is - tunnelling connection with the SSH. Your example is exactly what needs to be done, except that one should either use connect or tunnel_connection, not both.

Also, when specifying server to connect to, make sure to define the SSH port, not the ZMQ REP socket port. That is, instead of myuser@remote-server-ip:5555 you might try myuser@remote-server-ip or myuser@remote-server-ip:22.

import zmq
import zmq.ssh
context = zmq.Context()
socket = context.socket(zmq.REQ)
zmq.ssh.tunnel_connection(socket, "tcp://locahost:5555", "myuser@remote-server-ip")
socket.send(b"Hello")
reply = socket.recv()

Finally, make sure you've installed either pexpect or paramiko - they will do the tunnelling actually. Note that if you're using Windows, paramiko is the only solution which will work - pexpect openssh tunnelling won't work on Windows.

If you use paramiko instead of pexpect, make sure to set paramiko=True in the tunnel_connection arguments.

Answer 2

I have found ssh in Python to be iffy at best, even with paramiko and fabric libraries, so to debug, you might try setting up a tunnel separately, just to see if that's the issue with the broken connection.

For example:

ssh myuser@remote-server-ip -L 5050:localhost:5555 -N

This says: connect to myuser@remote-server-ip, and whenever I request a connection to localhost:5050 on my machine, forward it across the ssh connection so that the server at remote-server-ip thinks it's receiving a connection from localhost:5555.

-L constructs the tunnel, and -N means don't do anything else on the connection.

With that running in another shell, e.g., a different Terminal window, on your local development machine, try to connect to a zeromq server at localhost:5050, which will actually be the zeromq running on the remote server.

You could use 5555:localhost:5555 in the ssh command above, but I find that can be confusing and often conflicts with a local copy of the same service.