Architecture design for a web application that runs SSH connections to remote servers

Question

This is a followup to my question here.

I have to make a web GUI for a library that manages remote clusters via SSH.

The main point is that this library spawns threads and subprocesses (Popen) internally (for the SSH connections), so I think I am I/O bound by these SSH connections. I want to prevent my server from going down as soon as I get 20-50 simultaneous requests.

I have no idea of how to proceed with the architecture. I don't know if I can solve this at the server level with a more powerful server (gevent??) or if I should do something like using background threads (twisted reactor??)

Some considerations:

• I work with django so the server (if I finally have only one) must be WSGI.
• I need both blocking and non-blocking calls via SSH.
• I will probably need shared state between successive requests as well.

Answer 1

You really want to be looking at separating the concerns here -- have your web app make requests to a message queue, have the execution components process messages from the queue. This also makes security much less nightmarish because you can push the things manging SSH tasks well off the web tier.

The original answer [above] was a bit terse and didn't answer a few questions.

• How will this avoid SSH connection threads exhausting the server? It will avoid it because you can easily throttle most message queues to avoid that. And you can easily ad back-end workers if you are exhausing your initial setup without material change in your application.

• How can I have shared state between requests? The same way you would normally -- you send the request out to the queue, it completes and brings data back which can be passed to the next request.

• How would it appear to users? Could there be status updates? In most cases you can get the data out of the queuing system, or at least have status updates for task completion. Most message queues can easily give indication of failures or failures to start as well.

• Would I have to block the process when communicating between my web server in the message queue be blocking? Not unless you wrote it that way. You can easily setup an ajax request to tickle the queue every few seconds for status on a message to update the UI.

• Is this over-engineering? Not unless you decide to roll your own message queue.