Distributed Networked Applications (Python, Celery, 0MQ, etc)

Question

Hi Everyone, I have been developing with python for about a year. Now i have a need to develop distributed applications which communicate with each other.

I do not have an idea which modulue will be the best to learn, so i need your help and advise on this.

My app needs are,

1. Client Server must be able to communicate and exchange data with each other.

2. Tasks Needs to be scheduled on server and client. Kind of Cron like, no need for auto consuming stuff.

3. I would like to be able to complie everything into a exe,app,package to support Win,Mac,Linux. Thin Clients or workers, and fat servers

My issues are as follows.

1. Nearly everything i see is going towards the web application (Djano, Celery). I need to have a daemon worker running, and thats it. But if i want to use celery, then i would need to have the code in py files and running, which i am not so keen on. Not to mention pyinstaller does not support the packaging

2. I have researched quite a bit into it, and which module is best i am not sure. But i have sort of narrowed down to celery as the best bet. But i am thinking of ZeroMQ or Pyro as the compiling part later on i am not too sure.

Please do advise, Kumar

Answer 1

if the individual tasks dont need to communicate to other tasks then use Celery, otherwise Pyro as it allows communication between tasks/objects running parallely on worker nodes.