Access process data from a django view

Question

I'm quite new to python and Django and I'm struggling to understand what's the best way to tackle my problem, which is:

I have a view which has to activate a process/task/thread and return a success. The process/task/thread operates a device and it will update its status based on the device inputs. I then have another view which I will poll with ajax and this view shoule be able to query that background process/task/thread to know its status and return it to the caller.

I've read quite a few different options like multiprocessing, gevent, celery, session but I'm still can't get my head around it.

Tried with the session but obviously I don't have access to the request object from within the background task. Didn't try gevent or celery just because I thought there would have been a easier solution without using any additional frameworks (don't really wanna install RabbitMQ etc...). Tried the multiprocessing and that's the code:

def test_process(request):
    manager = Manager()
    d = manager.dict()
    p = Process(target=test_function, args=(d, ))
    p.daemon = True
    p.start()
    return HttpResponse(json.dumps('Ok'), content_type="application/json")


def test_function(d):
    d['test'] = 'alex'


def test_manager(request):
    manager = Manager()
    data = manager.dict().get('test')
    return HttpResponse(json.dumps(data), content_type="application/json")

After I wrote I realized that probably the dictionary is only shared by the background process and the process of the request that executed test_process and so test_manager gets and empty dictionary.

Dunno where to go from here

Any help ?

Cheers

Answer 1

To share data between a child and a parent process using the multiprocessing interface you may use one of the methods proposed in https://docs.python.org/2/library/multiprocessing.html, for instance a Queue or a Pipe.

Here's what you should do to use a queue to talk to a child process from within a Django web application (I suppose that the background/child process is controlling a single device for all users of the web application, so everybody will get the same results -- this could also be per session):

• Create a global queue object inside your views.py like this global_q = Queue().
• Create a view for initializing the process, and pass the global Queue to the process function:

def init_process(request):
    p = Process(target=the_process, args=(global_q, ))
    p.daemon = True
    p.start()
    return HttpResponse(json.dumps('Ok'), content_type="application/json")

• Create a different view that will read from the global Queue:

def read_process_status(request):
    data = global_q.get()
    return HttpResponse(json.dumps(data), content_type="application/json")

• Your process function handles the device and write events in the queue parameter when needed:

def the_process(local_q):
    # do some things
    local_q.put([6])
    # do some other things
    local_q.put([34])

For the above to work without problems you must check if th queue is empty or make it non-block etc, but you'll get the idea.