Question

I am new to python and Parallel Python. The problem is I have 4 jobs to be done: generation of 4 masks, multiplication of them with my input image, followed by further processing. Following is the piece of the code written for parallel processing.

inputs = range(4)
jobs = [(inpt, job_server.submit(PP, (inpt,input_data,size,(imageMultiply,blockCounter,imageQuantizer ), ("numpy","Image"))) for inpt in inputs]
job_server.print_stats()
for inpt, job in jobs:
  print "No of blocks in ", inpt, "is", job() ## accessing the result of pp

The output that I get is :

Starting pp with 4 workers
Job execution statistics:
 job count | % of all jobs | job time sum | time per job | job server
         4 |        100.00 |       0.0000 |     0.000000 | local
Time elapsed since server creation 0.0219678878784
4 active tasks, 4 cores

No of blocks in  0 is 52
No of blocks in  1 is 61
No of blocks in  2 is 104
No of blocks in  3 is 48

I cant understand that if its not processing simultaneously, still I am able to get the desired output, but the time taken is too big which is why I want to use pp. Please help me with this so that I can successfully reduce the time. Thanks in advance...

No correct solution

OTHER TIPS

From your print_stats() output (reformatted here,)

Job execution statistics:
 job count | % of all jobs | job time sum | time per job | job server
         4 |        100.00 |      16.0401 |     4.010028 | local
Time elapsed since server creation 4.04183793068
0 active tasks, 4 cores

everything seems to be fine. You have 4 CPU cores; you could complete 4 jobs within 4 seconds after the creation of the job server; the system consumed 16 CPU seconds in total to get all the jobs done.

You may want to try top -H, htop or Windows Sysinternals Process Monitor to observe CPU consumption in real time.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top