Python multiprocessing: Pool of custom Processes

Question

I am subclassing the Process class, into a class I call EdgeRenderer. I want to use multiprocessing.Pool, except instead of regular Processes, I want them to be instances of my EdgeRenderer. Possible? How?

Answer 1

From Jesse Noller:

It is not currently supported in the API, but would not be a bad addition. I'll look at adding it to python2.7/2.6.3 3.1 this week

Answer 2

I don't see any hook for it in the API. You might be able to get away with replicating your desired functionality by using initializer and initargs argument. Alternately, you can build the functionality into the callable object that you use for mapping:

class EdgeRenderTask(object):
    def op1(self,*args):
        ...
    def op2(self,*args):
        ...
p = Pool(processes = 10)
e = EdgeRenderTask()
p.apply_async(e.op1,arg_list)
p.map(e.op2,arg_list)

Answer 3

This seems to work:

import multiprocessing as mp

ctx = mp.get_context()  # get the default context

class MyProcess(ctx.Process):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        print("Hi, I'm custom a process")

ctx.Process = MyProcess  # override the context's Process

def worker(x):
    print(x**2)

p = ctx.Pool(4)
nums = range(10)
p.map(worker, nums)