Question

I was wondering if multiprocessing can be confined in separate python modules. For example, if I have a python module with multiprocessing as so:

#example.py
def f(q, a, m='No'):
    print m
    q.put(a)

if __name__ == '__main__':
    a = '123'
    m = 'Yes'
    q = Queue()
    p = Process(target=f, args=(q, a, m))
    p.start()
    print q.get()
    p.join()

Is there anyway of using this in another script as a module using import, whilst still retaining the multiprocessing:

#Call from another script
import example
example.f(q, a)
>>> 'Yes'    #Confirmation that multiprocessing was used
Was it helpful?

Solution

Yes, you can accomplish this by either creating a class or a function. You can import either into another script. Here is an example with a class:

# example.py
from multiprocessing import Process

class Example(object):
    def __init__(self, queue):
        """
        @type queue: multiprocessing.Queue
        """
        self.q = queue

    def run(self, a, m=None):
        p = Process(target=self.f, args=(a, m))
        p.start()
        print self.q.get()
        p.join()

    def f(self, a, m='No'):
        print m
        self.q.put(a)

Then import from your example:

>>> from multiprocessing import Queue
>>> from example import Example
>>> q = Queue()
>>> e = Example(q)
>>> e.run('123', m='Yes')
Yes
123
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top