Storing objects on a Process?

Question

I would like to store an object on a subclass of multiprocessing.Process as follows:

class TestProcess( Process ):
    def __init__( self, foo ):
        super( TestProcess, self ).__init__()
        self.foo = foo
    def run( self ):
        from time import sleep
        sleep( 3 )
        self.bar = self.foo

p = TestProcess( 5 )
p.start()
p.join()
print( p.bar ) # raises AttributeError

This works fine when Process is switched out with Thread, but I'm clearly missing something. What?

Answer 1

Because TestProcess.run() is running in a child process, it assigns self.bar in the child process, which cannot effect parent process. The following code will show you the different:

import os
from multiprocessing import Process, Queue

class TestProcess( Process ):
    def __init__( self, foo ):
        super( TestProcess, self ).__init__()
        self.foo = foo
        self.bar = None
        self.que = Queue()
    def run( self ):
        from time import sleep
        sleep( 1 )
        self.bar = self.foo
        print( "pid2=%s, p.bar=%s" % (os.getpid(), self.bar ))
        self.que.put(self.bar)

p = TestProcess( 5 )
p.start()
p.join()
print( "pid1=%s, p.bar=%s" % (os.getpid(), p.bar ))
print(p.que.get())

You will see pid1 and pid2 are different. And, of course, if you use Thread, they run in same process, so it works.

If you do need store data in child process, then transfer to parent process, then involve inter-process-communication. You can use Queue.

Answer 2

When you switch from threading to multiprocessing and back, you're switching from an environment where every thread shares the same memory to one where nothing is shared unless explicitly shared. If you want to share variables between processes, you can use the Value, Array, one of the Queue classes, or another one of the shared constructs.

import os
from multiprocessing import Process, Value

class TestProcess( Process ):
    def __init__( self, foo ):
        super( TestProcess, self ).__init__()
        self.foo = foo
        self.bar = Value('i', 0)
    def run( self ):
        from time import sleep
        sleep( 1 )
        self.bar.value = self.foo
        print( "pid2=%s, p.bar=%s" % (os.getpid(), self.bar.value ))

p = TestProcess( 5 )
p.start()
p.join()
print( "pid1=%s, p.bar=%s" % (os.getpid(), p.bar.value ))
print(p.bar.value)