Python: Attribute error with shelve when using 2 py modules

Question

I'm trying out the shelve module and I'm running into some trouble. My intention is to have one module call a function in a second one which will modify a file inside a shelve. I made a prototype (below) that simply tries to set a variable to something stored in the shelve and print that object.

When I attempt to do this I get the following attribute error:

AttributeError: 'module' object has no attribute 'User'

My guess is it doesn't realize my 'User' class exists but I'm unsure of the reason why. File1 calls the same function as File2 does in it's main function and it correctly works in that instance. I'd appreciate any help with this problem, I'm confused on what the issue is.

import SE_ex2

SE_ex2.cacheTest()

---

import shelve

cache_users = shelve.open("usersTest.cache")

class User(object):
    def __init__(self, name, num):
        self.name = name
        self.num = num

    def __str__(self):
        return "Name:{0}, Num={1}".format(self.name, self.num)

def cacheTest():
    obj = cache_users[ "user_red" ]
    print obj

if __name__ == "__main__":
    cacheTest()

#Used for creating the cache for the first time
#    red = User("red", 11)
#    green = User("green", 22)
#    blue = User("blue", 33)
#
#    cache_users['user_red'] = red
#    cache_users['user_green'] = green
#    cache_users['user_blue'] = blue
#    print "DONE"

---

Full Traceback:

Traceback (most recent call last):
  File "C:\Users\akaGrim\My Documents\Aptana Studio 3 Workspace\ShelveTest\SE_ex1.py", line 3, in <module>
    SE_ex2.cacheTest()
  File "C:\Users\akaGrim\My Documents\Aptana Studio 3 Workspace\ShelveTest\SE_ex2.py", line 14, in cacheTest
    obj = cache_users[ "user_red" ]
  File "C:\Python27\lib\shelve.py", line 122, in __getitem__
    value = Unpickler(f).load()
AttributeError: 'module' object has no attribute 'User'

Answer 1

Fix:

import SE_ex2
from SE_ex2 import User

SE_ex2.cacheTest()

Pickled class needs to be in the namespace

The reason is shelf uses pickle, so it uses the User class:

When working with your own classes, you must ensure that the class being pickled appears in the namespace of the process reading the pickle. Only the data for the instance is pickled, not the class definition. The class name is used to find the constructor to create the new object when unpickling. Take this example, which writes instances of a class to a file:

quoted: http://www.doughellmann.com/PyMOTW/pickle/index.html#problems-reconstructing-objects

Answer 2

I've had similar troubles with pickle before. Try putting the User class in another file, so you're importing it from the same module in both of these files. I think it's getting shelfed as User, but when you unshelf it you're expecting it to figure out that SE_ex2.User is what it had been calling User. It also works to import the User class explicitly, as in

from SE_ex2 import User
import SE_ex2
SE_ex2.cache_test()