Question

I have a problem where changes in a list are not reflected in the associated shelf (Python 2.6.9). Consider the following minimal example

File Context.py:

import shelve

class Cont:
    shelf = shelve.open( STATUSFILE, protocol = 0, writeback = True )

File test.py:

from Context import Cont

class T:
    jobQueue = list()

    @classmethod
    def save(cls):
        cls.jobQueue = [1,2,3]
        Cont.shelf['queue'] = cls.jobQueue
        Cont.shelf.sync()
        print(Cont.shelf['queue'])
        cls.jobQueue.pop()
        print(Cont.shelf['queue'])
        print(id(cls.jobQueue))
        print(id(Cont.shelf['queue']))

Output:

[1,2,3]
[1,2,3]
7804904
7899472

First, I assumed the second list output to be[1,2]. Second, why are the ids different? When assigning the list, only a reference should be copied, keeping the id intact.
The strange thing is that I cannot reproduce that on the Python shell. There the output ids are the same and the shelfed list shows the change on jobQueue.
It seems that when executing my program the writeback=True is ignored. I am greatful for every hint!

EDIT: I did reproduce the problem on the shell with the given minimal example. May this have something to do with the class structure (I'm pretty new to objective Python)? I can imagine that class Cont does not know class T and hence cannot store a reference to T.jobQueue but rather a copy of the list.

Was it helpful?

Solution

Without shelf.sync():

def save(cls):
    Cont.shelf['queue'] = cls.jobQueue   #1
    print(Cont.shelf['queue'])           #2
    # [1, 2, 3]        
    cls.jobQueue.pop()                   #3
    print(Cont.shelf['queue'])           #4
    # [1, 2]
  1. With writeback=True, assignments store the key/value pair in shelf.cache as well as shelf.dict.
  2. Attempts to retrieve the data at key 'queue' from shelf.cache.
  3. modifies cls.jobQueue, which is the same object as the one retrieved from the cache
  4. retrieves the data at key 'queue' from shelf.cache again. Since the cache holds a refence to cls.jobQueue, this is the same object.

However, if you call shelf.sync():

def save(cls):
    Cont.shelf['queue'] = cls.jobQueue   
    Cont.shelf.sync()                    #1
    print(Cont.shelf['queue'])           #2
    # [1, 2, 3]        
    cls.jobQueue.pop()                   #3
    print(Cont.shelf['queue'])           #4
    # [1, 2, 3]
  1. The shelve file is updated, and the cache is reset to an empty dict.
  2. Attempts to retrieve the data at key 'queue' from shelf.cache. Since the cache is empty, a new copy of the data is retrieved from the shelve file.
  3. modifies cls.jobQueue, which is not the same object as the copy just retrieved
  4. The cache is still empty, so this retrieves a new copy from the unupdated shelve file again

OTHER TIPS

After a lot of testing I found the problem:
The line Cont.shelf.sync() changes the id (and reference) of the data included in shelf. Hence the changes in cls.jobQueue will not be reflected after the .sync() call.
While the documentation of the shelve module does not forbid using it the way I did, it seems to work properly only the documented way. This means that all changes should be made like this:
Cont.shelf['queue'].pop()
or by reassignin jobQueue to the shelf after every modification.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top