Really weird issue with shelve (python)

Question

I create a file called foo_module.py containing the following code:

import shelve, whichdb, os

from foo_package.g import g

g.shelf = shelve.open("foo_path")
g.shelf.close() 

print whichdb.whichdb("foo_path")  # => dbhash
os.remove("foo_path")

Next to that file I create a directory called foo_package than contains an empty __init__.py file and a file called g.py that just contains:

class g:
    pass

Now when I run foo_module.py I get a weird error message:

Exception TypeError: "'NoneType' object is not callable" in ignored

But then, if I rename the directory from foo_package to foo, and change the import line in foo_module.py, I don't get any error. Wtf is going on here?

Running Python 2.6.4 on WinXP.

Answer 1

I think you've hit a minor bug in 2.6.4's code related to the cleanup at end of program. If you run python -v you can see exactly at what point of the cleanup the error comes:

# cleanup[1] foo_package.g
Exception TypeError: "'NoneType' object is not callable" in  ignored

Python sets references to None during the cleanup at the end of program, and it looks like it's getting confused about the status of g.shelf. As a workaround you could set g.shelf = None after the close. I would also recommend opening a bug in Python's bug tracker!

Answer 2

After days of hair loss, I finally had success using an atexit function:

  import atexit
  ...
  cache = shelve.open(path)
  atexit.register(cache.close)

It's most appropriate to register right after opening. This works with multiple concurrent shelves.

(python 2.6.5 on lucid)

Answer 3

This is indeed a Python bug, and I've posted a patch to the tracker issue you opened (thanks for doing that).

The problem is that shelve's del method calls its close method, but if the shelve module has already been through cleanup, the close method fails with the message you see.

You can avoid the message in your code by adding 'del g.shelf' after g.shelf.close. As long as g.shelf is the only reference to the shelf, this will result in CPython calling the shelve's del method right away, before the interpreter cleanup phase, and thus avoid the error message.

Answer 4

It seems to be an exception in a shutdown function registered by the shelve module. The "ignored" part is from the shutdown system, and might get its wording improved sometime, per Issue 6294. I'm still hoping for an answer on how to eliminate the exception itself, though...

Answer 5

for me a simple shelve.close() on an unclosed one did the job.

shelve.open('somefile') returns a "persistent dictionary for reading and writing" object which i used throughout the app's runtime. when I terminated the app I received the "TypeError" Exception as mentioned. I plased a 'close()' call in my termination sequence and that seemed to fix the problem.

e.g. shelveObj = shelve.open('fileName') ... shelveObj.close()

Answer 6

OverByThere commented on Jul 17, 2018

This seems to be fixable.

In short open /usr/lib/python3.5/weakref.py and change line 109 to:

 def remove(wr, selfref=ref(self), _atomic_removal=_remove_dead_weakref):

And line 117 to:

_atomic_removal(d, wr.key)

Note you need to do this with spaces, not tabs as this will cause other errors.