How to access Aʟʟ global variables in a function?

Question

I'm trying to mimic the matlab load and save functions. I'm following this thread: Shelve Code gives KeyError

It is smart. However, if I write that code in a separate module, and try to import that module and invoke that function, then it can't access the global variables.

Specifically, I write a happy.py and have the functions inside:

def save(filename='tmp', globals_=None):
    if globals_ is None:
        globals_ = globals()
        
    globals()
    import shelve
    my_shelf = shelve.open(filename, 'n')
    for key, value in globals_.items():
        if not key.startswith('__'):
            try:
                my_shelf[key] = value
            except Exception:
                print('ERROR shelving: "%s"' % key)
            else:
                print('shelved: "%s"' % key)
    my_shelf.close()

def load(filename='tmp', globals_=None):
    import shelve
    my_shelf = shelve.open(filename)
    for key in my_shelf:
        globals()[key] = my_shelf[key]
    my_shelf.close()

and when I try

a = 1
b = 2
happy.save()

It would not give save a and b.

Is this because global() would not give the objects outside the module? How can I do what I want to do then?

Answer 1

You can use inspect to look at the stack. This silly (poorly named function) that I've defined seems to do an OK job of picking up the global variables from the calling namespace although I haven't tested it extensively. I am also unsure about whether it will work with different python implementations. (I mention this because the inspect.currentframe function is definitely implementation dependent). It seems to work OK with Cpython for what it's worth.

import inspect
def save(globals=None):
    if globals is None:
        frames = inspect.stack()
        caller_frame = frames[-1][0]
        globals = dict((k,v) for (k,v) in caller_frame.f_globals.items() if not k.startswith('__'))
    return globals


if __name__ == "__main__":
    a = 1
    b = 2
    print save()

Answer 2

The following will work as a separate module:

import shelve
import sys
import types

EXCLUDED_TYPES = (types.ModuleType,)  # Everything can't be shelved.

def save(filename='tmp', globals_=None):
    if globals_ is None:
        globals_ = sys._getframe(1).f_globals  # Caller's globals.

    with shelve.open(filename, 'n') as my_shelf:
        for key, value in globals_.items():
            if not (key.startswith('__') or isinstance(value, EXCLUDED_TYPES)):
                try:
                    my_shelf[key] = value
                except Exception as e:
                    print('ERROR shelving: "%s"' % key, 'Exception:', e)
                else:
                    print('shelved: "%s"' % key)

def load(filename='tmp', globals_=None):
    if globals_ is None:
        globals_ = sys._getframe(1).f_globals  # Caller's globals.

    with shelve.open(filename) as my_shelf:
        for key in my_shelf:
            globals_[key]=my_shelf[key]
            print('unshelved: "%s"' % key)

Generally speaking, I don't think it's a good idea for a function to create global variables like this. Also note that load() might silently change existing values in the caller's namespace.

You can't easily save all global namespaces, since there's one associated with every module loaded, in addition to __main__'s. If you really want to do that, it might be possible to do so by iterating through the contents of sys.modules.

Answer 3

I don't have a problem with this code when it is pasted into an console:

>>> def save(filename='tmp',globals_=None):
...     import shelve
...     globals_ = globals_ or globals()
...     my_shelf=  shelve.open(filename, 'n')
...     for key, value in globals_.items():
...         if not key.startswith('__'):
...             try:
...                 my_shelf[key] = value
...             except Exception:
...                 print('ERROR shelving: "%s"' % key)
...             else:
...                 print('shelved: "%s"' % key)
...     my_shelf.close()
... 
>>> def load(filename='tmp',globals_=None):
...     import shelve
...     my_shelf = shelve.open(filename)
...     for key in my_shelf:
...         globals()[key]=my_shelf[key]
...     my_shelf.close()
... 
>>> a, b = 1, 2
>>> save()
shelved: "load"
shelved: "a"
shelved: "b"
shelved: "save"

And then:

>>> def save(filename='tmp',globals_=None):
...     import shelve
...     globals_ = globals_ or globals()
...     my_shelf=  shelve.open(filename, 'n')
...     for key, value in globals_.items():
...         if not key.startswith('__'):
...             try:
...                 my_shelf[key] = value
...             except Exception:
...                 print('ERROR shelving: "%s"' % key)
...             else:
...                 print('shelved: "%s"' % key)
...     my_shelf.close()
... 
>>> def load(filename='tmp',globals_=None):
...     import shelve
...     my_shelf = shelve.open(filename)
...     for key in my_shelf:
...         globals()[key]=my_shelf[key]
...     my_shelf.close()
... 
>>> load()
>>> a, b
(1, 2)

But it is a bit odd when you use it as a module:

>>> from happy import *
>>> a, b
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'a' is not defined
>>> load()
>>> a, b
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'a' is not defined
>>> happy.a
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'happy' is not defined
>>> from happy import *
>>> a, b
(1, 2)

Is there enough here for you to have a work-around?