Store many variables in a file

Question

I'm trying to store many variables in a file. I've tried JSON, pickle and shelve but they all seem to only take one variable

import shelve
myShelve = shelve.open('my.shelve')
myShelve.update(aasd,
    basd,
    casd,
    dasd,
    easd,
    fasd,
    gasd,
    hasd,
    iasd,
    jasd)
myShelve.close()

And pickle

import pickle
with open("vars.txt", "wb") as File:
    pickle.dumps(aasd,
        basd,
        casd,
        dasd,
        easd,
        fasd,
        gasd,
        hasd,
        iasd,
        jasd,
        File)

The errors I'm getting are along the lines of

TypeError: update() takes at most 2 positional arguments (11 given)

and

TypeError: pickle.dumps() takes at most 2 positional argument (11 given)

I'm not sure if there's any other way of storing variables except using a database, but that's a bit over what I'm currently capable of I'd say.

Answer 1

You can only pickle one variable at a time, but it can be a dict or other Python object. You could store your many variables in one object and pickle that object.

import pickle

class Box:
    pass

vars = Box()

vars.x = 1
vars.y = 2
vars.z = 3

with open("save_vars.pickle", "wb") as f:
    f.write(pickle.dumps(vars))

with open("save_vars.pickle", "rb") as f:
    v = pickle.load(f)

assert vars.__dict__ == v.__dict__

Answer 2

using pickle, you dump one object at a time. Each time you dump to the file, you add another "record".

import pickle
with open("vars.txt", "wb") as File:
    for item in (aasd,
                 basd,
                 casd,
                 dasd,
                 easd,
                 fasd,
                 gasd,
                 hasd,
                 iasd,
                 jasd)
    pickle.dump(item,File)

Now, on when you want to get your data back, you use pickle.load to read the next "record" from the file:

import pickle
with open('vars.txt') as fin:
   aasd = pickle.load(fin)
   basd = pickle.load(fin)
   ...

---

Alternatively, depending on the type of data, assuming the data is stuff that json is able to serialize, you can store it in a json list:

import json
# dump to a string, but you could use json.dump to dump it to a file.
json.dumps([aasd,
            basd,
            casd,
            dasd,
            easd,
            fasd,
            gasd,
            hasd,
            iasd,
            jasd])

Answer 3

EDIT: I just thought of a different way to store your variables, but it is a little weird, and I wonder what the gurus think about this.

You can save a file that has the python code of your variable definitions in it, for example vars.py which consists of simple statements defining your values:

x = 30
y = [1,2,3]

Then to load that into your program, just do from vars import * and you will have x and y defined, as if you had typed them in.

---

Original normal answer below...

There is a way using JSON to get your variables back without redefining their names, but you do have to create a dictionary of variables first.

import json

vars={}  # the dictionary we will save.

LoL = [ range(5), list("ABCDE"), range(5) ]
vars['LOList'] = LoL
vars['x'] = 24
vars['y'] = "abc"

with open('Jfile.txt','w') as myfile:
    json.dump(vars,myfile,indent=2)

Now to load them back:

with open('Jfile.txt','r') as infile:
    D = json.load(infile)

    # The "trick" to get the variables in as x,y,etc:
    globals().update(D)

Now x and y are defined from their dictionary entries:

print x,y
24 abc

There is also an alternative using variable-by-variable definitions. In this way, you don't have to create the dictionary up front, but you do have to re-name the variables in proper order when you load them back in.

z=26
w="def"

with open('Jfile2.txt','w') as myfile:
    json.dump([z,w],myfile,indent=2)

with open('Jfile2.txt','r') as infile:
    zz,ww = json.load(infile)

And the output:

print zz,ww
26 def