Python to SQL: list of lists

Question

I am currently facing a problem in exporting a python list of list into mySQL DB. Iam using MySQLdb for the same and utilising the executemany command.

For example, I have a list of list:

k1=[[['Bob', 'Alfred', 'Jen'],
['123 Elm Street', '55 Ninth Ave', '1 Paved Rd'],
[00123, 34565, 30094],
['Newark', 'Salinas', 'Los Angeles'],
['NJ', 'CA', 'CA']],[['Bob1', 'Alfred1', 'Jen1'],
['123 Elm Street1', '55 Ninth Ave1', '1 Paved Rd1'],
[001231, 345651, 300941],
['Newark1', 'Salinas1', 'Los Angeles1'],
['NJ1', 'CA1', 'CA1']]]

and I try (after initialising the cursor and the database to use):

cursor.executemany('''INSERT INTO addresses
                 (name, street, zipcode, city, state)
                 VALUES
                 (%s, %s, %s, %s, %s)''',zip(k1))

and I get an error:

 _mysql_exceptions.ProgrammingError: not enough arguments for format string

Well, basically, I would like the names, street, zipcode, city and state from the list of list k1 to mySQL db table "addresses" with the corresponding field entries.

Would be greatful if anyone could point me in the right direction.

Answer 1

Your datastructure is really awkward. I'm assuming you wanted to transform k1 into this:

>>> sum([zip(*x) for x in k1], [])
<<< 
[('Bob', '123 Elm Street', 83, 'Newark', 'NJ'),
 ('Alfred', '55 Ninth Ave', 34565, 'Salinas', 'CA'),
 ('Jen', '1 Paved Rd', 30094, 'Los Angeles', 'CA'),
 ('Bob1', '123 Elm Street1', 665, 'Newark1', 'NJ1'),
 ('Alfred1', '55 Ninth Ave1', 345651, 'Salinas1', 'CA1'),
 ('Jen1', '1 Paved Rd1', 300941, 'Los Angeles1', 'CA1')]

Now you can do executemany:

sql = "INSERT INTO addresses (name, street, zipcode, city, state) VALUES (%s, %s, %s, %s, %s)"
data = sum([zip(*x) for x in k1], [])
cursor.executemany(sql, data)

Answer 2

I don't think this is really a SQL question; let's just concentrate on getting the data in the right form before you even try to call executemany().

Your k1 example is a list of 2 lists, each of which inner lists is also a list of lists, which are names/address components. So you actually have a list of lists of lists. I don't know what's the point of the outer layer of list, but the inner layer seems amenable to zip.

Ignoring that outer layer, I think what you want is

zip(*k1[0])

Which yields sane-looking results for the first element in k1. Again, I don't know what the intent is behind repeating the same data in k1[0] and k1[1] so I won't try to guess what you will do with the further elements in k1.

Again, before worrying about how to integrate this with your SQL database, just try playing with it in the python interpreter:

>>> k1=[[['Bob', 'Alfred', 'Jen'],['123 Elm Street', '55 Ninth Ave', '1 Paved Rd'],[00123, 34565, 30094],['Newark', 'Salinas', 'Los Angeles'],['NJ', 'CA', 'CA']],[['Bob1', 'Alfred1', 'Jen1'],['123 Elm Street1', '55 Ninth Ave1', '1 Paved Rd1'],[001231, 345651, 300941],['Newark1', 'Salinas1', 'Los Angeles1'],['NJ1', 'CA1', 'CA1']]]
>>> zip(k1)
[([['Bob', 'Alfred', 'Jen'], ['123 Elm Street', '55 Ninth Ave', '1 Paved Rd'], [83, 34565, 30094], ['Newark', 'Salinas', 'Los Angeles'], ['NJ', 'CA', 'CA']],), ([['Bob1', 'Alfred1', 'Jen1'], ['123 Elm Street1', '55 Ninth Ave1', '1 Paved Rd1'], [665, 345651, 300941], ['Newark1', 'Salinas1', 'Los Angeles1'], ['NJ1', 'CA1', 'CA1']],)]
>>> k1[0]
[['Bob', 'Alfred', 'Jen'], ['123 Elm Street', '55 Ninth Ave', '1 Paved Rd'], [83, 34565, 30094], ['Newark', 'Salinas', 'Los Angeles'], ['NJ', 'CA', 'CA']]
>>> k = k1[0]
>>> zip(k)
[(['Bob', 'Alfred', 'Jen'],), (['123 Elm Street', '55 Ninth Ave', '1 Paved Rd'],), ([83, 34565, 30094],), (['Newark', 'Salinas', 'Los Angeles'],), (['NJ', 'CA', 'CA'],)]
>>> zip(k[0], k[1])
[('Bob', '123 Elm Street'), ('Alfred', '55 Ninth Ave'), ('Jen', '1 Paved Rd')]
>>> zip(k[0], k[1], k[2], k[3], k[4])
[('Bob', '123 Elm Street', 83, 'Newark', 'NJ'), ('Alfred', '55 Ninth Ave', 34565, 'Salinas', 'CA'), ('Jen', '1 Paved Rd', 30094, 'Los Angeles', 'CA')]
>>> zip(*k)
[('Bob', '123 Elm Street', 83, 'Newark', 'NJ'), ('Alfred', '55 Ninth Ave', 34565, 'Salinas', 'CA'), ('Jen', '1 Paved Rd', 30094, 'Los Angeles', 'CA')]

Answer 3

I think you mean to pass each list inside k1 (which each themselves contain a few lists) to zip, like this:

>>> [zip(*x) for x in k1]
[[('Bob', '123 Elm Street', 83, 'Newark', 'NJ'), ('Alfred', '55 Ninth Ave', 34565, 'Salinas', 'CA'), ('Jen', '1 Paved Rd', 30094, 'Los Angeles', 'CA')], [('Bob1', '123 Elm Street1', 665, 'Newark1
', 'NJ1'), ('Alfred1', '55 Ninth Ave1', 345651, 'Salinas1', 'CA1'), ('Jen1', '1 Paved Rd1', 300941,   'Los Angeles1', 'CA1')]]
>>> 

Note too that your zip codes should be strings, not numbers, in order to preserve the leading zeroes, and avoid having them misparsed as things you don't intend (see the '83' zip code in there? It was 00123 I believe).

Answer 4

If you get your data structure to look like this: (not formatting but organization. You need to represent rows rather than columns to do inserts into SQL.)

[        
    ['Bob',     '123 Elm Street',  00123, 'Newark',     'NJ'],        
    ['Alfred',   '55 Ninth Ave',    34565, 'Salinas',   'CA'],
    ['Jen',     '1 Paved Rd',      30094, 'Los Angeles','CA']
]

your SQL INSERT may have a chance of working.