Python CSV DictReader/Writer issues

Question

I'm trying to extract a bunch of lines from a CSV file and write them into another, but I'm having some problems.

import csv

f = open("my_csv_file.csv", "r")
r = csv.DictReader(f, delimiter=',')
fieldnames = r.fieldnames

target = open("united.csv", 'w')
w = csv.DictWriter(united, fieldnames=fieldnames)

while True:
try:
    row = r.next()
    if r.line_num <= 2: #first two rows don't matter
        continue
    else:
        w.writerow(row)

except StopIteration:
    break

f.close()
target.close()

Running this, I get the following error:

Traceback (most recent call last):
File "unify.py", line 16, in <module>
    w.writerow(row)
File "C:\Program Files\Python25\lib\csv.py", line 12
    return self.writer.writerow(self._dict_to_list(row
File "C:\Program Files\Python25\lib\csv.py", line 12
    if k not in self.fieldnames:
TypeError: argument of type 'NoneType' is not iterable

Not entirely sure what I'm dong wrong.

Answer 1

I don't know either, but since all you're doing is copying lines from one file to another why are you bothering with the csv stuff at all? Why not something like:

f = open("my_csv_file.csv", "r")
target = open("united.csv", 'w')

f.readline()
f.readline()
for line in f:
    target.write(line)

Answer 2

To clear up the confusion about the error: you get it because r.fieldnames is only set once you read from the input file for the first time using r. Hence the way you wrote it, fieldnames will always be initialized to None.

You may initialize w = csv.DictWriter(united, fieldnames=fieldnames) with r.fieldnames only after you read the first line from r, which means you would have to restructure your code.

This behavior is documented in the Python Standard Library documentation

DictReader objects have the following public attribute:

csvreader.fieldnames

If not passed as a parameter when creating the object, this attribute is initialized upon first access or when the first record is read from the file.

Answer 3

As for the exception, looks like this line:

w = csv.DictWriter(united, fieldnames=fieldnames)

should be

w = csv.DictWriter(target, fieldnames=fieldnames)

Answer 4

The reason you're getting the error is most likely that your original CSV file (my_csv_file.csv) doesn't have a header row. Therefore, when you construct the reader object, its fieldnames field is set to None.

When you try to write a row using the writer, it first checks to make sure there are no keys in the dict that are not in its list of known fields. Since fieldnames is set to None, an attempt to dereference the key name throws an exception.