Python - CSV Reader List Comprehension

Question

I am trying to read the columns in a file efficiently using CSV reader. The code is:

import csv
csv.register_dialect('csvrd', delimiter='\t', quoting=csv.QUOTE_NONE)


with open('myfile.txt', 'rb') as f:
    reader = csv.reader(f,'csvrd')
    a0=[x[0] for x in reader]
    a1=[x[1] for x in reader]

I obtain the values in the first column, but a1 is empty. If I write a1 first, then a0 is empty.

I know a simple solution, inserting

reader=[x for x in reader]

But just curious about the reason. When you read an entry from the reader, is it removed?

A sample myfile.txt

c11 c21 c31
c21 c22 c32

Answer 1

You cannot loop over the reader more than once, not without rewinding the underlying file to the start again.

Don't do that, however; transpose the rows to columns using zip(*reader) instead:

a0, a1, a2 = zip(*reader)

Demo:

>>> import csv
>>> csv.register_dialect('csvrd', delimiter='\t', quoting=csv.QUOTE_NONE)
>>> data = '''\
... c11\tc21\tc31
... c21\tc22\tc32
... '''
>>> reader = csv.reader(data.splitlines(True), 'csvrd')
>>> a0, a1, a2 = zip(*reader)
>>> a0
('c11', 'c21')
>>> a1
('c21', 'c22')
>>> a2
('c31', 'c32')

Answer 2

csv.reader returns a generator. In order to read from a generator again, you will have to re-declare it. This answer has a detailed explanation on how Python's generators work.