Using “with” statement for CSV files in Python
-
22-07-2019 - |
Question
Is it possible to use the with
statement directly with CSV files? It seems natural to be able to do something like this:
import csv
with csv.reader(open("myfile.csv")) as reader:
# do things with reader
But csv.reader doesn't provide the __enter__
and __exit__
methods, so this doesn't work. I can however do it in two steps:
import csv
with open("myfile.csv") as f:
reader = csv.reader(f)
# do things with reader
Is this second way the ideal way to do it? Why wouldn't they make csv.reader directly compatible with the with statement?
Solution
The primary use of with
statement is an exception-safe cleanup of an object used in the statement. with
makes sure that files are closed, locks are released, contexts are restored, etc.
Does csv.reader have things to cleanup in case of exception?
I'd go with:
with open("myfile.csv") as f:
for row in csv.reader(f):
# process row
You don't need to submit the patch to use csv.reader
and with
statement together.
import contextlib
Help on function contextmanager in module contextlib:
contextmanager(func)
@contextmanager decorator.
Typical usage:
@contextmanager
def some_generator(<arguments>):
<setup>
try:
yield <value>
finally:
<cleanup>
This makes this:
with some_generator(<arguments>) as <variable>:
<body>
equivalent to this:
<setup>
try:
<variable> = <value>
<body>
finally:
<cleanup>
Here's a concrete example how I've used it: curses_screen.
OTHER TIPS
Yes. The second way is correct.
As to why? Who ever knows. You're right, it's probably an easy change. It's not as high priority as other things.
You can easily make your own patch kit and submit it.
The problem is csv.reader doesn't really manage a context. It can accept any iterable, not just a file. Therefore it doesn't call close on its input (incidentally if it did you could use contextlib.closing). So it's not obvious what context support for csv.reader would actually do.
import csv
class CSV(object):
def __init__(self,path,mode):
self.path = path
self.mode = mode
self.file = None
def __enter__(self):
self.file = open(self.path,self.mode)
if self.mode == 'r':
return csv.reader(self.file)
elif self.mode == 'w':
return csv.writer(self.file)
def __exit__(self, exc_type, exc_val, exc_tb):
self.file.close()
with CSV('data.csv','r') as reader:
for row in reader:
print row
It's easy to create what you want using a generator function:
import csv
from contextlib import contextmanager
@contextmanager
def opencsv(path):
yield csv.reader(open(path))
with opencsv("myfile.csv") as reader:
# do stuff with your csvreader