I'd do it the first way, simply to avoid closing and re-opening the same output file over and over. I doubt the fine details of exactly how much memory each approach uses matter, and if those details do matter so much, you need to write code in something other than Python.
As a general rule, file access is slower than memory, so I wouldn't worry about memory usage at all here unless you have an actual problem you can demonstrate.