Problem with file concatenation in Python?

Question

I have 3 files 1.txt, 2.txt, and 3.txt and I am trying to concatenate together the contents of these files into one output file in Python. Can anyone explain why the code below only writes the content of 1.txt and not 2.txt or 3.txt? I'm sure it's something really simple, but I can't seem to figure out the problem.

import glob
import shutil

for my_file in glob.iglob('/Users/me/Desktop/*.txt'):
    with open('concat_file.txt', "w") as concat_file:
        shutil.copyfileobj(open(my_file, "r"), concat_file)

Thanks for the help!

Answer 1

you constantly overwrite the same file.

either use:

with open('concat_file.txt', "a")

or

with open('concat_file.txt', "w") as concat_file:
    for my_file in glob.iglob('/Users/me/Desktop/*.txt'):
        shutil.copyfileobj(open(my_file, "r"), concat_file)

Answer 2

I believe that what's wrong with your code is that in every loop iteration, you are essentially adding files to themselves.

If you manually unroll the loop you will see what I mean:

# my_file = '1.txt'
concat_file = open(my_file)
shutil.copyfileobj(open(my_file, 'r'), concat_file)
# ...

I'd suggest deciding beforehand which file you want all the files to be copied to, maybe like this:

import glob
import shutil

output_file = open('output.txt', 'w')

for my_file in glob.iglob('/Users/me/Desktop/*.txt'):
    with open('concat_file.txt', "w") as concat_file:
        shutil.copyfileobj(open(my_file, "r"), output_file)