A substite for "for looping" in text files

Question

from unidecode import *
reader = open("a.txt",'w')
def unite():
    for line in reader:
        line = unidecode(line)
        print (line)
unite()

Now, I get an error saying for looping is not allowed in write mode. Is there any other way, I can modify each individual line like so, that it can be converted using unidecode?

Answer 1

You could hold it all in memory.

from unidecode import *

reader = open("a.txt",'w')
lines = reader.readlines()
def unite(lines):
    for line in lines:
        line = unidecode(line)
        print (line)
unite()

You could also use a temp file.

from unidecode import *
import os

reader = open('a.txt','r')
temp = open('~a.txt', 'w')
for line in reader():
    line = unidecode(line)
    temp.write(line)
reader.close()
temp.close()

os.remove('a.txt')
os.rename('~a.txt', 'a.txt')

Answer 2

You can open it in append mode:

def unite():
    with open('somefile.txt','a+') as f:
        for line in f:
            f.write(unidecode(line))
            print line

unite()

This will write stuff to the end of the file. To write stuff from the start of the file, use mode r+.

For example:

sample.txt:

hello world

When you run this:

with open('sample.txt','a+') as f:
    line = f.readline()
    f.write('{} + again'.format(line.strip()))

The file will have:

hello world
hello world again

If you run:

with open('sample.txt','r+') as f:
    line = f.readline()
    f.write('{} + once more'.format(line.strip()))

The file will have:

hello world
hello world once more
hello world again

If you want to replace the contents of the file, then you can read the file, save the lines, close it and open it in write mode to write the lines back.

Answer 3

It is a little dirty secret, but very few applications are able to really change a file "in place". Most of the time it looks like the application is modifying the file, but under the hood the edited file is written to a temporary location and then moved over to replace the original.

If you think about it, when you are inserting a few bytes in the middle of the file, you have to rewrite the whole file from this point, anyway.

Since the ascii output tends to be smaller then the unicode input, you probably can pull off something like this (unix only, I guess):

    #!/usr/bin/env python
    import os
    from unidecode import unidecode

    def unidecode_file(filename):
        # open for write without truncating
        fd = os.open(filename, os.O_WRONLY) 
        pos = 0 # keep track of file length
        # open for read
        with open(filename) as input:
            for line in input:
                ascii = unidecode(line.decode('utf-8'))
                pos += len(ascii)
                os.write(fd, ascii)
        os.ftruncate(fd, pos) # truncate output
        os.close(fd) # that is all, folks

    if __name__ == '__main__':
        unidecode_file('somefile.txt')

This stunt is not safe and is not the canonical way to edit a file (you will sure run into trouble if the output is bigger than input). Use the tempfile approach suggested by Drew, but ensure filename uniqueness (the most secure way is to generate a random filename for temporary files).