String formatting without index in python2.6

Question

I've got many thousands of lines of python code that has python2.7+ style string formatting (e.g. without indices in the {}s)

"{} {}".format('foo', 'bar')

I need to run this code under python2.6 which requires the indices.

I'm wondering if anyone knows of a painless way allow python2.6 to run this code. It'd be great if there was a from __future__ import blah solution to the problem. I don't see one. Something along those lines would be my first choice.

A distant second would be some script that can automate the process of adding the indices, at least in the obvious cases:

"{0} {1}".format('foo', 'bar')

Answer 1

It doesn't quite preserve the whitespacing and could probably be made a bit smarter, but it will at least identify Python strings (apostrophes/quotes/multi line) correctly without resorting to a regex or external parser:

import tokenize
from itertools import count
import re

with open('your_file') as fin:
    output = []
    tokens = tokenize.generate_tokens(fin.readline)
    for num, val in (token[:2] for token in tokens):
        if num == tokenize.STRING:
            val = re.sub('{}', lambda L, c=count(): '{{{0}}}'.format(next(c)), val)
        output.append((num, val))

print tokenize.untokenize(output) # write to file instead...

Example input:

s = "{} {}".format('foo', 'bar')
if something:
    do_something('{} {} {}'.format(1, 2, 3))

Example output (note slightly iffy whitespacing):

s ="{0} {1}".format ('foo','bar')
if something :
    do_something ('{0} {1} {2}'.format (1 ,2 ,3 ))

Answer 2

You could define a function to re-format your format strings:

def reformat(s):
    return "".join("".join((x, str(i), "}")) 
                   for i, x in list(enumerate(s.split("}")))[:-1])

Answer 3

Maybe a good old sed-regex like:

sed source.py -e 's/{}/%s/g; s/\.format(/ % (/'

your example would get changed to something like:

"%s %s" % ('foo', 'bar')

Granted you loose the fancy new style .format() but imho it's almost never useful for trivial value insertions.

Answer 4

A conversion script could be pretty simple. You can find strings to replace with regex:

fmt = "['\"][^'\"]*{}.*?['\"]\.format"

str1 = "x; '{} {}'.format(['foo', 'bar'])"
str2 = "This is a function; 'First is {}, second is {}'.format(['x1', 'x2']); some more code"
str3 = 'This doesn't have anything but a format. format(x)'
str4 = "This has an old-style format; '{0} {1}'.format(['some', 'list'])"
str5 = "'{0}'.format(1); '{} {}'.format(['x', 'y'])"

def add_format_indices(instr):
    text = instr.group(0)
    i = 0
    while '{}' in text:
        text = text.replace('{}', '{%d}'%i, 1)
        i = i+1
    return text

def reformat_text(text):
    return re.sub(fmt, add_format_indices, text)

reformat_text(str1)
"x; '{0} {1}'.format(['foo', 'bar'])"
reformat_text(str2)
"This is a function; 'First is {0}, second is {1}'.format(['x1', 'x2']); some more code"
reformat_text(str3)
"This doesn't have anything but a format. format(x)"
reformat_text(str4)
"This has an old-style format; '{0} {1}'.format(['some', 'list'])"
reformat_text(str5)
"'{0}'.format(1); '{0} {1}'.format(['x', 'y'])"

I think you could throw a whole file through this. You can probably find a faster implementation of add_format_indices, and obviously it hasn't been tested a whole lot.

Too bad there isn't an import __past__, but in general that's not something usually offered (see the 2to3 script for an example), so this is probably your next best option.