Down-converting dictionary comprehensions to generator expressions for Python 2.6 compliance [duplicate]

Question

This question already has an answer here:

Alternative to dict comprehension prior to Python 2.7 (1 answer)

Closed 8 years ago.

I have Python 2.7 code filled with dictionary comprehensions {x(i):y(i) for i in ...}. I have access to a compute cluster on which I can only run Python 2.6.6. The dictionary comprehensions are not part of the time-consuming part of the code, so I'm not very worried about speed.

I'm looking for the easiest way to make my code Python 2.6.6 and Python 2.7.6 compliant, preferably actually using the dictionary comprehensions where that is supported. Manually changing each expression to dict((x(i),y(i)) for i in ...) isn't preferred.

Answer 1

You can always write a source-processor that converts your 2.7 code to 2.6 code.

Or, just use one that someone else wrote. The 3to2 project has, among its fixers, one named dctsetcomp.* And it looks like it does exactly what you want. Here's a sample of its output:

RefactoringTool: Refactored dc.py
--- dc.py   (original)
+++ dc.py   (refactored)
@@ -1 +1 @@
-{x(i):y(i) for i in z}
+dict((x(i), y(i)) for i in z)
RefactoringTool: Files that need to be modified:
RefactoringTool: dc.py

There's not much documentation on how to use it, but basically it's meant to work exactly like 2to3, which is documented nicely, and used in hundreds of projects on PyPI if you need examples.

You can manually run this on your code before you're ready to distribute it, or run it as a pre-build step in setup.py (either unconditionally, or only if running on 2.6). setuptools makes it trivial to add a 2to3 step to the build/install process; subverting that to run 3to2 instead should be easy, but I haven't actually tried it.

---

You might want to look at the source to 3to2, or the stdlib's 2to3, if you want to write your own fixers, or if you need to debug/improve this one, or want to write a simpler parser from scratch.

---

A third place you can run a fixer is at runtime, with an import hook. Look at MacroPy as a great example of an import hook.

MacroPy is also a great example of code that works on ASTs instead of raw source. If you wanted to build a distribution-time fixer, this might be handy, because then you could let Python 2.7 parse the Python 2.7 source so you don't have to.

---

If 3to2 doesn't work out, you can probably do something a lot simpler, by limiting what you write in your code to what your parser can handle. For example, you could set yourself the rule that a dict comprehension must be the only dict display on the line, and be followed by #dc. Then you just have to search for lines ending with #dc, find the braces, and parse the result—which means you only have to parse a small subset of Python that's a regular language, which means you can do it with simple regexps.

---

* Why would a tool designed to port 3.2+ code to 2.7 have a dict comprehension fixer? Because it was originally designed to port 3.1+ code to 2.6, and the author never removed the no-longer-needed fixers. So, you got lucky. :)