Short-circuiting list comprehensions [duplicate]

Question

This question already has answers here:

Using while in list comprehension or generator expressions (2 answers)

Closed 6 years ago.

On several occasions I've wanted python syntax for short-circuiting list comprehensions or generator expressions.

Here is a simple list comprehension, and the equivalent for loop in python:

my_list = [1, 2, 3, 'potato', 4, 5]
[x for x in my_list if x != 'potato']

result = []
for element in my_list:
    if element != 'potato':
        result.append(element)

There isn't support in the language for a comprehension which short-circuits. Proposed syntax, and equivalent for loop in python:

[x for x in my_list while x != 'potato']
# --> [1, 2, 3]

result = []
for element in my_list:
    if element != 'potato':
        result.append(element)
    else:
        break

It should work with arbitrary iterables, including infinite sequences, and be extendible to generator expression syntax. I am aware of list(itertools.takewhile(lambda x: x != 'potato'), my_list) as an option, but:

• it's not particularly pythonic - not as readable as a while comprehension
• it probably can't be as efficient or fast as a CPython comprehension
• it requires an additional step to transform the output, whereas that can be put into a comprehension directly, e.g. [x.lower() for x in mylist]
• even the original author doesn't seem to like it much.

My question is, was there any theoretical wrinkle about why it's not a good idea to extend the grammar to this use case, or is it just not possible because python dev think it would be rarely useful? It seems like a simple addition to the language, and a useful feature, but I'm probably overlooking some hidden subtleties or complications.

Related: this and this

Answer 1

Turns out, as @Paul McGuire noted, that it had been proposed in PEP 3142 and got rejected by Guido:

I didn't know there was a PEP for that. I hereby reject it. No point wasting more time on it.

He doesn't give explanations, though. In the mailing list, some of the points against it are:

• "[comprehension are] a carefully designed 1 to 1 transformation between multiple nested statements and a single expression. But this proposal ignores and breaks that. (here)." That is, the while keyword does not correspond to a while in the explicit loop - it is only a break there.
• "It makes Python harder to learn because it adds one more thing to learn." (cited here)
• It adds another difference between generator expressions and list-comprehension. Seems like adding this syntax to list comprehension too is absolutely out of the question.

I think one basic difference from the usual list comprehension is that while is inherently imperative, not declarative. It depends and dictates an order of execution, which is not guaranteed by the language (AFAIK). I guess this is the reason it is not included in Haskell's comprehensions, from which Python stole the idea.

Of course, generator expressions do have direction, but their elements may be precomputed - again, AFAIK. The PEP mentioned did propose it only for generator expressions - which makes some sense.

Of course, Python is an imperative language anyway, but it will raise problems.

What about choosing out of a non-ordered collection?

[x for x in {1,2,3} while x!=2]

You don't have it in simple for loops too, but that's something you can't enforce by the language. takewhile answers this question, but it is an arbitrary answer.

---

One last point, note that for consistency you will want support for dropwhile. something like

[x for x in my_list from x != 'potato']

Which is even less related to the equivalent for construct, and this time it is not possibly short circuit if my_list is just an iterable.