Strange (?) for comprehension evaluation in Scala

Question

Now, it took me a while to figure out why my recursion is somehow managing to blow the stack. Here it is, the part causing this problem:

scala> for {
     |   i <- List(1, 2, 3)
     |   j = { println("why am I evaluated?"); 10 } if false
     | } yield (i, j)
why am I evaluated?
why am I evaluated?
why am I evaluated?
res0: List[(Int, Int)] = List()

Isn't this, like, insane? Why at all evaluate j = ... if it ends in if false and so will never be used?

What happens when instead of { println ... } you have a recursive call (and recursion guard instead of if false), I have learned. :<

Why?!

Answer 1

If you structure your loop like this, it will solve your problem:

scala> for {
     |   i <- List(1, 2, 3)
     |   if false
     |   j = { println("why am I evaluated?"); 10 }
     | } yield (i, j)
res0: List[(Int, Int)] = List()

Scala syntax in a for-loop treats the if statement as a sort of filter; this tutorial has some good examples.

One way to think of it is to walk through the for loop imperatively, and when you reach an if statement, if that statement evaluates to false, you continue to the next iteration of the loop.

Answer 2

I'm going to go out on a limb and say the accepted answer could say more.

This is a parser bug.

Guards can immediately follow a generator, but otherwise a semi is required (actual or inferred).

Here is the syntax.

In the following, the line for res4 should not compile.

scala> for (i <- (1 to 5).toList ; j = 2 * i if j > 4) yield j
res4: List[Int] = List(6, 8, 10)

scala> for (i <- (1 to 5).toList ; j = 2 * i ; if j > 4) yield j
res5: List[Int] = List(6, 8, 10)

What happens is that the val def of j gets merged with the i generator to make a new generator of pairs (i,j). Then the guard looks like it just follows the (synthetic) generator.

But the syntax is still wrong. Syntax is our friend! It was our BFF long before the type system.

On the line for res5, it's pretty obvious that the guard does not guard the val def.

Update:

The implementation bug was downgraded (or upgraded, depending on your perspective) to a specification bug.

Checking for this usage, where a guard looks like a trailing if controlling the valdef that precedes it, like in Perl, falls under the purview of your favorite style checker.

Answer 3

When I have questions like that I seek to see how the disassembled code looks like (feeding the .class files to JD-GUI for instance).

The beginning of this for-comprehension disassembled code looks like this:

((TraversableLike)List..MODULE$.apply(Predef..MODULE$.wrapIntArray(new int[] { 1, 2, 3 })).map(new AbstractFunction1() { public static final long serialVersionUID = 0L;

      public final Tuple2<Object, BoxedUnit> apply(int i) { Predef..MODULE$.println("why am I evaluated?"); BoxedUnit j = BoxedUnit.UNIT;

        return new Tuple2(BoxesRunTime.boxToInteger(i), 
          j);
      }
    }...//continues

where we can see that the array of ints in the i parameter maps to an AbstractFunction1() whose apply method first performs the println nomatter what and then allocates Unit to the parameter j finally returning a tuple of two(i,j) to further pipe it into further filter/map operations (omitted). So essentially the if false condition doesn't have any effect and essentially is removed by the compiler.