scalaz Iteratees from scala Iterator

Question

I edited the code below as I believe I had been combining the IterV objects incorrectly on top of the iter.next issue.

I'm experimenting with Iteratee in scalaz and I am wondering why the following does not work. Here is what I have:

import scalaz._
import Scalaz._
import IterV._

implicit val iteratorEnumerator = new Enumerator[Iterator] {
  def apply[E,A](iter: Iterator[E], i: IterV[E,A]): IterV[E,A] =
    if (iter.isEmpty) i
    else i.fold(done = (acc,input) => i, 
                cont = k => apply(iter, k(El(iter.next))))
}

/* probably incorrect
val iter = Iterator(1,2,3)
println("peek(iter) " + peek(iter).run)
println("peek(iter) " + peek(iter).run)
*/

def peekpeek[E]: IterV[E, (Option[E],Option[E])] = 
  for (a <- peek; b <- peek) yield (a,b)
def peekheadpeek[E]: IterV[E, (Option[E],Option[E],Option[E])] = 
  for (a <- peek; b <- head; c <- peek) yield (a,b,c)

peekpeek(Iterator(1,2,3,4)).run
peekheadpeek(Iterator(1,2,3,4)).run

This returns:

res0: (Option[Int], Option[Int]) = (Some(1),Some(2)) 
res1: (Option[Int], Option[Int], Option[Int]) = (Some(1),Some(2),Some(3)) 

Where I was expecting (Some(1),Some(1)) and (Some(1),Some(1),Some(2)).

I suspect this has to do with iter.next being side effecting. What's the best way to deal with that?

For comparison, this taken directly from the source code examples from the scalaz web site works properly:

implicit val StreamEnumerator = new Enumerator[Stream] {
  def apply[E, A](e: Stream[E], i: IterV[E, A]): IterV[E, A] = e match {
    case Stream() => i
    case x #:: xs => i.fold(done = (_, _) => i, 
                            cont = k => apply(xs, k(El(x))))
  }
}

Answer 1

I think I figured this out. It seemed to be primarily due to El using a by name parameter, which recomputes iter.next - as well as how I initially called the computation incorrectly with two different peek(iter).run. I rewrote the enumerator to first assign iter.next to a val (and also made it tail recursive in the process):

implicit val iteratorEnumerator = new Enumerator[Iterator] {
  @annotation.tailrec
  def apply[E,A](iter: Iterator[E], i: IterV[E,A]): IterV[E,A] = i match {
    case _ if iter.isEmpty => i
    case Done(acc, input) => i
    case Cont(k) => 
      val x = iter.next
      apply(iter, k(El(x)))
  }
}

Then:

def peekpeek[A] = 
  for (a1 <- peek[A]; a2 <- peek[A]) yield (a1,a2)
def peekheadpeek[A] = 
  for (a1 <- peek[A]; a2 <- head[A]; a3 <- peek[A]) yield (a1,a2,a3)
def headpeekhead[A] = 
  for (a1 <- head[A]; a2 <- peek[A]; a3 <- head[A]) yield (a1,a2,a3)

peekpeek(Iterator(1,2,3)).run
peekheadpeek(Iterator(1,2,3)).run
headpeekhead(Iterator(1,2,3)).run
length(Iterator.from(1).take(1000000)).run

would return this:

res13: (Option[Int], Option[Int]) = (Some(1),Some(1))
res14: (Option[Int], Option[Int], Option[Int]) = (Some(1),Some(1),Some(2))
res15: (Option[Int], Option[Int], Option[Int]) = (Some(1),Some(2),Some(2))
res16: Int = 1000000

Answer 2

You're correct about iter.next causing side effects. I think that the question boils down to what is the difference between a Stream and an Iterator. This question has relevant information.

Answer 3

Iteratees are lazy. Side-effects (iterator) and laziness do not mix. Maybe this will do the right thing:

implicit val iteratorEnumerator = new Enumerator[Iterator] {
  def apply[E,A](iter: Iterator[E], i: IterV[E,A]): IterV[E,A] =
    iter.foldRight(i)((x, y) =>
      y.fold(done = (acc, input) => y,
             cont = k => apply(iter, k(El(x))))
    )
}

Then again, maybe not. Only the source of foldRight will know. Side-effects are like that.

Answer 4

You can avoid side effects by creating a throwaway iterator of size one:

implicit val iteratorEnumerator = new Enumerator[Iterator] {
  def apply[E,A](iter: Iterator[E], i: IterV[E,A]): IterV[E,A] =
    if (iter.isEmpty) i
    else i.fold(done = (acc,input) => i, 
                cont = k => apply(iter, k(El(iter.take(1).next))))
}