clojure rest and next related

Question

I was following The Joy of Clojure and I am puzzled with these 2 statements

(def very-lazy (-> (iterate #(do (print \.) (inc %)) 1) rest rest rest))

(def less-lazy (-> (iterate #(do (print \.) (inc %)) 1) next next next))

accordingly, the output was

(println (first very-lazy)) ; .4

(println (first less-lazy)) ; 4

The book continued to explain that

Grabbing the first element in a lazy seq built with rest causes a realization as expected. But the same doesn’t happen for a seq built with next because it’s already been previously realized. Using next causes a lazy seq to be one element less lazy, which might not be desired if the cost of realization is expensive.

My burning question is why there is an extra dot for 'very-lazy'? My thought was 'print' would print its argument when invoked regardless if next or rest.

Thanks

Answer 1

You're missing something important. Count the dots:

user> (def very-lazy (-> (iterate #(do (print \.) (inc %)) 1) rest rest rest))
..#'user/very-lazy

user> (def less-lazy (-> (iterate #(do (print \.) (inc %)) 1) next next next))
...#'user/less-lazy

It always helps to run the code for real instead of just reading a book.

Answer 2

Print actually did exactly the same thing in both cases, it printed only the number. The extra . was printed by the code inside the list, it just happened to occur at the same time as the printing of the 4 and so it ended up next to it on the screen.

the extra dot is a side effect of the lazy sequence being created on the fly. I would propose a more verbose example to clarify this:

start with two identical lists, both fully-lazy:

esc.core=> (def a (iterate #(do (print "making new element") (inc %)) 1)) 
#'esc.core/a
esc.core=> (def b (iterate #(do (print "making new element") (inc %)) 1))
#'esc.core/b

then make two more identical lists that start on the fourth element of a and b respectively

esc.core=> (def a-partially-realized (-> a rest rest rest))
making new elementmaking new element#'esc.core/a-partially-realised
esc.core=> (def b-more-fully-realized (-> b next next next))
making new elementmaking new elementmaking new element#'esc.core/b-more-fully-realised
esc.core=> 

the first three elements of a-partially-realized have been pre-computed
while the first four elements of b-more-fully-realized have been pre-computed.

when we read the first element (the fourth in the original list) of a-partially-realized it has not yet been computed so we will see it being computed.

esc.core=> (print (first a-partially-realized))
making new element4nil

when we do the same to b-more-fully-realised it will already have the value cached so we just get the result immediately.

esc.core=> (print (first b-more-fully-realized))
4nil

Answer 3

As Joost points out, the magic happens when you define very-lazy and less-lazy.

The point is, rest just has to return a sequence. That sequence is lazy. Therefore, (-> x rest rest rest) needs to process two values of x and then return the lazy sequence. Hence, two dots.

next, on the other hand, has to return nil if there are no more elements in the sequence, so it's got to evaluate x a third time to determine whether or not the sequence was empty. So three dots.

When you actually ask for the value, any remaining laziness needs to be discarded. So, you get one more dot for rest. The next version is done, so no more dots.

Answer 4

What is unanswered so far is why next is more eager than rest which is at the heart of the difference you describe. next returns nil for any list less than 2 in length. In contrast, rest returns the empty list in these scenarios. This suggest that when calling next on a list of 1 element we are implicitly done handling our sequence (in recursion for example) and thus we can safely assume it is time to realize the first element. That accounts for the early versus late realization of the final dot.

Answer 5

next executes the #() function as soon as it is called, whereas rest does not because it does not have to yet - it waits as long as possible - so no realization/execution occurs in this example until another rest is called, or the value needs to be returned, which is the case when printing.

I too missed the dots on first reading. Only two dots for rest because there hasn't been a good enough reason (yet) for the last rest to be realized.