Translating this to Common Lisp

Question

I've been reading an article by Olin Shivers titled Stylish Lisp programming techniques and found the second example there (labeled "Technique n-1") a bit puzzling. It describes a self-modifying macro that looks like this:

(defun gen-counter macro (x)
       (let ((ans (cadr x)))   
     (rplaca (cdr x)       
         (+ 1 ans))
     ans))

It's supposed to get its calling form as argument x (i.e. (gen-counter <some-number>)). The purpose of this is to be able to do something like this:

> ;this prints out the numbers from 0 to 9.
  (do ((n 0 (gen-counter 1)))
      ((= n 10) t)
    (princ n))
0.1.2.3.4.5.6.7.8.9.T
>

The problem is that this syntax with the macro symbol after the function name is not valid in Common Lisp. I've been unsuccessfully trying to obtain similar behavior in Common Lisp. Can someone please provide a working example of analogous macro in CL?

Answer 1

Common Lisp:

(defmacro gen-counter (&rest x)
  (let ((ans (car x)))   
    (rplaca x (+ 1 ans))
    ans))

But above only works in the Interpreter, not with a compiler.

With compiled code, the macro call is gone - it is expanded away - and there is nothing to modify.

Note to unsuspecting readers: you might want to read the paper by Olin Shivers very careful and try to find out what he actually means...

Answer 2

Why the code works

First, it's useful to consider the first example in the paper:

> (defun element-generator ()
    (let ((state '(() . (list of elements to be generated)))) ;() sentinel.
      (let ((ans (cadr state)))       ;pick off the first element
        (rplacd state (cddr state))   ;smash the cons
        ans)))
ELEMENT-GENERATOR
> (element-generator)
LIST
> (element-generator)
OF
> (element-generator)

This works because there's one literal list

(() . (list of elements to be generated)

and it's being modified. Note that this is actually undefined behavior in Common Lisp, but you'll get the same behavior in some Common Lisp implementations. See Unexpected persistence of data and some of the other linked questions for a discussion of what's happening here.

Approximating it in Common Lisp

Now, the paper and code you're citing actually has some useful comments about what this code is doing:

  (defun gen-counter macro (x)    ;X is the entire form (GEN-COUNTER n)
    (let ((ans (cadr x)))         ;pick the ans out of (gen-counter ans)
      (rplaca (cdr x)             ;increment the (gen-counter ans) form
              (+ 1 ans))
      ans))                       ;return the answer

The way that this is working is not quite like an &rest argument, as in Rainer Joswig's answer, but actually a &whole argument, where the the entire form can be bound to a variable. This is using the source of the program as the literal value that gets destructively modified! Now, in the paper, this is used in this example:

> ;this prints out the numbers from 0 to 9.
  (do ((n 0 (gen-counter 1)))
      ((= n 10) t)
    (princ n))
0.1.2.3.4.5.6.7.8.9.T

However, in Common Lisp, we'd expect the macro to be expanded just once. That is, we expect (gen-counter 1) to be replaced by some piece of code. We can still generate a piece of code like this, though:

(defmacro make-counter (&whole form initial-value)
  (declare (ignore initial-value))
  (let ((text (gensym (string 'text-))))
    `(let ((,text ',form))
       (incf (second ,text)))))

CL-USER> (macroexpand '(make-counter 3))
(LET ((#:TEXT-1002 '(MAKE-COUNTER 3)))
  (INCF (SECOND #:TEXT-1002)))

Then we can recreate the example with do

CL-USER> (do ((n 0 (make-counter 1)))
             ((= n 10) t)
           (princ n))
023456789

Of course, this is undefined behavior, since it's modifying literal data. It won't work in all Lisps (the run above is from CCL; it didn't work in SBCL).

But don't miss the point

The whole article is sort of interesting, but recognize that it's sort of a joke, too. It's pointing out that you can do some funny things in an evaluator that doesn't compile code. It's mostly satire that's pointing out the inconsistencies of Lisp systems that have different behaviors under evaluation and compilation. Note the last paragraph:

Some short-sighted individuals will point out that these programming techniques, while certainly laudable for their increased clarity and efficiency, would fail on compiled code. Sadly, this is true. At least two of the above techniques will send most compilers into an infinite loop. But it is already known that most lisp compilers do not implement full lisp semantics -- dynamic scoping, for instance. This is but another case of the compiler failing to preserve semantic correctness. It remains the task of the compiler implementor to adjust his system to correctly implement the source language, rather than the user to resort to ugly, dangerous, non-portable, non-robust ``hacks'' in order to program around a buggy compiler.

I hope this provides some insight into the nature of clean, elegant Lisp programming techniques.

—Olin Shivers