Code Generation in Clojure

Question

(Disclaimer: I am a C# guy. I've just started learning Clojure.)

I understand that a Clojure program has the ability to manipulate itself or easily generate other programs. It has something to do with everything in Clojure being a data structure and that generating programs would be the same as creating any other type of data structure.

Does anybody have a good sample program (or a reference to one) that shows this?

If you generate a program, can you "serialize" that program out to disk for later execution?

Just for reference:

1. I'm trying to play with Genetic Programming. I want to generate a lot of little programs, evaluate them, and use the successful ones to generate more programs. See more here and here.

2. I think I'm misusing terms here. By program I actually mean a clojure list and by Code Generation I mean "List Generation". I just need the list to contain actual function calls and parameters. I would need to be able to control when this list gets "executed".

Answer 1

Consider (+ 1 2). As data, it's a linked list of three items: the Symbol + and two Integers. As code, it's a function call, saying "Call the function called + with these two Integers as arguments and give me the result". You can do anything to this list that you can do to any other list of data. You can also eval it to get a result.

user> (def x '(+ 1 2))
#'user/x
user> (first x)
+
user> (rest x)
(1 2)
user> (map class x)
(clojure.lang.Symbol java.lang.Integer java.lang.Integer)
user> (reverse x)
(2 1 +)
user> (concat x (rest x))
(+ 1 2 1 2)
user> (eval x)
3
user> (defn foo []
        (let [ops '[+ - * /]               ; SO's lisp-highlighting sucks
              nums (repeatedly #(rand-int 5))
              expr (list* (rand-elt ops) (take 10 nums))]
          (prn expr)
          (prn (eval expr))))
user> (foo)
(+ 4 1 0 3 2 3 4 3 1 2)
23
nil
user> (foo)
(- 1 3 2 2 1 2 1 4 0 1)
-15
nil

Answer 2

Clojure is a LISP, and that means that it is a homoiconic language: there is no structural distinction between data and code. Its lists all the way down. It also has an extensible compiler which allows you to extend the syntax through macros. But its not clear from your problem statement that you really need such a thing.

You're basically running code that generates lists (which are really next gen programs), saving them, and then running the new programs. Unless your generational evolution will require new syntax, you probably wouldn't need to resort to macros.

Answer 3

Found a partial answer in this article:

The pr and prn functions are like their print and println counterparts, but their output is in a form that can be read by the Clojure reader. They are suitable for serializing Clojure data structures. By default, they do not print metadata. This can be changed by binding the special symbol *print-meta* to true.

This at least answers the second part of my question.

Answer 4

The question is somewhat misleading as Clojure also performs on-the-fly "code generation" as it compiles Clojure source in Java Byte Code.

In this particular case, I believe you are interesting in Lisp Macros in particular. I think these may be interesting:

Clojure documentation itself

Video, Macros (in Clojure) in 20 minutes

Standard issue: Wikipedia - Clojure

Note that macros in Clojure work very similar to Common Lisp macros (a type-2 lisp), and not quite-so-much-like Scheme macros.

Happy coding.

Answer 5

Take a look at macros. For example,

(defmacro defmacro-
  "Same as defmacro but yields a private definition"
  [name & decls]
  (list* `defmacro (with-meta name (assoc (meta name) :private true)) decls))

With macros, you don't need to serialize out the macroexpansion; the compile will use it automatically.