Clojure Agent Parellel HTTP IllegalStateException and await-for

Question

I am working through the example of making parallel http requests in Clojure,

http://lethain.com/a-couple-of-clojure-agent-examples/

In particular

(ns parallel-fetch
  (:import [java.io InputStream InputStreamReader BufferedReader]
           [java.net URL HttpURLConnection]))

(defn get-url [url]
  (let [conn (.openConnection (URL. url))]
    (.setRequestMethod conn "GET")
    (.connect conn)
    (with-open [stream (BufferedReader.
                       (InputStreamReader. (.getInputStream conn)))]
      (.toString (reduce #(.append %1 %2)
                          (StringBuffer.) (line-seq stream))))))

(defn get-urls [urls]
  (let [agents (doall (map #(agent %) urls))]
    (doseq [agent agents] (send-off agent get-url))
    (apply await-for 5000 agents)
    (doall (map #(deref %) agents))))

(prn (get-urls '("http://lethain.com" "http://willarson.com"))) 

When I run this in the

IllegalStateException await-for in transaction 

What does this mean and how do I fix it?

Answer 1

Taking the comment on the question into account:

A transaction is being set up in the process of loading your namespace, and since it has a call to get-urls at the top-level, the await-for happens in that transaction and throws the exception.

The best way to fix that is to put the prn / get-urls form inside a function and only call it once the namespace is loaded. (If you wanted to run this code as a standalone app, with lein run or java -jar on an überjar, you'd put a call to that function inside -main.)

Incidentally, the transaction is set up when you use :reload-all, but not without it. (See the private functions load-lib, which checks for the presence of :reload-all and decides to use the private function load-all if it's there, and load-all itself, which is where the transaction is set up. Here's a link to the 1.5.1 source.)