Way to synchronize reads and writes in Clojure?

Question

In a web app I'm trying to generate a unique thread safe id from a limited id pool. The problem I'm facing is that between reading and writing another thread may already have changed the data structure; this is why I have to resort to compare-and-set!.

(def sid-batch 10)
(def sid-pool (atom {:cnt 0
                     :sids '()}))

(defn get-sid []
  (let [{:keys [cnt sids] :as old} @sid-pool]

    ; use compare-and-set! here for atomic read & write
    (if (empty? sids)

      ; generate more sids
      (if (compare-and-set!
            sid-pool
            old
            (-> old
              (assoc :sids (range (inc cnt) (+ sid-batch cnt)))
              (assoc :cnt (+ cnt sid-batch))))

        ; return newest sid or recur till "transaction" succeeds
        cnt
        (recur))

      ; get first sid
      (if (compare-and-set! sid-pool old (update-in old [:sids] next))

        ; return first free sid or recur till "transaction" succeeds
        (first sids)
        (recur)))))

Is there an easier way to synchronize reads and writes without having to perform STM "by hand" and without abusing a field in sid-pool as a return value from swap!?

Answer 1

You can do it with an atom, by adding a field to sid-pool in the way you seem to be suggesting. I agree that's a little gross, but using compare-and-swap! for something so simple is godawful. Instead, use the atom; or a ref, which lets you return whatever you want from a dosync block while still being transactionally safe:

(defn get-sid []
  (dosync
   (let [{:keys [cnt sids]} @sid-pool]
     (if (empty? sids)
       (do 
         (alter sid-pool
                (fn [old]
                  (-> pool
                      (assoc :sids (range (inc cnt) (+ sid-batch cnt)))
                      (update-in [:cnt] + sid-batch))))
         cnt)
       (do
         (alter sid-pool update-in [:sids] next)
         (first sids))))))

Answer 2

(def sid-batch 10)
(def sid-pool (atom {:cnt 0
                     :sids '()}))

(defn get-sid []
  (first (:sids (swap! sid-pool
                  (fn [{:keys [cnt sids]}]
                    (if-let [sids (next sids)]
                      {:cnt cnt :sids sids}
                      {:sids (range cnt (+ sid-batch cnt))
                       :cnt (+ cnt sid-batch)}))))))

Like I said in my comment I think you have the right idea with "abusing a field in sid-pool". Except you don't need a field, just call (comp first sids) on the return value from swap!

I removed the inc in the call to range because it caused the generator to skip multiples of 10.

And to return a sid to the pool:

(defn return-sid [sid]
  (swap! sid-pool (fn [{:keys [cnt [_ & ids]]}]
                    {:cnt cnt
                     :sids (list* _ sid ids)})))

Answer 3

Maybe I'm confused about what you are trying to do but the canonical way to create unique IDs in Clojure would simply be:

(let [counter (atom 0)]
  (defn get-unique-id []
    (swap! counter inc)))

There's no need for any complex locking. Note that:

• The closure encapsulates the let-bound atom so you can be sure that no one else can touch it.
• The swap! operation ensures atomic safety in concurrent situations, so the get-unique-id function can be shared across different threads.