Can I get rid of these eval?

Question

I'd like to build a list of operations to execute in a Redis pipeline. I was using accession since it's much simpler than carmine, but now I need connection pooling (missing from accession) and thus I'm looking at carmine again (it seems the go-to and most complete clojure library for redis)

I managed to get this working:

; require [taoensso.carmine :as redis]
(defn execute1 [request] (redis/wcar {} (eval request)))
(defmacro execute [body] `(redis/wcar {} ~@(eval body)))

(def get1 `(redis/get 1))
(execute1 get1)
(execute [get1])

but given that I'll build vectors of thousands of elements, I'm a bit worried about the possible performance hit of eval (besides the fact that I've been taught to always avoid eval if possible). I reckon that defmacro instead is/can be evaluated at macro-expansion time, which might be earlier (during AOT compilation?) and without using eval.

Is there anything that I can do? should I move to a different library? (I had a look at carmine's source code: my pain is only due to a small convenience for the author: a *context* used to avoid passing around a couple extra argument: getting rid of it should be simple enough, but I'm not invested enough in it... I might as well decide to move to another data store in the future)

edit: I've been asked to write an example of what I think is the boilerplate that I'd prefer to avoid writing in my actual code, so: (the following is untested, it's just a POC)

(defn hset [id key val]
  #(redis/hset id key val))

(defn hsetnx [id key val]
  #(redis/hsetnx id key val))

(defn hincrby [id key increment]
  #(redis/hincrby id key increment))

(defn hgetall [id key]
  #(redis/hgetall id key))

(defn sadd [id el]
  #(redis/sadd id el))

(defn scard [id]
  #(redis/scard id))

(defn smembers [id]
  #(redis/smembers id))

(defmacro execute [forms]
  `(redis/wcar {} ~@(map apply forms)))

; end boilerplate

(defn munge-element [[a b c]]
  (conj
    (mapcat #(hincrby a :whatever %) b)
    (sadd c b)
    (hsetnx a c))

(defn flush-queue! [queue_]
  (execute queue_)
  [])

(defn receive [item]
  (if (< (count @queue) 2000)
    (swap! queue conj (munge-element item))
    (swap! queue flush-queue!)))

Obviously, I could write something like this, but if this was truly the inteneded way to use carmine, these curried functions would be provided along (or instead of) the normal ones. Also a lot of lines could be shaved off by constructing up the defs with syntax quoting, but this is accidental complexity, not inherent the original problem.

Answer 1

The following piece of code does not differ very much from yours:

(defmacro execute [& forms]
   `(redis/wcar {} ~@forms))

(defn get1 []
  (redis/get 1))

(execute (get1) (get1) ...)

This is basically how carmine should be used (following this suggestion in the README). If it doesn't suit your needs, could you clarify why?

---

After the question was edited to present more clearly what should be accomplished I think I have a solution for you. You're trying to create a list of statements to be executed at some specific time in the future. To achieve this you're wrapping each statement inside a function and I agree, that's a lot of boilerplate.

But unnecessary. You can get the same result by having a macro that automatically creates thunks for you:

(defmacro statements [& forms]
  `(vector
     ~@(for [f forms]
         `(fn [] ~f))))

Now, whatever you pass to this macro will result in a vector of zero-parameter functions that can be evaluated e.g. using:

(defn execute-statements [fns]
   (redis/wcar {} (mapv #(%) fns))

And your example turns into something along the lines of:

(defn munge-element [[a b c]]
  (statements
    (mapcat #(redis/hincrby a :whatever %) b)
    (redis/sadd c b)
    (redis/hsetnx a c))

(defn flush-queue! [queue_]
  (mapv execute-statements queue_)
  [])

Edit: This executes every batch in its own pipeline. If you want to do it in a single one, use concat instead of conj to build up your queue (in receive) and (execute-statements queue_) instead of (mapv execute-statements queue_).

Note: IIRC correctly, this:

(redis/wcar {} a [b c]])

returns the same result as this:

(redis/wcar {} a b c)

I.e., carmine collects the results somewhere and always returns a vector for all of them. Even if not, I think you can still avoid your dreaded boilerplate by just tweaking the stuff presented here a little bit.