Lexical scope in Emacs: compatibility with older Emacsen

Question

Emacs 24 added optional lexical bindings for local variables. I would like to use this functionality in my module, while maintaining compatibility with XEmacs and the previous Emacs versions.

Before Emacs 24, the easiest way to get closures was by using the lexical-let form defined in cl-macs, which emulated lexical scope with some clever macro trickery. While this was never hugely popular among elisp programmers, it did work, creating real and efficient closures, as long as you remembered to wrap them in lexical-let, as in this pseudocode:

(defun foo-open-tag (tag data)
  "Maybe open TAG and return a function that closes it." 
  ... do the work here, initializing state ...
  ;; return a closure that explicitly captures internal state
  (lexical-let ((var1 var1) (var2 var2) ...)
    (lambda ()
      ... use the captured vars without exposing them to the caller ...
      )))

The question is: what is the best way to use the new lexical bindings, while retaining support for Emacs 23 and for XEmacs? Currently I solved it by defining a package-specific macro that expands into lexical-let or into ordinary let depending on whether lexical-binding is bound and true:

(defmacro foo-lexlet (&rest letforms)
  (if (and (boundp 'lexical-binding)
           lexical-binding)
      `(let ,@letforms)
    `(lexical-let ,@letforms)))
(put 'foo-lexlet 'lisp-indent-function 1)

... at the end of file, turn on lexical binding if available:
;; Local Variables:
;; lexical-binding: t
;; End:

This solution works, but it feels clunky because the new special form is non-standard, doesn't highlight properly, can't be stepped into under edebug, and generally draws attention to itself. Is there a better way?

---

EDIT

Two examples of ideas for smarter (not necessarily good) solutions that allow the code to continue using standard forms to create closures:

• Use an advice or a compiler macro to make lexical-let expand to let under lexical-bindings iff the lexical-let only assigns to symbols which are lexically scoped anyway. This advice would only be temporarily activated during byte-compilation of foo.el, so that the meaning of lexical-let remains unchanged for the rest of Emacs.

• Use a macro/code-walker facility to compile let of non-prefixed symbols to lexical-let under older Emacsen. This would again only apply during byte-compilation of foo.el.

Do not be alarmed if these ideas smell of overengineering: I am not proposing to use them as-is. I'm interested in the alternatives to the above macro where the package gets the benefit of nicer portable usage of closures for the price of some additional complexity of loading/compilation.

---

EDIT 2

As no one has stepped up with a solution that would allow the module to keep using let or lexical-let without breaking them for the rest of Emacs, I am accepting Stefan's answer, which states that the above macro is the way to do it. In addition to that, the answer improves on my code by using bound-and-true-p and adding an elegant declaration for edebug and lisp-indent.

If someone has an alternative proposal for this compatibility layer, or an elegant implementation of the above ideas, I encourage them to answer.

Answer 1

Since lexical-let and lexical-binding's let do not do quite the same (more specifically lexical-let always uses lexical binding, whereas let uses either dynamic binding or lexical binding depending on whether the var was defvar'd or not), I think your approach is about as good as it gets. You can easily make Edebug step into it, tho:

(defmacro foo-lexlet (&rest letforms)
  (declare (indent 1) (debug let))
  (if (bound-and-true-p lexical-binding)
      `(let ,@letforms)
    `(lexical-let ,@letforms)))

If you don't want to depend on declare, you can use (put 'foo-lexlet 'edebug-form-spec 'let).

Answer 2

One possible solution is to use defadvice to hook lexical-let expansion. I wrote the following advice, and it seems to work fine. This is also byte-compile aware.

(defadvice lexical-let (around use-let-if-possible (bindings &rest body) activate)
  (if (and (>= emacs-major-version 24)
           (boundp 'lexical-binding)
           lexical-binding)
      (setq ad-return-value `(let ,bindings . ,body))
    ad-do-it))

Answer 3

Lexical-let looks to have the same arglist format as let, so what about something like this:

(if (older-emacs-p)
  (setf (macro-function 'let) (macro-function 'lexical-let))
  (setf (macro-function 'lexical-let) (macro-function 'let)))

This shim should allow newer Emacs to read lexical-let portions of older code, as well as the other way (allow older Emacs to read let portions of newer code).

That's Common Lisp though. Anyone care to translate that to Emacs?

And you may run into trouble if lexical-let/let is implemented as a special form (not a macro).

Also, this may totally break the forward-compatible case if let is defined in the older Emacs. Is it? (I know very little about Emacs; it's not my chosen editor). But the backward-compatible case may be more important regardless.