You pay a small performance penalty for this type of lenses. It comes from all the higher rank types with constraints that cause dictionary passing to happen.
This is one of the rare cases when you want to go back to data-lens, which doesn't have that problem and can even make your code faster. Data-lens, if you decode the Store
comonad indirection, uses about the most straightforward representation you can have for a lens:
newtype Lens s a = Lens (s -> (a, a -> s))
While the library doesn't support polymorphic lenses by itself you can build your own lens type that does and still gives you the high performance:
newtype Lens s t a b = Lens (s -> (a, b -> t))
For your particular purpose you may also be interested in the linear package.