Benes (Fat Tree) Network for number of inputs/outputs that is not 2^n

Question

I was reading about Benes Network construction in this book. Their construction is easy for a number of inputs and outputs that is an exponent of two.

However it seems to me that for a number of i/o that falls between $2^{n}$ and $2^{(n+1)}$ one has to construct a network with $2^{(n+1)}$ ports and leave many unattached, which is wasteful. Is there a synthesis method for Benes networks or maybe an alternative topology that has the same properties (re-arrangeable not blocking) for arbitrary number of I/O?