No, you can't. The reason is you would get duplicate symbols. A universal library is also no option, since the OS selects different binaries depending on the current architecture.
You can however build two libraries distinguished by name, e.g.:
libFoo.a and libFoo_debug.a
and ship those two to your clients. Then, the client application will simply link against the respective library (per build settings in "Other Linker Flags") in the Debug and Release Configuration. This setup needs to be made just once.