Version your wrapper independently from the library. Your version number should indicate changes to your wrapper code. The library is a dependency for your wrapper, just like any other library dependencies. Maintain a table listing compatibility between versions of your wrapper and versions of the library.
If you don't like the idea of having your wrapper at version 1.1.0 while the library is at 4.3.2, consider adding a "marketing" version number to your wrapper. Your wrapper could be called "Wrapper 4" to indicate that it is compatible with version 4 of the library, but internally you could still identify your wrapper's exact version as 1.1.0.