The solution is quite simple:
- you add into your package the DLLs for all supported versions of MVC (3 and 4 in your case);
- then write install.ps1 script to find out what version of MVC is used in the target project and remove unnecessary DLLs from project references.
Here you can find more detailed instructions and an example of install.ps1 script: http://blogs.korzh.com/2013/12/nuget-package-different-mvc-versions.html