This might be abit of hack, but you could something like this:
nuget pack mypackage.nuspec -Symbols -Exclude "*/*.exe;*/*.dll"
Where "*/*.exe;*/*.dll"
are the files you want to ignore and then run
nuget pack mypackage.nuspec
Where your mypackage.nuspec includes everything that you want in your regular nuget package.
This shouldn't cause any problems, since the dll, pdb and nuget packages should still have all matching versions.