You can do either of the following:
Remove the reference to the NuGet extension. This will result in a risk of package load errors or other errors if your users do not have NuGet installed, or if they have an outdated version of NuGet that your extension does not support.
Ship separate VSIX packages of your extension for Visual Studio 2010-2012 and Visual Studio 2013.
The latter is likely the best approach. You can create both extensions from the same codebase by following the approach used in the Cloud Explorer extension (note there will be subtle differences because this extension doesn't support Visual Studio 2010, but yours does).
Create a separate project file for the VS2013 extension in the same folder as the extension that targets 2010-2012. I highly recommend you use a visual diff tool to compare Rackspace.VisualStudio.CloudExplorer.11.csproj and Rackspace.VisualStudio.CloudExplorer.12.csproj to show the subtle differences that allow these to live side-by-side.
Create subfolders Properties\vs2010 and Properties\vs2013 to hold the two separate versions of your source.extension.vsixmanifest file. Use the correct NuGet reference for each of these files.
If your project is open source on GitHub, I could help apply the changes and send a pull request.