How to test for backwards compatibility in NuGet?

Question

Let us say I am working on a project with maybe 100 nuget packages installed and 50 of these packages require an update.

I would ike to get a clear overview on which package updates cause any problems. I was thinking of something like this but any other ideas are welcome:

1. Foreach nuget package installed
2. Check if it needs to be updated
3. Update and remember version
4. Build solution, if succes goto 5, if failure goto 6
5. Run unit tests if succes goto 7, if failure goto 6
6. Restore package to old version and output which step failed and corresponding error message.
7. Update was succesful :)

So is there a nuget api or some way I could do this?

I came across powershell already with which you can update a single nuget package programmatically but I cannot think of a way to find all nuget packages installed and iterate through them while checking for errors etc.

Answer 1

Package authors should be respecting compatibility with the proper use of the package version. What you're intending to do sounds like it would be a lot of work if at all possible. Your project using the NuGet packages shouldn't "assume" that the latest and greatest version of the packages is going to make your project that much better. Just seems lazy IMO (all due respect). Best option I can think of is to set up your CI so that it updates each package "safely" (updates to latest version of the package using the same major/minor versions) - nuget update myproject.csproj -safe. If something breaks there, I'd be wary of whether the package author's practices...

Answer 2

I would start small and simple.

1. Create a branch with CI
2. Update all packages
3. Run CI with all tests and stuff
4. If everything passes, merge branch into master
5. If something fails, update the packages manually

From my experience with NuGet, backwards compatibility is really good. Chance of package breaking something is small. So updating all packages and running all your CI is good start and should mostly succeed. You also want to run this as frequently as possible. Maybe once a day. This way, the amount of packages to be updated should be relatively small.

Only after you find out this causes way too many update failure and manual interventions should you start considering doing smaller batches of packages. But I believe that will not be necessary.