The Downloads page you mentioned is for the client library itself (Google.Apis and Google.Apis.Auth assemblies for example). Each API (YouTube, Drive, Plus, etc.) is dependent on that library.
To download a specific API you should download it from NuGet, and it will download all the dependencies you need (like Newtonsoft.Json, Microrosft.Http.Client, Google.Apis, etc.).
So, if you want to work with Drive API for example, you should query the following: https://www.nuget.org/packages?q=google.apis.drive&prerelease=true&sortOrder=relevance. Remember that the client library is beta, so you should "include "prerelease" in your search.