The nuget package Microsoft.SqlServer.Types is only for c# projects.
You can modify the class to be compatible with vb.net projects.
Install support into your project using:
Install-Package Microsoft.SqlServer.Types
Locate the following file:
/SqlServerTypes/Loader.cs
Convert the code in Loader.cs to vb (you can do this with an online converter) and save the file as: Loader.vb
Now you will be able to access the SqlServerTypes namespace from your project.