If you're using the register, you should let the installer to create the default key/values pairs, and let the uninstaller delete them.
Another way is to use the app config file, which is more simple then dealing with registry, See this posts: Simplest way to have a configuration file in a Windows Forms C# Application