It actually isn't as bad as you think. Refresh tokens are application specific, that meaning specific to your client id. If the user installs your application a number of times then they have a number of Refresh tokens related to your application.
I ran into this issue with a SSIS Connection manager if the user had my connection manager running on more then 20 SSIS packages the first one the installed would stop working.
https://developers.google.com/accounts/docs/OAuth2#expiration
Token expiration
You should write your code to anticipate the possibility that a granted token might
no longer work.
A token might stop working for one of these reasons:
- The user has revoked access.
- The token has not been used for six months.
- The user account has exceeded a certain number of token requests.
There is currently a 25-token limit per Google user account. If a user account has 25 valid - tokens, the next authentication request succeeds, but quietly invalidates the oldest outstanding token without any user-visible warning.
If you need to authorize multiple programs, machines, or devices, one workaround is to limit the number of clients that you authorize per user account to 15 or 20. If you are a Google Apps admin, you can create additional admin users and use them to authorize some of the clients.
So as long as your application isn't being installed more then 15 times by the same user you shouldn't have a problem. If it is a problem you can suggest that they use a different / dedicated login for your application.