In my opinion, it is good to do so in most cases.
In your situation, depending on how much time you have, you may want to do one of the following:
1. Contribute to the gem
If the gem is open source, you may fork it, update to the newest version, and do a pull request.
By this way you also give contribution to the rails whole and the others who are facing the same problem as well.
Downside is this takes time. You have to wait for the author to accept the pull request and wait for the next version of the gem. But you can point your Gemfile to use your forked version until the new version is out ;)
2. Write your own gem
Writing a gem for rails providing assets is actually not difficult. You may follow other existing gem's structure and should be easy to understand.
A good example is https://github.com/rails/jquery-rails
Downside is you have to maintain the gem. Otherwise when fabric 1.5 is out, another one would ask the same question as yours again.
3. Put the assets in your vendor directory
Rails project by default do have a vendor directory. It's ok to put external assets here as well.
The above are my preferred way to manage external assets.