To my knowledge, there isn't a tool that will "remove all unused extension tables" automagically, as that could potentially be quite dangerous (removing still used, or tables where the extension is just disabled, etc).
If you want to proceed, BACK IT UP, BACK IT UP, BACK IT UP!!. Then, and only then, manually go through the database and look at each table (phpMyAdmin makes this a lot easier). If it starts with an extension's namespace that is not being used (usually table names are like: namespace_module_tablename
), look at the table to see if it has current/relevant data in it. There is a handy reference for the Magento DB schema (http://www.magereverse.com/). However, I don't know how familiar you are with Magento DB, and if one is not careful/knowledgeable, you can cause tremendous damage.
If you aren't super familiar with Magento, I would steer you away from doing this operation. Depending on if it is a live store or just your development machine, you could always start over. Then you know it is a good, clean start.