Maybe the safest thing to do would be to catch Throwable (or Exception and Error separately) and pass in the reference of what was caught to your runRegisteredFinalizers() giving the user a chance to decide if it is something they should care about.
However, errors that you would be catching would be either specific to your toolkit (not necessarily from the users code), or things that the user did not account for (assuming they would have trapped the case themselves). In these cases, the type of exception doesnt matter.
In either case though, depending on the nature of your toolkit and the potential impact on missing a Error, it might make sense to catch that as well.