These event handlers aren't designed to let you recover from the exception, they're a last resort to give you an opportunity to do something like writing to an error log before the application is terminated.
You've mentioned that you want to log errors - that should work fine, but displaying an error to the user might not be possible because your Application will have reached a point where it is not even able to do that.
As the comment on your question mentions, it is a bad idea to handle exceptions like this.
While you might have a very specific expectation of when this is going to be called, your App could throw an exception at any point - and for any reason. It's impossible to design it so that it handles everything correctly.
Even if you could write something to deal with any exception safely, your App will still be terminated because of the unhandled exception.
The Microsoft documentation for AppDomain.CurrentDomain.UnhandledException gives some more information on this:
http://msdn.microsoft.com/en-us/library/system.appdomain.unhandledexception.aspx