From MSDN:
"This event provides notification of uncaught exceptions. It allows the application to log information about the exception before the system default handler reports the exception to the user and terminates the application."
So the ugly Popup will be displayed regardless what you are doing in your Handler and the application will be terminated after that.
If you don't want the default error popup to be displayed you should add a handler to Application.Current.DispatcherUnhandledException
log your errors and call Application.Current.Shutdown()
yourself.