You could put in some code to catch the error and log it to a file.
Something possibly simpler is to compile your application as a Console Application. This can be done via Project Options - Application - Output type. Then you will get a console window when you run your WPF application and any exception that happens at startup will be logged to this window.