The native libraries were the problem. I used NativeSwing, to embed a WebBrowser in Swing. When doing NativeInterface.open() or NativeInterface.initialize() the proxy settings are changed and cannot be changed afterwards. I therefore now read the proxy settings and save them to a static field. Then when I need them I read them from this static field.
The story continues: The above approach only worked from within Eclipse. If I double clicked on the jar I got "The native side did not receive an incoming connection!". The reason for that is not because of the NativeInterface I'm using, but has something to do with how java handles system proxy settings.
I found this article: http://ideen2011.blogspot.de/2011/08/java-proxyselector-usesystemproxies-and.html
In short: Use ProxySelector.setDefault(null); if you want socket connections to be not affected by system proxies etc. At least that helped me, but I'm still trying to understand, what exactly is going on in the background.
Story continues in 2015: The previouly mentioned approach caused this issue: JavaFX webview set Proxy
So I recommend instead of
ProxySelector.setDefault(null)
to use
ProxySelector.setDefault(ProxySelector.getDefault())