(This was a comment but was requested to be provided as an answer)
You are saying this exception happens when you create and use Swing objects over multiple threads, but not when you use a single thread? This sounds like a common Swing concurrency issue that arises because Swing isn't thread safe. You have to do everything Swing related on the EDT.
On the topic of how your panel creation is taking long, maybe you can parallelize the creation of the non-Swing objects, and then at the end when you have a list of them or whatever you have, you can create the panels real quick on the EDT (I'm assuming the construction of the non Swing objects are what's taking long to execute).