I thought of using CachedThreadPool but my worry is if too many threads are created would it have any performance impact on the application?
You need to test for the application for performance impact.
If none of them fits into the application or having some issues then you can use customized thread pool executor java.util.concurrent.ThreadPoolExecutor
You can customize according your needs with configuiring core pool size, configuring the blocking queue. Blocking queue will be used and task will be queued when pool size is reached.