64 bit Centos Java JVM unable to create a native thread

Question

I am using the Java Executor Service to create a singlethread. Code:-

ExecutorService executor = Executors.newSingleThreadExecutor();
try {
    executor.submit(new Runnable() {

        @Override
        public void run() {
            Iterator<FileObject> itr = mysortedList.iterator();
            while (itr.hasNext()) {
                myWebFunction(itr.next();
            }
        };
    }).get(Timeout * mysortedList.size() - 10, TimeUnit.SECONDS);
} catch (Exception ex) {

} finally {
    executor.shutdownNow();
}

Details: myWebfunction processes files of different size and content.Processing involves extracting the entire content and applying further actions on the file content. The program runs in 64bit Centos.

Problem: When the myWebfunction gets file of size greater than some threshold, say 10MB, the executor service is unable to create a native thread. I tried various -Xmx and -Xms settings, but still the executor service throws the same error.

Answer 1

My guess is you calling this many times, and you are not waiting for the thread which has timed out, leaving lots of threads lying around. When you run out of stack space, or you reach about 32K threads, you cannot create any more.

I suggest using a different approach which doesn't use so many threads or kills them off when you know you don't need them any more. E.g. have the while loop check for interrupts and call Future.cancel(true) to interrupt it.