Avoiding PermGen out of memory and GC overhead limit exceeded

Question

I'm trying to generate classes and load them at run time.
I'm using a ClassLoader object to load the classes. Since I don't want to run out of PermGen memory, from time to time I un-reference the class loader and create a new one to load the new classes to be used. This seems to work fine and I don't get a PermGen out of memory. The problem is that when I do that, after a while I get the following error:

java.lang.OutOfMemoryError: GC overhead limit exceeded 

So my question is, when should I un-reference the class loader to avoid both errors?:
Should I monitor in my code the PermGen usage so that I un-reference the class loader and call System.gc() when the PermGen usage is close to the limit?
Or should I follow a different approach?
Thanks

Answer 1

There is no single correct answer to this.

On the one hand, if unlinking the classloader is solving your permgen leakage problems, then you should continue to do that.

On the other hand, a "GC overhead limit exceeded" error means that your application is spending too much time garbage collection. In most circumstances, this means that your heap is too full. But that can mean one of two things:

• The heap is too small for your application's requirements.

• Your application has a memory leak.

You could assume that the problem is the former one and just increase the heap size. But if the real problem is the latter one, then increasing the heap size is just postponing the inevitable ... and the correct thing to do would be to find and fix the memory leak.

---

Don't call System.gc(). It won't help.

Answer 2

Are you loading the same class multiple times? Because you should cache the loaded class.

If not, how many classes are you loading? If they are plenty you may have to fix a limit of loaded classes (this number can be either based on heap size or a number based on how much memory does it take to have a loaded class) and discard the least used when loading the next one.

Answer 3

I had somewhat similar situation with class unloading.

I'm using several class loaders to simulate multiple JVM inside of JUnit test (this is usually used to work with Oracle Coherence cluster, but I was also successfully used this technique to start multi node HBase/Hadoop cluster inside of JVM).

For various reasons tests may require restart of such "virtual" JVM, which means forfeiting old ClassLoader and creating new one.

Sometimes JVM delays class unloading event if you for Full GC, which leads to various problems later.

One technique I found usefully for forcing JVM to collect PermSpace is following.

public static void forcePermSpaceGC(double factor) {
    if (PERM_SPACE_MBEAN == null) {
        // probably not a HotSpot JVM
        return;
    }
    else {
        double f = ((double)getPermSpaceUsage()) / getPermSpaceLimit();
        if (f > factor) {

            List<String> bloat = new ArrayList<String>();
            int spree = 0;
            int n = 0;
            while(spree < 5) {
                try {
                    byte[] b = new byte[1 << 20];
                    Arrays.fill(b, (byte)('A' + ++n));
                    bloat.add(new String(b).intern());
                    spree = 0;
                }
                catch(OutOfMemoryError e) {
                    ++spree;
                    System.gc();
                }
            }
            return;
        }
    }
}   

Full sourcecode

I'm filling PermSpace with String using intern() until JVM would collect them.

But

• I'm using that technique for testing
• Various combination of hardware / JVM version may require different threshold, so it is often quicker to restart whole JVM instead of forcing it to properly collect all garbage