Java: Is 'tenured' memory the same as permgen?

Question

I am trying to gather information on an occasional issue we are having, where after a few weeks of operation the app slows down, works fine, slows down, works fine, with the intervals between being slow and working fine getting shorter and shorter. My theory is that as time goes on we are garbage collecting more often. The other key information is that we have occasionally experienced OOM PermGen issues.

I put enabled verbose:gc, and now see GC output in catalina.out. I think I need to add the PrintGCDetails flag, however, based on the information here:

http://www.oracle.com/technetwork/java/gc-tuning-5-138395.html

Enabled the latter flag will print out information on collection of "Tenured" memory. The question is, is that the memory that causes PermGen errors, or is it something different? And if it is different, how can I log information that will show the PermGen space?

EDIT -- I cannot attach any of the jvm monitoring tools in this environment, unfortunately.

EDIT -- I added the said config options, as well as one for printing tenuring distribution, I get stuff like

27.701: [GC 27.701: [ParNew
Desired survivor size 2162688 bytes, new threshold 4 (max 4)
- age   1:    1906560 bytes,    1906560 total
- age   2:       2064 bytes,    1908624 total
- age   3:       5064 bytes,    1913688 total
- age   4:     650368 bytes,    2564056 total
: 35684K->2678K(38336K), 0.0068580 secs] 224179K->191173K(1065664K), 0.0069700 secs] [Times: user=0.01 sys=0.00, real=0.01 secs] 

Is the ParNew generation the permgen space?

and

 (concurrent mode failure): 25387K->31940K(1027328K), 0.2983200 secs] 50714K->31940K(1065664K), [CMS Perm : 35273K->35139K(35392K)], 0.2985210 secs] [Times: user=0.30 sys=0.00, real=0.30 secs] 
 (concurrent mode failure): 25356K->31941K(1027328K), 0.3032690 secs] 50861K->31941K(1065664K), [CMS Perm : 35264K->35129K(35392K)], 0.3034800 secs] [Times: user=0.30 sys=0.00, real=0.31 secs]

the failures are bothering me.

Thanx in advance

Answer 1

Tenured and PermGen are not the same, no. They are related, but not the same thing. The exact details depend on the implementation in the JVM you are using, but, from the document you linked:

"A third generation closely related to the tenured generation is the permanent generation. The permanent generation is special because it holds data needed by the virtual machine to describe objects that do not have an equivalence at the Java language level. For example objects describing classes and methods are stored in the permanent generation."

Interned Strings and class details and such are typically stored in PERM, whereas long lived Java objects are TENURED.

Here is a decent article explaining PermGen (and how to tweak it): http://blogs.oracle.com/jonthecollector/entry/presenting_the_permanent_generation

Answer 2

Are you interning any Strings or loading any classes?

The two things I've had eat up permgen space are when you intern Strings that shouldn't be (thus making all sorts of ungarbagecollectable Strings hange around), and old class definitions caused by loading in new classes (usually from Tomcat redeploying, after a dozen or so this could happen for us).

I seem to remember that Tomcat 6 was better on the redeploy issue, and 7 was supposed to fix it, but that's off the top of my head.

Do you use any external libraries or native code libraries? I wouldn't be surprised if resources they leak count against permgen (just a guess).