Capture frequency of javacores during workloads

Question

I often run workloads against my own web applications to try find performance issues.
Sometimes I see memory leaks etc at variable duration.
So I created a bash script to take javacores, kill -3 pid , every minute for 10 minutes and script is executed on the hour.
For a workload that runs for 120 hours, this will produce 1200 javacores.

I'm wondering,

• Is this overkill? I'd like a continuous view of system (javacore every 5 minutes, for 120 hours), but don't want to impact perf
• what is a reasonable frequency to automatically capture javacores against servlet based app?

Answer 1

Looks like we are looking at two issues:

1. Performance
2. OutOfMemoryError

Performance: for performance, determine the longest request you can tolerate and generate the javacores when its 3 to 5 times that amount. (Anything below 5 minutes to me is fine tuning and can difficult)

Let's say your longest request you want is 3 minutes, I would generate 3 javacores at evenly from 9 minutes to 15 minutes. I usually suggest the link (collect manually) below but if you already wrote your own script use it
"MustGather: Performance, Hang or High CPU Issues on Linux"
http://www.ibm.com/support/docview.wss?rs=180&uid=swg21115785

OutOfMemoryError: see if your product is leaking, follow the steps in URL below and go to collect manually and google IBM heap analyzer (stand alone and free) and review the heap dump for potential leak suspects.
"MustGather: Native Memory Issues on Linux"
http://www.ibm.com/support/docview.wss?rs=180&uid=swg21138462
Personnaly, I prefer looking at heap dumps memory use to equal the XMX or nearly that.

Answer 2

Since this is an IBM JVM you could try using Health Center instead of taking javacores regularly:

http://www.ibm.com/developerworks/java/jdk/tools/healthcenter/

This has profiling and memory monitoring views so should give you the data you are looking and save you analysing the javacore files yourself.