Random JVM crash with stack only showing Hibernate data

Question

For our web application we are experiencing some JVM crashes on a regular base. Until now we don't have a clue what could be the exact cause of the problem though.

All crashes seem to have the same 'problematic frame' through, which makes us suspect that they are quite similar and related to each other. The problematic frame in question is inside Hibernate code, there seems to be no trace of JNI code though.

If we run the same code with the -Xint flag active (disables the JIT), we don't appear to have the problem anymore.

Our Hibernate version is 3.6.7.

Does anyone have suggestions where we could look further on what causes this problem?

Thanks!

The JVM report:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00000000022b5be6, pid=2960, tid=1736
#
# JRE version: 6.0_21-b07
# Java VM: Java HotSpot(TM) 64-Bit Server VM (17.0-b17 mixed mode windows-amd64 )
# Problematic frame:
# J  org.hibernate.type.descriptor.java.MutableMutabilityPlan.deepCopy(Ljava/lang/Object;)Ljava/lang/Object;
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#

---------------  T H R E A D  ---------------

Current thread (0x0000000008d35000):  JavaThread "http-8002-6" daemon [_thread_in_Java, id=1736, stack(0x000000000a980000,0x000000000a9c0000)]

siginfo: ExceptionCode=0xc0000005, reading address 0xffffffffffffffff

Registers:
EAX=0x00000001038a6ca8, EBX=0x00000001038de1e0, ECX=0x0000000000000018, EDX=0x3e64492f3c202020
ESP=0x000000000a9bb0f0, EBP=0x000000008e4e0678, ESI=0x00000000b2f9dc30, EDI=0x0000000001d0dd10
EIP=0x00000000022b5be6, EFLAGS=0x0000000000010246

Top of Stack: (sp=0x000000000a9bb0f0)
0x000000000a9bb0f0:   0000000000f55e63 00000000b0dd1830
0x000000000a9bb100:   00000000b353bbf8 000000000a9bb108
0x000000000a9bb110:   00000001000167f5 000000008e4e1220
0x000000000a9bb120:   00000001040b8278 0000000000000000
0x000000000a9bb130:   0000000001d0dd10 00000000b0540968
0x000000000a9bb140:   000000000a9bb180 00000000b2f9dd00
0x000000000a9bb150:   000000008e4e0678 0000000001c6db68
0x000000000a9bb160:   00000000b053ec30 00000000b0dd1830
0x000000000a9bb170:   00000000b2f9dd00 0000000001d882f0
0x000000000a9bb180:   000000110000000f 00000000b2f9dc30
0x000000000a9bb190:   000000008e481a20 000000008e2b5180
0x000000000a9bb1a0:   000000008e481a20 000000008e2b5180
0x000000000a9bb1b0:   000000008e4e1220 0000000000000078
0x000000000a9bb1c0:   000000000a9bb1f0 0000000107623440
0x000000000a9bb1d0:   0000000001d97634 000000008e4e1160
0x000000000a9bb1e0:   0000000107623440 0000000001d97678 

Instructions: (pc=0x00000000022b5be6)
0x00000000022b5bd6:   62 8e 03 01 00 00 00 4d 3b d3 0f 85 b7 01 00 00
0x00000000022b5be6:   4c 8b 52 38 49 bb 38 01 30 00 01 00 00 00 4d 3b 


Stack: [0x000000000a980000,0x000000000a9c0000],  sp=0x000000000a9bb0f0,  free space=ec0000000000000000k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
J  org.hibernate.type.descriptor.java.MutableMutabilityPlan.deepCopy(Ljava/lang/Object;)Ljava/lang/Object;

Answer 1

Regardless of the cause, this shouldn't occur on a JVM. I would suggest the simplest solution is to upgrade your JVM to the latest minor point release.

Note that Java 6 has now reached it's end of life, and you should likely be running on a Java 7 VM. That's a bigger upgrade, however, and you should perhaps schedule time for testing that. I would expect a Java 6 point upgrade to be trivial however.

Answer 2

It tries to do a "mov r10,QWORD PTR [rdx+0x38]"

Without a memory map (which you did not provide) it is hard to tell what's at 0x3e64492f3c202020 + 0x38 address. moreover, 3e64492f3c202020 looks more like text ">dI/< " and not like an address.

I suggest you upgrade the JVM and test your RAM as well.