Troubleshooting Oracle+JDBC hang

Question

Every so often we see Oracle (11gR2) hang while trying to do an update via JDBC. The thread accessing Oracle is the only thread in the app connected to the database and it just sits forever waiting for Oracle to return data. It's possible another process could have accessed the same database and updated a table, but I was under the impression Oracle would automatically detect a deadlock and return an error. In our case the database just hangs. Any ideas on what could cause this or ways to debug via some DBA command(s)? Here's the stack trace I see when it's stuck:

Thread 7315: (state = IN_NATIVE)
 - java.net.SocketInputStream.socketRead0(java.io.FileDescriptor, byte[], int, int, int) @bci=0 (Compiled frame; information may be imprecise)
 - java.net.SocketInputStream.read(byte[], int, int, int) @bci=79 (Compiled frame)
 - java.net.SocketInputStream.read(byte[], int, int) @bci=11 (Compiled frame)
 - oracle.net.ns.Packet.receive() @bci=180, line=308 (Compiled frame)
 - oracle.net.ns.DataPacket.receive() @bci=1, line=106 (Compiled frame)
 - oracle.net.ns.NetInputStream.getNextPacket() @bci=48, line=324 (Compiled frame)
 - oracle.net.ns.NetInputStream.read(byte[], int, int) @bci=33, line=268 (Compiled frame)
 - oracle.net.ns.NetInputStream.read(byte[]) @bci=5, line=190 (Compiled frame)
 - oracle.net.ns.NetInputStream.read() @bci=73, line=107 (Compiled frame)
 - oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket() @bci=94, line=143 (Compiled frame)
 - oracle.jdbc.driver.T4CSocketInputStreamWrapper.read() @bci=18, line=80 (Compiled frame)
 - oracle.jdbc.driver.T4CMAREngine.unmarshalUB1() @bci=6, line=1137 (Compiled frame)
 - oracle.jdbc.driver.T4CTTIfun.receive() @bci=11, line=350 (Compiled frame)
 - oracle.jdbc.driver.T4CTTIfun.doRPC() @bci=63, line=227 (Compiled frame)
 - oracle.jdbc.driver.T4C8Oall.doOALL(boolean, boolean, boolean, boolean, boolean, oracle.jdbc.internal.OracleStatement$SqlKind, int, byte[], int, oracle.jdbc.driver.Accessor[], int, oracle.jdbc.driver.Accessor[], int, byte[], char[], short[], int, oracle.jdbc.driver.DBConversion, byte[], java.io.InputStream[][], byte[][][], oracle.jdbc.oracore.OracleTypeADT[][], oracle.jdbc.driver.OracleStatement, byte[], char[], short[], oracle.jdbc.driver.T4CTTIoac[], int[], int[], int[], oracle.jdbc.driver.NTFDCNRegistration) @bci=769, line=531 (Compiled frame)
 - oracle.jdbc.driver.T4CPreparedStatement.doOall8(boolean, boolean, boolean, boolean, boolean) @bci=749, line=208 (Compiled frame)
 - oracle.jdbc.driver.T4CPreparedStatement.executeForRows(boolean) @bci=226, line=1046 (Compiled frame)
 - oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout() @bci=301, line=1336 (Compiled frame)
 - oracle.jdbc.driver.OraclePreparedStatement.executeInternal() @bci=119, line=3613 (Compiled frame)
 - oracle.jdbc.driver.OraclePreparedStatement.executeUpdate() @bci=13, line=3694 (Compiled frame)
 - oracle.jdbc.driver.OraclePreparedStatementWrapper.executeUpdate() @bci=4, line=1354 (Compiled frame)
 - sun.reflect.GeneratedMethodAccessor16.invoke(java.lang.Object, java.lang.Object[]) @bci=40 (Compiled frame)
 - sun.reflect.DelegatingMethodAccessorImpl.invoke(java.lang.Object, java.lang.Object[]) @bci=6 (Compiled frame)
 - java.lang.reflect.Method.invoke(java.lang.Object, java.lang.Object[]) @bci=57 (Compiled frame)
 - oracle.ucp.jdbc.proxy.StatementProxyFactory.invoke(java.lang.Object, java.lang.reflect.Method, java.lang.Object[]) @bci=513, line=294 (Compiled frame)
 - oracle.ucp.jdbc.proxy.PreparedStatementProxyFactory.invoke(java.lang.Object, java.lang.reflect.Method, java.lang.Object[]) @bci=122, line=137 (Compiled frame)
 - com.sun.proxy.$Proxy3.executeUpdate() @bci=9 (Compiled frame)

Answer 1

A deadlock would be when session 1 holds a lock that session 2 is waiting on while session 2 simultaneously holds a lock that session 1 is waiting on. It sounds like what you're describing is a simple blocking lock where session 1 (your application) is blocked waiting on a lock held by session 2 (some other application) that simply hasn't ended its transaction yet. In that case, session 1 will block indefinitely waiting on session 2.

If you want to take a look from the back end, gv$session will have a row for every session that is open in the database. Since your application only has a single thread accessing the database, I would guess that some combination of osuser, process, machine, terminal, and program would allow you to identify the session associated with your application. Once you do that, look at the blocking_instance and blocking_session columns. If those are populated, your session is blocked waiting on some other session to either commit or roll back. You can use the v$session row for that session to try to figure out what application is incorrectly holding a lock for too long.

The "classic" problem here tends to be applications which lock rows across user interactions. If I write an application that does pessimistic locking, for example, I'd lock a row when I present it to a user and wait for the user to modify the row before releasing the lock. The problem, though, is that if the user goes off to lunch without doing anything else (or if the application crashes), the row remains locked in the database indefinitely until a DBA comes along and kills the session that's holding the lock.

Answer 2

Have a look at v$active_session_history which shows the last wait events the session was waiting for.