How to multi-thread a Java EE database audit logger?

Question

I'm working on a Java EE 5 application running on Websphere 7.0, and am trying to find a thread-safe and performant way to multi-thread the persistence of database audit log records. Are there any known methods for performing multi-threaded audit logging safely and efficiently within a Java EE application?

If you need some background information: the app is a web service, and each request message that it receives results in the creation of 100 or 200 audit log messages which must be persisted to a database. Originally the audit logging was done with a custom audit handler class that extends java.util.logging.Handler, and the publish method would open a database connection, populate a prepared statement from the LogRecord, and execute the insert. Since this custom Handler was running within the EJB's thread, audit logging could add up to several seconds to the response time for each request message and was causing SLAs to be missed.

So the audit handler was replaced with a wrapper handler that adds creates a separate thread (yes, with new Thread(), against the rules of Java EE). The wrapper handler uses a Vector to queue the audit records, and it persists them as fast as possible in the separate thread using the audit handler.

While it breaks the rules of Java EE threading, this wrapper has worked quite well... until we allowed concurrent invocations on the MDB. The wrapper has the potential to screw up when multiple EJB invocations are allowed, and potentially will save each log record to the database multiple times. This seems to indicate that the wrapper or thread creation logic has a bug.

I was going to work on identifying and fixing that problem, but thought I'd first ask SO if there's a better way.

Answer 1

Using JMS, put those audit messages on a queue, and then invoke some other service that will pick them up and store them in database. This, of course, means that all logs won't necessarily be stored in database in real time, but this approach will offload some work from Websphere and you won't have standard breaking multi threading in your code.