Spring transaction that uses JPA throws Exception during AfterCompletion phase

Question

I have a web application that uses the Spring Framework (3.1) and persistence through JPA (2.0) backened by Hibernate (4.1.1)

Hibernate Search is also enabled (4.1 RC). Hibernate's Second Level cache is Infinispan (5.1.3).

I also use Infinispan as Lucene (3.5) Directory Provider (for Hibernate Search).

Everything runs over XA transactions, with Bitronix (2.1.2) as JTA manager.

I have methods annotated with @Transactional. When the method ends, and the transaction commits, the following exception occurs:

21:08:54,784  WARN BitronixTransaction:499 - Synchronization.afterCompletion() call failed for org.hibernate.engine.transaction.synchronization.internal.RegisteredSynchronization@704ecb9a
java.util.ConcurrentModificationException
    at java.util.LinkedHashMap$LinkedHashIterator.nextEntry(LinkedHashMap.java:390)
    at java.util.LinkedHashMap$KeyIterator.next(LinkedHashMap.java:401)
    at org.hibernate.engine.transaction.internal.SynchronizationRegistryImpl.notifySynchronizationsAfterTransactionCompletion(SynchronizationRegistryImpl.java:78)
    at org.hibernate.engine.transaction.internal.TransactionCoordinatorImpl.sendAfterTransactionCompletionNotifications(TransactionCoordinatorImpl.java:335)
    at org.hibernate.engine.transaction.internal.TransactionCoordinatorImpl.afterTransaction(TransactionCoordinatorImpl.java:147)
    at org.hibernate.engine.transaction.synchronization.internal.SynchronizationCallbackCoordinatorImpl.afterCompletion(SynchronizationCallbackCoordinatorImpl.java:126)
    at org.hibernate.engine.transaction.synchronization.internal.RegisteredSynchronization.afterCompletion(RegisteredSynchronization.java:61)
    at bitronix.tm.BitronixTransaction.fireAfterCompletionEvent(BitronixTransaction.java:497)
    at bitronix.tm.BitronixTransaction.commit(BitronixTransaction.java:244)
    at bitronix.tm.BitronixTransactionManager.commit(BitronixTransactionManager.java:120)
    at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1010)
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:754)
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:393)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:120)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
    at $Proxy79.newTweet(Unknown Source)
    at com.lucho.controller.TweetController.newTweet(TweetController.java:33)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:789)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:641)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:311)
    at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:116)
    at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
    at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
    at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:101)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
    at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
    at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:146)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
    at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
    at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
    at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:182)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
    at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
    at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
    at org.springframework.security.web.session.ConcurrentSessionFilter.doFilter(ConcurrentSessionFilter.java:125)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:173)
    at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
    at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:987)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:579)
    at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1805)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)

You'll notice the Hibernate class "SynchronizationRegistryImpl". This one holds a set of javax.transaction.Synchronization objects. What I think is happening is that Hibernate itself is registered as a Synchronization in this set. So,

1. After commit, two events fire: beforeTransactionCompletion and afterTransactionCompletion.
2. beforeTransactionCompletion goes well.
3. afterTransactionCompletation starts and the method notifySynchronizationsAfterTransactionCompletion is called.
4. This method calls afterCompletion on each of the Synchronization elements of the Set.
5. One of the item in the set (not the last) is an Hibernate item, which calls some clean up.
6. This clean up empties the Set.
7. ConcurrentModificationException is thrown because the set was cleaned while it was being iterated.

Maybe I have configured Hibernate incorrectly. My Spring XML are too big to be put here. Luckily some of you guys have an idea of why this is happening.

More data (which may or may not be useful) to establish context:

• Other of the items in the Synchronization set belongs to Hibernate Search.
• The database datasource is PostgreSQL.
• I'm using XA because there is also an ActiveMQ JMS in the transaction, and at first Infinispan was set as a XA resource too (now it hooks to the JTA Synchronization).
• Spring Security is enabled.
• Hibernate connects to JTA by CMTTransactionFactory.
• I'm not sure if the PersistenceContext is "transactional" or "extended", but from debugging it looks like the latter.

Answer 1

It turned out that there was more Infinispan configuration needed, that I didn't know it existed. In the transaction tag (under default configuration), even though I set up the transanction manager lookup, I had to explicity add the attribute transactionMode="TRANSACTIONAL".

I was also using the default configuration for the caches that Hibernate uses, their names are: entity, local-query, and timestamp. There is a file named infinispan-configs.xml located in the hibernate-infinispan.jar that helps to setup the desired configuration.

UPDATE: disregard this answer. The issue appeared again, and it's this bug.