Atomikos + Spring + JMS read messages batch transactionally

Question

I am trying to design an application using spring + atomikos that reads a batch of N messages and persists them to a database within a single transaction. It has to be a batch of messages because the data is only consistent when it's in batches, i.e., a single message is not enough data for a consistent transaction. Moreover having one transaction per message would absolutely kill my performance. This is not the typical JMS + DB application so I'm having a hard time finding examples online (I tried with a MessageListener as suggested on the atomikos website but that creates one transaction per message). What's the best way to achieve this using Spring? Thanks

Giovanni

Answer 1

Figured out what I need to do. In my spring configuration file, I need to configure the following:

• an instance of org.springframework.transaction.jta.JtaTransactionManager class (configured for example to use atomikos) with the "transactionManager" and "userTransaction" dependencies injected, e.g., with instances of the classes com.atomikos.icatch.jta.UserTransactionManager and com.atomikos.icatch.jta.UserTransactionImp;
• the <tx:annotation-driven transaction-manager="txManager"> tag to use annotation-driven transactions in the client code;
• register the JMS connection factory with the transaction manager, e.g., by creating an instance of com.atomikos.jms.AtomikosConnectionFactoryBean;
• a queue object, e.g., an instance of com.tibco.tibjms.TibjmsQueue or similar;
• a JmsTemplate with the dependency "defaultDestination" injected with the previously created queue;
• register the database(s) with the transaction manager, e.g., by creating an instance of com.atomikos.jdbc.AtomikosDataSourceBean (or com.atomikos.jdbc.nonxa.AtomikosNonXADataSourceBean for a last resource gambit scenario);
• one or more SqlUpdate (or BatchSqlUpdate or JdbcTemplate) instances (one per data source) each with the "dataSource" property injected with the previously defined data source(s);

• an instance of a custom DAO class with injected the previously defined JmsTemplate and SqlUpdate objects: this class will have one or more methods annotated @Transactional that will perform transactions using the JmsTemplate to receive messages and the SqlUpdate(s) to persist them on the databases, e.g., it could be a class that looks like this:

  public class MyDao{        
    JmsTemplate jmsTemplate; // getter/setter omitted for clarity
    BatchSqlUpdate sqlUpdate; // getter/setter omitted for clarity
  
    @Transactional
    public void persistMessages(int n){
      // map used for the sqlUpdate object
      Map<String, Object> params = new HashMap<>();
      // need to reset this since it's being reused
      sqlUpdate.reset();
      for(int i=0;i<n;i++){
        // retrieve a message synchronously
        Message msg = jmsTemplate.receive();
        // transform the message
        doSomeMagic(msg,params);
        // set parameters in the SQL
        sqlUpdate.updateByNamedParam(params);
      }
      // now the batch can be flushed
      sqlUpdate.flush();
    }
  
    private void doSomeMagic(Message msg, Map<String,Object> params){
      // implementation is application-dependent!
      // the only assumption is that somehow the
      // message can be used to set the named
      // parameters in the sqlUpdate object
    }
  }
  

The only thing worth noting is that the DAO must be spring-managed because the use of annotations requires spring to create a proxy.

Answer 2

I think your problem has nothing to do with Spring or Atomikos. It's more a design issue.

If you need all messages to be commited within a single transaction, merge them all in a bigger structure before sending the full message to your persistence layer.