Camel jms to external activeMQ with request-reply gives javax.jms.InvalidDestinationException: Cannot publish to a deleted Destination:

Question

We have third party applications that make restful calls to camel cxfrs endpoint which is then routed to an external activeMQ. There are applications consuming these JMS messages and provide XML responses. This is all done synchronously using camel InOut exchangePattern. The architecture is very straight forward and simple. We were using activeMQ 5.5.0-fuse, camel-jms 2.8.x and activemq-pool 5.6.

Using this configuration, we see this exception at random times:

 javax.jms.InvalidDestinationException: Cannot publish to a deleted Destination: temp-  queue://ID:testserver-37266-1366126830205-0:0:1
    at org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1696)
    at org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:231)
    at org.apache.activemq.pool.PooledProducer.send(PooledProducer.java:74)
    at org.apache.activemq.pool.PooledProducer.send(PooledProducer.java:55)

When this happens the server just stalls and none of our services respond until we restart activeMQ, tomcat and all other services.

camel config:

<import resource="classpath:META-INF/cxf/cxf.xml"/>

 <bean id="routeBuilder" class="gov.nasa.arc.tmi.route.TMIServiceRoute"/>

 <bean id="jaxbProvider" class="org.apache.cxf.jaxrs.provider.JAXBElementProvider">
     <property name="marshallerProperties" ref="propertiesMap"/>
    </bean>
    <util:map id="propertiesMap">
        <entry key="jaxb.formatted.output">
           <value type="java.lang.Boolean">true</value>
        </entry>
    </util:map>

<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
    <routeBuilder ref="routeBuilder"/>
</camelContext>


<bean id="activemq" 
  class="org.apache.activemq.camel.component.ActiveMQComponent">
  <property name="brokerURL" value="tcp://localhost:61616"/>

Camel Router class: //reroute from("cxfrs:/rr?resourceClasses=x.y.z.route.RerouteResource") .setExchangePattern(ExchangePattern.InOut) .process(new RerouteProcessor()) .to("activemq:queue:x.y.z.tmi.request");

Here is the spring configuration of the application that listens on queue:x.y.z.tmi.request consumes JMS messages:

<context:annotation-config/>
<bean id="connectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
    <property name="brokerURL">
       <value>tcp://localhost:61616? wireFormat.maxInactivityDurationInitalDelay=30000</value>
    </property>
</bean> 

<bean id="pooledConnectionFactory" 
   class="org.apache.activemq.pool.PooledConnectionFactory" init-method="start" destroy-method="stop">
   <property name="maxConnections" value="8" />
   <property name="connectionFactory" ref="connectionFactory" />
</bean>

<bean id="destination" class="org.apache.activemq.command.ActiveMQQueue">
    <constructor-arg value="gov.nasa.arc.tmi.request"/>
</bean>

<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">    
    <property name="connectionFactory" ref="pooledConnectionFactory"/>
    <property name="defaultDestination" ref="destination" />
</bean> 

<jms:listener-container  connection-factory="pooledConnectionFactory" concurrency="10">
 <jms:listener  destination="gov.nasa.arc.tmi.request" ref="tmiQueryListener" />

After googling, I came across these bugs:

https://issues.apache.org/jira/browse/CAMEL-6229 https://issues.apache.org/jira/browse/AMQ-3457

Based on these, we upgraded to camel 2.10.4, activeMq 5.7 and activemq-pool 5.7. Even so, the problem remains.

I am really stuck and don't know how to resolve this issue. Can some one kindly point out what could be wrong?

Thanks.

Answer 1

I wonder if its because the processing the JMS message on the other side takes too long, and then the inactive monitor of ActiveMQ deletes the temp destination as its been inactive for > 30 seconds. http://activemq.apache.org/activemq-inactivitymonitor.html

Maybe try to set the timeout to a higher value, or disable it.

Another option is to use fixed queues for reply queues instead of temp queues.