How Activemq Virtual topic subscription propagation in Network of brokers works?

Question

Could somebody clarify behavior of activemq virtual topics in a context of Network of Brokers? I have a confusion about subscription propagation.

For example, there is one broker which has network connector to another one. Lets say broker mq001 has following network connector open to the broker mq002:

<networkConnectors>
            <networkConnector name="connectorToRemoteBroker" uri="static:(tcp://mq002:61616)?maxReconnectAttempts=0" duplex="false"                networkTTL="3" decreaseNetworkConsumerPriority="true">
</networkConnectors>

Then I run consumer (A) to a Virtual topic on the broker mq001: endpointURI: activemq:Consumer.A.VirtualTopic.tempTopic

I can notice some interesting behavior in the activemq console. First of all, there is no topic "VirtualTopic.tempTopic" created. However, there is queue (underlying physical queue of virtual topic) available - Consumer.A.VirtualTopic.tempTopic And this queue has one active local consumer.

Then I start another consumer (B) to the same virtual topic but already on the broker 2 (mq002).

endpointURI - activemq:Consumer.B.VirtualTopic.tempTopic

If I take a look at activemq console on the broker 2 now. I still do not see any virtual topic available. There is another physical queue created Consumer.B.VirtualTopic.tempTopic which has one active consumer (also local for mq002).

When I take a look at console on the broker one I see two queues now:

Consumer.A.VirtualTopic.tempTopic - with an active local consumer Consumer.B.VirtualTopic.tempTopic - with an active remote consumer.

So subscription propagation works on the level of physical queues at least. And because it is not an duplex it works from mq002 to mq001 only.

Then I publish message to the topic:

 activemq:topic:VirtualTopic.tempTopic

It is being consumed by both consumers on mq001 and mq002. Also there is finally topic available in the activemq console (VirtualTopic.tempTopic).

So each consumer consumed exactly one message. If I repeat it with bigger number of messages it still works the same. No duplicates arrived and there are also no lost messages. The number of enqueued messages on the each physical queue matches with the number on the virtual topic.

That is exactly behavior I would expect from a virtual topic in case of network of brokers.

But now the source of my confusion:

http://activemq.apache.org/virtual-destinations.html#VirtualDestinations-AvoidingDuplicateMessageinaNetworkofBrokers

it is likely you will get duplicate messages if you use the default network configuration. This is because a network node will not only forward message sent to the virtual topic, but also the associated physical queues.

First of all I have not seen any duplicates, and it worked well. But what would happen if I will follow the advice and disable the physical queue destination?

<networkConnectors>
            <networkConnector name="connectorToRemoteBroker" uri="static:(tcp://mq002:61616)?maxReconnectAttempts=0" duplex="false" networkTTL="3" decreaseNetworkConsumerPriority="true">
                <excludedDestinations>
                        <queue physicalName="Consumer.*.VirtualTopic.>"/>
                </excludedDestinations>
          </networkConnector>
        </networkConnectors> 

Then when I do start consumers, I do not see remote consumer anymore listening to the physical queue Consumer.B on the broker mq001. And if I publish a messages to the virtual topic, then it is consumed by Consumer.A (local) only. So it looks like subscription propagation is ignored for virtual topics and works on the physical queues only.

It looks for me like activemq documentation is a little bit outdated. Can anybody confirm or refute it?

Thanks in advance!

Answer 1

So your tests above are correct. I just updated the docs to specify that you can get the dups when using both traditional topic subscribers AND virtual topic subscribers to the same destination over the network. That means, in your example, if I had a topic subscriber to "VirtualTopic.tempTopic" on mq002 as well as a consumer to queue "Consumer.B.VirtualTopic.tempTopic" then I can end up with dups. Hope that's clears things up. If you're using ONLY queue-based subscribers, then don't exclude the queue-based demand forwarding.

I have written a unit test that you can take a look at here:

http://svn.apache.org/viewvc/activemq/trunk/activemq-unit-tests/src/test/java/org/apache/activemq/usecases/TwoBrokerVirtualTopicForwardingTest.java?view=markup