Setting up ehcache replication - what multicast settings do I need?

Question

I am trying to set up ehcache replication as documented here: http://ehcache.sourceforge.net/EhcacheUserGuide.html#id.s22.2
This is on a Windows machine but will ultimately run on Solaris in production.

The instructions say to set up a provider as follows:

 <cacheManagerPeerProviderFactory
     class="net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory"
     properties="peerDiscovery=automatic, multicastGroupAddress=230.0.0.1,
     multicastGroupPort=4446, timeToLive=32"/>

And a listener like this:

<cacheManagerPeerListenerFactory
    class="net.sf.ehcache.distribution.RMICacheManagerPeerListenerFactory"
    properties="hostName=localhost, port=40001,
    socketTimeoutMillis=2000"/>

My questions are:
Are the multicast IP address and port arbitrary (I know the address has to live within a specific range but do they have to be specific numbers)?
Do they need to be set up in some way by our system administrator (I am on an office network)?

I want to test it locally so am running two separate tomcat instances with the above config. What do I need to change in each one? I know both the listeners can't listen on the same port - but what about the provider?
Also, are the listener ports arbitrary too?

I've tried setting it up as above but in my testing the caches don't appear to be replicated - the value added in one tomcat's cache is not present in the other cache.
Is there anything I can do to debug this situation (other than packet sniffing)?

Thanks in advance for any help, been tearing my hair out over this one!

Answer 1

I want to test it locally so am running two separate tomcat instances with the above config.

As I have just submitted an answer regarding cherouvims related question I'd just like to highlight here too that they are in fact providing an example doing something similar at least (multiple nodes per host, though one instance only): see section Full Example in the RMI Distributed Caching documentation.

It's indeed important to change the TCP port for each cacheManagerPeerListenerFactory but the multicast settings can (I'd think they must) stay the same: you can see this in action within the ehcache.xml's for the Full Example mentioned above: the port number is increased one by one from 40001 up to 40006 per node while the multicast settings stay identical.

If you have obeyed that, your problem might just be related to Tomcat running on Windows - see the related paragraph within section Common Problems in RMI Distributed Caching:

There is a bug in Tomcat and/or the JDK where any RMI listener will fail to start on Tomcat if the installation path has spaces in it.
[...]
As the default on Windows is to install Tomcat in "Program Files", this issue will occur by default.

Answer 2

One mistake I encountered during EHCache peer replication. This might be useful so someone attempting peer replication on a single local box.

I wanted to run 3 peer-replicated instances of EhCache on a single Windows Vista box using peerDiscovery=automatic (the eventual target environment was Linux).

The peer-replication did not work. Oddly, all EhCache instances were starting-up without any errors or complaints in the log.

Took me a while to figure my mistake; without anything in the logs I had to grok around: I was using the same listenerPort (40001) in the peer listener configuration for each of the EhCache instances. Using different listenerPorts (40001, 40002...) did the trick.

<cacheManagerPeerListenerFactory 
class="net.sf.ehcache.distribution.RMICacheManagerPeerListenerFactory" 
properties="port=40001, socketTimeoutMillis=3000"/>

<cacheManagerPeerListenerFactory 
class="net.sf.ehcache.distribution.RMICacheManagerPeerListenerFactory" 
properties="port=40002, socketTimeoutMillis=3000"/>

Note that this is the 'peer listener' configuration that needed different port numbers. The multicast 'peer discovery' still uses identical multicast address and port numbers for every ehcache instance.

Answer 3

Make sure your servers have multicast enabled for starters. Not sure what platform you are running on.

Answer 4

Just as a quick follow-up.

We did get this working with two separate machines which solved the original problem of getting it to work at all. As far as I recall Multicast should work "out of the box" but it's worth checking with your local sysadmin (ours suggested a tweak to the multicastGroupAddress).

We eventually hit all sorts of problems on Solaris and ended up abandoning multicast for Manual Peer Discovery instead.

Finally, in terms of debugging, using jconsole to monitor cache values proved to be the best way to do it. Didn't have to resort to packet sniffing. :-)

Answer 5

 <cacheManagerPeerListenerFactory class="net.sf.ehcache.distribution.RMICacheManagerPeerListenerFactory"/>

just doing the above, ehcache will use the next available one ! So you don't have to worry about configuration on new nodes that you want to deploy in the cluster.

Answer 6

May help Someone implementing multicasting. I have verified by running my app in my local tomcat from eclipse and running the same app in the local tomcat from eclipse of my colleague.

Running below on your Windows Machine will show you some ips and ensures that multi-casting should work.

netsh interface ip show joins

the attribute multicastGroupAddress=230.0.0.1 means that your application want to use this IP, this can be changed but ensure to use the Class D IP Range only. After running your application, you could see this IP 230.0.0.1 Also in as below

To see all magics, ensure to enable logs properly.

    <logger name="net.sf.ehcache">  <level value="debug"/></logger>
   <logger name="net.sf.ehcache.distribution.RMICachePeer" level="debug" />

Ensure to Have below lisners in your default cache or required cache configuration

         <cacheManagerPeerProviderFactory
       class="net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory"
       properties="port=40003, peerDiscovery=automatic,
                   multicastGroupAddress=230.0.0.1,
                   multicastGroupPort=4446,
                   timeToLive=32"/>
    <cacheManagerPeerListenerFactory 
        class="net.sf.ehcache.distribution.RMICacheManagerPeerListenerFactory"
        properties="port=40001, socketTimeoutMillis=3000" />
    <cacheManagerPeerListenerFactory 
        class="net.sf.ehcache.distribution.RMICacheManagerPeerListenerFactory" 
        properties="port=40002, socketTimeoutMillis=3000"/>
 <defaultCache
     eternal="false"
     timeToIdleSeconds="300"
     timeToLiveSeconds="300"
     diskExpiryThreadIntervalSeconds="305"
     memoryStoreEvictionPolicy="LRU"
     statistics="true">
     <persistence strategy="localTempSwap"/>
     <cacheEventListenerFactory class="net.sf.ehcache.distribution.RMICacheReplicatorFactory"/>
     <bootstrapCacheLoaderFactory class="net.sf.ehcache.distribution.RMIBootstrapCacheLoaderFactory" properties="bootstrapAsynchronously=true, maximumChunkSizeBytes=5000000"/>
        </defaultCache>

After Deploying the application you could see below in logs

13:31:27.073  DEBUG n.s.e.d.PayloadUtil assembleUrlList - Cache peers for this CacheManager to be advertised:......
15:15:14.110  DEBUG net.sf.ehcache.distribution.MulticastKeepaliveHeartbeatReceiver processPayload rmiUrls received....(YOur configured caches with Ips)

Which will be helpfull if you need to configure manual peer discovery process.