Tomcat active passive failover

Question

I'm trying to make my application highly available using active/passive method. I currently deploy the application on a Tomcat server, which could either be hosted on Windows or Linux platform.

So, now I deploy it on two Tomcat servers. Each server runs on a separate machine. I configure the Tomcats to be of the same cluster via Tomcat configuration which gives me session replication (storing in DB). I use some web server / load balancer to redirect requests (not decided yet).

The problem is that the application cannot run simultaneously on both Tomcats since it is currently stateful (and it's too costly to make it stateless). I actually need that only one Tomcat will be up at the same time. Or, for the least, that only one application will be started on a Tomcat.

To sum it up, I need a software that can:

1. Identify invalid state of application via HTTP request (either machine/Tomcat is down or application is otherwise unresponsive);
2. Start Tomcat instance or application on Tomcat when the other application is identified as invalid;
3. Stop Tomcat instance or application on Tomcat when it's identified as invalid.
4. Support both Linux and Windows platforms...

This product can be a library that's used on our JVM, it may be separate monitoring processes that reside on the Tomcats machines and communicate with each-other and with the Tomcats, it may use our DB. It may use the web server machine...

I searched for off-of-the-shelf products that do it (Pacemaker/CoroSync/keepalived), all not supporting Windows (as far as I understand).

Answer 1

I ended up doing the following:

Both Tomcats are always started. The application is also started on both of the Tomcats.

A special synchronization service was hand-written. This service is part of the application and started along with the application startup. It is responsible for determination of the node that is currently active and which one should be active.

My application used a Spring context for managing my services lifecycle. I separated it into two different contexts: a wrapper Spring context and a child Spring context.

The wrapper context was started along with the application startup, always available and working on both Tomcats. The synchronization service was configured as a bean in the wrapper context.

The child context actually included all services that my application provided for external clients. This is the context that I designated to be started only when current node is active and stopped when becoming passive.

How was the child context swithched on / off when active / passive?

This context wasn't started automatically along the application startup. When the synchronization service concluded that the current node should become active, an event was raised that caused the child context to get started, thus making all services available on that node. And the other way around: when the synchronization service concluded that the current node shuold become passive, a proper event was raise that caused the child context to shutdown, making all services unavailable on that node.

Another important feature that was added was an interceptor for incoming requests. An incoming request hitting the one of the Tomcats, actually implies that the load balancer / web server has decided that the target node is the currently available or preferred node. In such a case, even if it happened that the target node was the passive one, it should now become active. So the intercepror would raise the same event that the synchronization service did, again making the child context to start and the services available on that node (and the synchronization serivces on both Tomcats identified the switch and the formerly active node would passivate).