Scheduled job in a multi node environment

Question

I am working on a scheduled job that will run at certain interval (eg. once a day at 1pm), scheduled through Cron. I am working with Java and Spring.

Writing the scheduled job is easy enough - it does: grab list of people will certain criteria from db, for each person do some calculation and trigger a message.

I am working on a single-node environment locally and in testing, however when we go to production, it will be multi-node environment (with load balancer, etc). My concern is how would multi node environment affect the scheduled job?

My guess is I could (or very likely would) end up with triggering duplicate message.

• Machine 1: Grab list of people, do calculation
• Machine 2: Grab list of people, do calculation
• Machine 1: Trigger message
• Machine 2: Trigger message

Is my guess correct?

What would be the recommended solution to avoid the above issue? Do I need to create a master/slave distributed system solution to manage multi node environment?

Answer 1

If you have something like three Tomcat instances, each load balanced behind Apache, for example, and on each your application runs then you will have three different triggers and your job will run three times. I don't think you will have a multi-node environment with distributed job execution unless some kind of mechanism for distributing the parts of the job is in place. If you haven't looked at this project yet, take a peek at Spring XD. It handles Spring Batch Jobs and can be run in distributed mode.