Spring-Batch: How to ensure when a Job is running, it is not allowed to run again at the same time

StackOverflow https://stackoverflow.com/questions/15653913

  •  29-03-2022
  •  | 
  •  

Question

How to ensure when a Job is running, it is not allowed to run again at the same time?

We have BJ that takes 1 hour to process the feed and populate the temp tables. First step of this BJ is to clear the temp tables and start populating the data from the Main store front tables.

Consider a scenario that when the BJ is started (first time running), if we start the BJ again, it will delete content from temp tables as part of Step one.

So please suggest on how i can hold the second execution till the first is not COMPLETED?

Was it helpful?

Solution

You may create custom Tasklet as your first step and use JobExecutionDao in it to find all JobExecutions. If there are more than one running - throw an exception.

OTHER TIPS

I am sure, this will not be the best solution, but I hope this will serve your situation nevertheless.

While executing the job, make sure, you run the job with always the same parameter. After the completion of the successful execution of your job, configure your calling-scripts to delete the entry corresponding to that execution of the batch-job.

This way, it will give error and won't allow you to run 2 executions of the same job at the same time. Deletion will ensure the serial execution is allowed.

ALTERNATIVE METHOD: Write your job with a single parameter job-execution-id. Everytime before you execute the job, query the maximum value of job-execution-id for the completed jobs from the batch tables for the job. Now, execute the job with job-execution-id incremented by 1 as input parameter.

I think this is a better method than above. I am not sure, if springbatch itself provides any easy-take-away options for implementing this scenario.

Maybe I'm misinterpreting your question, but you can limit the number of parallel executions of any single step by specifiying a throttle-limit on the Tasklet within a Step. Specifying one should make sure, that you have only one execution at a time:

<batch:step id="stepA" next="stepB">
  <batch:tasklet throttle-limit="1">
    <batch:chunk reader="myReader" writer="myWriter" commit-interval="100"/>
  </batch:tasklet>
</batch:step>

You can set up the spring-batch-Admin UI to view the status of the jobs (failed/running/completed,etc). With proper set-up of Spring Batch Admin UI, you can even view the status of the several tasks inside different jobs.

Implementing this within a single JVM should be do-able using a Binary semaphore. This will help avoid parallel executions of the same job. Making the second instance wait will be a little tricky if you dont want it to simply skip execution if semaphore value is set.

You can do more complex serialization (including across Spring batch nodes) using a suitable "Leader Election" implementation. I have used Netflix Curator (an Apache Zookeeper recipe) in my project. Some pointers here : https://github.com/regunathb/Trooper/wiki/Useful-Batch-Libraries

I did this by writing special incrementer that increments properties only when previous job execution is completed.

    public class CompletedJobRunIdIncrementer extends RunIdIncrementer {
    private final JobRepository jobRepository;
    private final String jobName;

    public CompletedJobRunIdIncrementer(JobRepository jobRepository, String jobName) {
        this.jobRepository = jobRepository;
        this.jobName = jobName;
    }

    @Override
    public JobParameters getNext(JobParameters parameters) {
        JobExecution lastJobExecution = jobRepository.getLastJobExecution(jobName, parameters);
        return lastJobExecution == null || lastJobExecution.getStatus() == BatchStatus.COMPLETED ? super.getNext(parameters) : parameters;
    }
}

and the Job with this incrementer:

jobBuilders.get("myJob").incrementer(new CompletedJobRunIdIncrementer(jobRepository, "myJob").start(someTask()).build()

You can add custom implementation of JobExecutionListener.

Below is the sample listener implementation: 

    @Component
    public class JobExecutionListener implements JobExecutionListener{


@Autowired
private JobExplorer jobExplorer;


@Override
public void beforeJob(JobExecution jobExecution) {
    int runningJobsCount = jobExplorer.findRunningJobExecutions(jobExecution.getJobInstance().getJobName()).size();
    if(runningJobsCount > 1){
        throw new RuntimeException("There are already active running instances of this job, Please cancel those executions first.");
    }
}

@Override
public void afterJob(JobExecution jobExecution) {

}

}

This will mark the current launch of job as failed if you have any instance of this job already running. You can handle this exception based on your business requirement.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top