How to get the number of entries written to DB in Spring Batch?

Question

I have a JobExecutionListener that writes the job status after execution into a log file. How can I also get the number of items processed?

@Component
public class JobListener implements JobExecutionListener {
    @Override
    public void afterJob(JobExecution task) {
          log.info(task.getStatus());
          //number of entries??
    }
}

Answer 1

I see 2 ways to do it.

1. You can implement the ItemProcessListener. This interface is called after/before an item is processed. This interface also reported any errors.

   public class ItemCountsListener implements ItemProcessListener<Object, Object> {
   
   private static final AtomicLong count = new AtomicLong(1);
   
   public void afterProcess(Object item, Object result) {
       count.getAndIncrement();
   }
   
   public void beforeProcess(Object item) {}
   public void onProcessError(Object item, Exception e) {  }
   

   }

2. Or you can call the method jobExecution.getStepExecutions(). This method returns a Collection<StepExecution> object. In this class, there is a method getWriteCount which returns the current number of items written for this execution. I would do something similar to this code :

 public void afterJob(JobExecution jobExecution) {
         int nbItemsProcessed;
         for (StepExecution stepExecution : jobExecution.getStepExecutions()) {
            nbItemsProcessed += stepExecution.getWriteCount();
         }   
      }

Answer 2

Using the JobExplorer, you can look at the previous StepExecutions to get the number of items read, etc in each step. You can read more about the JobExplorer here: http://docs.spring.io/spring-batch/trunk/apidocs/org/springframework/batch/core/explore/JobExplorer.html

Update

You actually don't even need to use the JobExplorer. Since you have the JobExecution, you already have references to all the StepExecutions. Each StepExecution contains the number of items read, processed, written, skipped, etc.

Answer 3

this is how i did it:

JobExecution job = jobLauncher.run(praepJob, getJobParameters());
job.getStepExecutions().stream().findFirst().get().getWriteCount()

Answer 4

Two ways,

1. Use JobExecution, here you can get access to StepExecution and thereby can get step read, write or errors
2. Spring Batch stores all its job and step data in batch tables. You can refer the docs here : Spring Batch Meta-Data

If you are using Meta-data tables, then you can configure jobExplorer that provides you to get the meta-data data.

Using xml you can configure it as,

<bean id="jobOperator"
        class="org.springframework.batch.core.launch.support.SimpleJobOperator">
        <property name="jobRegistry" ref="jobRegistry" />
        <property name="jobExplorer" ref="jobExplorer" />
        <property name="jobLauncher" ref="jobLauncher" />
        <property name="jobRepository" ref="jobRepository" />
    </bean>

    <bean id="jobRegistry" class="org.springframework.batch.core.configuration.support.MapJobRegistry" />


    <bean id="jobExplorer"
        class="org.springframework.batch.core.explore.support.JobExplorerFactoryBean">
        <property name="dataSource" ref="dataSource" />
        <property name="tablePrefix" value="BATCH_" />
    </bean>

    <bean id="jobLauncher"
        class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
        <property name="jobRepository" ref="jobRepository" />
        <property name="taskExecutor" ref="batchTaskExecutor"></property>
    </bean>

    <bean id="jobRepository"
        class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean">
        <property name="dataSource" ref="dataSource" />
        <property name="databaseType" value="oracle" />
        <property name="transactionManager" ref="transactionManager" />
    </bean>

    <bean id="batchTaskExecutor"
        class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
        <property name="corePoolSize" value="10" />
        <property name="maxPoolSize" value="25" />
        <property name="queueCapacity" value="30" />
    </bean>