Integration Test for Spring Batch Job failing

Question

Ok, I know that this sounds simple from the title but it really has me stumped as to why this might be occurring.

So, I'm using Spring Batch to generate emails to be sent using Amazon's Simple Email Service. Inside my CustomItemProcessor, I am using @Autowired to wire in my AmazonEmailService service as normal. The AmazonEmailService class implements my EmailSender interface.

The AmazonEmailService has an @Autowired AmazonSimpleEmailServiceClient that it uses to actually call the Amazon Simple Email Service to do things.

My AmazonSimpleEmailServiceClient bean is defined in my root-servlet.xml:

<bean id="amazonSimpleEmailServiceClient"
    class="com.amazonaws.services.simpleemail.AmazonSimpleEmailServiceClient">
    <constructor-arg ref="basicAWSCredentials" />
</bean>

<bean id="basicAWSCredentials" class="com.amazonaws.auth.BasicAWSCredentials">
    <constructor-arg index="0" value="${aws.accessKey}"/>
    <constructor-arg index="1" value="${aws.secretKey}" />
</bean>

All this works fine.

My problem is that when I run my Spring Batch Job integration tests. They hang when trying to send an email. My logging shows that execution just stops at the call to amazonSimpleEmailServiceClient.send(emailRequest) and doesn't proceed.

The thing that has me absolutely flummoxed is that if I run the unit tests for the AmazonEmailService prior to running the integration test for my Spring Batch Jobs, then everything completes successfully. I want to know why this is the case. Its worth mentioning that the batch job that actually sends the emails has a TaskExecutor running on a single thread.

The integration test for the Spring Batch job, is supposed to verify that the job is successfully generating an email for every input it receives and that those emails can be sent using Amazon SES. It also tests that it is correctly reading and writing objects to queues that are setup, but that isn't relevant to my issue. The unit test for the AmazonEmailService just sends out 3 emails to the Amazon email simulator.

I'll post an abridged class diagram below so you can see how things stick together.

• The EmailService interface is my own interface.
• The AmazonEmailService is my own service. This service actually performs the sending of the emails using the AmazonSimpleEmailServiceClient which is an object that was given to me by Amazon.
• The CustomItemProcessor is my own object that implements the ItemProcessor interface. This is what Spring Batch uses to actually do processing on the items in the batch job. The emails are supposed to be generated and sent by this class.
• The AmazonEmailServiceTest is just a unit test that tests the ability of the AmazonEmailService to actually send emails.

Things you may assume when considering why I am having this stupid issue:

1. I have correctly configured my unit/integration tests to run in the Spring application context.
2. The AmazonEmailServiceTest runs successfully by itself.
3. The Spring Batch integration tests run successfully if I revert to using a simple JavaMail email sender.
4. My Spring Batch jobs, application context and beans are correctly configured and defined using XML config for all my other classes.
5. My Amazon credentials are valid and work correctly.
6. Classes are autowired in correctly.
7. No exception is thrown at any stage of trying to run the Spring Batch test either in isolation or with the AmazonEmailServiceTest
8. When running the tests together, the same instance of the AmazonEmailService is autowired into both tests (ie same memory address for both tests etc)
9. There is no way for me to check the credentials used in the actual unit tests.
10. I have correctly configured the PropertyPlaceholderConfigurer inside the XML config.
11. The call to amazonSimpleEmailServiceClient.send(emailRequest) will hang until I manually stop the unit test.
12. The beans concerned are discoverable in the application context for autowiring.

If you need any more information, like classes or config, don't hesitate to ask. I am literally sitting in front of my computer waiting for reply's.

AmazonEmailService:

@Service
public class AmazonEmailService implements EmailService {

    @Autowired
    private AmazonSimpleEmailServiceClient amazonSimpleEmailServiceClient;

    @Override
    public String sendEmail(Email email){
        //build request using Builder pattern//
        return amazonSimpleEmailServiceClient.sendEmail(emailRequest);
    }
}

CustomItemProcessor:

public class CustomProcessQueueItemProcessor implements
    ItemProcessor<Foo, Bar> {

    @Autowired
    private EmailService amazonEmailService;

    @Override
    public Bar process(Foo foo) throws Exception {
        //generate email from Foo object//
        String result = amazonEmailService.sendEmail(email);
        //create Bar object from result//
        return bar;
    }
}

AmazonEmailServiceTest:

public class AmazonEmailServiceTest extends SpringTest{

@Autowired
private EmailService amazonEmailService;

@Test
public void testSendEmailSuccess() {
    Email successEmail = MockObjectFactory.setTestSuccessEmail();
    String emailResultId = amazonEmailService.sendEmail(successEmail);
    assertNotNull("The returned emailResultId was null", emailResultId);
}
}

The SpringTest class is where I configure my unit tests to run in the Spring application context. My MockObjectFactory, is what its name suggests, a class containing static methods to generate test objects.

Batch Servlet:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:batch="http://www.springframework.org/schema/batch"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.2.xsd
        http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-2.2.xsd">

    <import resource="jobs/fill-queue-job.xml" />
    <import resource="jobs/process-queue-job.xml" />

    <batch:job-repository id="jobRepository"
        data-source="dataSource" transaction-manager="transactionManager" />

    <bean id="jobLauncher"
        class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
        <property name="jobRepository" ref="jobRepository" />
        <property name="taskExecutor" ref="defaultTaskExecutor"></property>
    </bean>

    <bean id="jobRegistry"
        class="org.springframework.batch.core.configuration.support.MapJobRegistry" />

    <bean id="jobRegistryBeanPostProcessor"
        class="org.springframework.batch.core.configuration.support.JobRegistryBeanPostProcessor">
        <property name="jobRegistry" ref="jobRegistry" />
    </bean>

    <bean id="jobLoader"
        class="org.springframework.batch.core.configuration.support.DefaultJobLoader">
        <property name="jobRegistry" ref="jobRegistry" />
    </bean>

    <bean id="jobExplorer"
        class="org.springframework.batch.core.explore.support.JobExplorerFactoryBean">
        <property name="dataSource" ref="dataSource" />
    </bean>

    <bean id="jobOperator"
        class="org.springframework.batch.core.launch.support.SimpleJobOperator">
        <property name="jobLauncher" ref="jobLauncher" />
        <property name="jobRepository" ref="jobRepository" />
        <property name="jobRegistry" ref="jobRegistry" />
        <property name="jobExplorer" ref="jobExplorer" />
    </bean>

    <bean id="domainObjectIdQueue" class="java.util.concurrent.ConcurrentLinkedQueue" />

    <bean id="mainQueue" class="java.util.concurrent.ConcurrentLinkedQueue" />

    <bean id="notificationQueue" class="java.util.concurrent.ConcurrentLinkedQueue" />

    <bean id="fillQueueItemReader"
        class="au.com.mail.batch.itemreaders.CustomServiceItemReader"
        scope="step">
        <constructor-arg ref="emailTaskServiceImpl" />
    </bean>

    <bean id="fillQueueItemProcessor"
        class="au.com.mail.batch.itemprocessors.CustomFillQueueItemProcessor"
        scope="step" />

    <bean id="fillQueueCompositeItemWriter"
        class="org.springframework.batch.item.support.CompositeItemWriter">
        <property name="delegates">
            <list>
                <bean id="fillQueueItemWriter"
                    class="au.com.mail.batch.itemwriters.CustomQueueItemWriter"
                    scope="step" />
                <bean id="emailTaskItemWriter"
                    class="au.com.mail.batch.itemwriters.CustomServiceItemWriter"
                    scope="step">
                    <constructor-arg ref="emailTaskServiceImpl" />
                </bean>
            </list>
        </property>
    </bean>

    <bean id="processQueueItemReader"
        class="au.com.mail.batch.itemreaders.CustomQueueItemReader"
        scope="step">
        <constructor-arg>
            <value type="java.lang.Class">au.com.mail.domainobject.messagewrappers.MainQueueMessageWrapper
            </value>
        </constructor-arg>
    </bean>

    <bean id="processQueueItemProcessor"
        class="au.com.mail.batch.itemprocessors.CustomProcessQueueItemProcessor"
        scope="step" />

    <bean id="processQueueItemWriter"
        class="au.com.mail.batch.itemwriters.CustomQueueItemWriter"
        scope="step" />

    <bean id="defaultTaskExecutor"
        class="org.springframework.scheduling.quartz.SimpleThreadPoolTaskExecutor">
        <property name="threadCount" value="5" />
    </bean>
    
    <bean id="processQueueTaskExecutor"
        class="org.springframework.scheduling.quartz.SimpleThreadPoolTaskExecutor">
        <property name="threadCount" value="1" />
    </bean>
    

    <bean id="customStepExecutionListener" class="au.com.mail.batch.CustomStepExecutionListener"
        scope="step" />

    <bean id="jobLauncherTestUtils" class="org.springframework.batch.test.JobLauncherTestUtils">
        <property name="job" ref="fillQueue" />
        <property name="jobRepository" ref="jobRepository" />
        <property name="jobLauncher" ref="jobLauncher" />
    </bean>
</beans>

Process Queue Job Definition:

<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/batch"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:beans="http://www.springframework.org/schema/beans"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.2.xsd
        http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-2.2.xsd">

    <beans:bean id="repeatQueueOpenTasklet"
        class="au.com.mail.batch.CustomQueueRetryTaskletImpl" scope="step" />

    <job id="processQueue" job-repository="jobRepository">
        <step id="getQueue">
            <tasklet ref="repeatQueueOpenTasklet" task-executor="processQueueTaskExecutor">
            </tasklet>
            <end on="FAILED" exit-code="NOOP" />
            <next on="*" to="sendEmail" />
        </step>
        <step id="sendEmail">
            <tasklet task-executor="processQueueTaskExecutor">
                <chunk reader="processQueueItemReader" processor="processQueueItemProcessor"
                    writer="processQueueItemWriter" commit-interval="5" />
            </tasklet>
            <listeners>
                <listener ref="customStepExecutionListener"></listener>
            </listeners>
        </step>
    </job>
</beans:beans>

BIG BIG BIG UPDATE: I removed my processQueueTaskExecutor on the process queue tasklet and removed the defaultTaskExecutor from the jobLauncher and the Amazon service call succeeds. Now I just need to know why this is the case.

Answer 1

Ok, I finally sorted it all out. It was nothing to do with my unit/integration tests and everything to do with the multithreaded Spring Batch task executor. The task executor was calling the Amazon Email Service, while a class called AwsSdkMetrics had a lock on a synchronised method called useDefaultMetrics in the main thread. This meant that the execution couldn't proceed inside the task executor and so it hung waiting for the main thread to release that synchronised method.

So I fork'd my jUnit JVM from the ANT JVM and everything started working like a charm.

Answer 2

It's not clear from your post what your "unit" test is actually trying to test. It's safe to assume that Amazon's email service is well tested, and doesn't need to be actually tested in your unit test.

Instead you can create a new test spring context for your unit test that provides a mock of the EmailService, and then verify in your unit test that the emailService.sendEmail(...) method is actually being called when you expect it to be, with the content you expect. This way your test never need interact with any actual email service.