Ok, I know that this sounds simple from the title but it really has me stumped as to why this might be occurring.
So, I'm using Spring Batch to generate emails to be sent using Amazon's Simple Email Service. Inside my CustomItemProcessor
, I am using @Autowired
to wire in my AmazonEmailService
service as normal. The AmazonEmailService
class implements my EmailSender
interface.
The AmazonEmailService
has an @Autowired
AmazonSimpleEmailServiceClient
that it uses to actually call the Amazon Simple Email Service to do things.
My AmazonSimpleEmailServiceClient
bean is defined in my root-servlet.xml:
<bean id="amazonSimpleEmailServiceClient"
class="com.amazonaws.services.simpleemail.AmazonSimpleEmailServiceClient">
<constructor-arg ref="basicAWSCredentials" />
</bean>
<bean id="basicAWSCredentials" class="com.amazonaws.auth.BasicAWSCredentials">
<constructor-arg index="0" value="${aws.accessKey}"/>
<constructor-arg index="1" value="${aws.secretKey}" />
</bean>
All this works fine.
My problem is that when I run my Spring Batch Job integration tests. They hang when trying to send an email. My logging shows that execution just stops at the call to amazonSimpleEmailServiceClient.send(emailRequest)
and doesn't proceed.
The thing that has me absolutely flummoxed is that if I run the unit tests for the AmazonEmailService
prior to running the integration test for my Spring Batch Jobs, then everything completes successfully. I want to know why this is the case. Its worth mentioning that the batch job that actually sends the emails has a TaskExecutor
running on a single thread.
The integration test for the Spring Batch job, is supposed to verify that the job is successfully generating an email for every input it receives and that those emails can be sent using Amazon SES. It also tests that it is correctly reading and writing objects to queues that are setup, but that isn't relevant to my issue. The unit test for the AmazonEmailService
just sends out 3 emails to the Amazon email simulator.
I'll post an abridged class diagram below so you can see how things stick together.
- The EmailService interface is my own interface.
- The AmazonEmailService is my own service. This service actually performs the sending of the emails using the AmazonSimpleEmailServiceClient which is an object that was given to me by Amazon.
- The CustomItemProcessor is my own object that implements the ItemProcessor interface. This is what Spring Batch uses to actually do processing on the items in the batch job. The emails are supposed to be generated and sent by this class.
- The AmazonEmailServiceTest is just a unit test that tests the ability
of the AmazonEmailService to actually send emails.
Things you may assume when considering why I am having this stupid issue:
- I have correctly configured my unit/integration tests to run in the Spring application context.
- The
AmazonEmailServiceTest
runs successfully by itself.
- The Spring Batch integration tests run successfully if I revert to using a simple JavaMail email sender.
- My Spring Batch jobs, application context and beans are correctly configured and defined using XML config for all my other classes.
- My Amazon credentials are valid and work correctly.
- Classes are autowired in correctly.
- No exception is thrown at any stage of trying to run the Spring Batch test either in isolation or with the
AmazonEmailServiceTest
- When running the tests together, the same instance of the
AmazonEmailService
is autowired into both tests (ie same memory address for both tests etc)
- There is no way for me to check the credentials used in the actual unit tests.
- I have correctly configured the PropertyPlaceholderConfigurer inside the XML config.
- The call to
amazonSimpleEmailServiceClient.send(emailRequest)
will hang until I manually stop the unit test.
- The beans concerned are discoverable in the application context for autowiring.
If you need any more information, like classes or config, don't hesitate to ask. I am literally sitting in front of my computer waiting for reply's.
AmazonEmailService:
@Service
public class AmazonEmailService implements EmailService {
@Autowired
private AmazonSimpleEmailServiceClient amazonSimpleEmailServiceClient;
@Override
public String sendEmail(Email email){
//build request using Builder pattern//
return amazonSimpleEmailServiceClient.sendEmail(emailRequest);
}
}
CustomItemProcessor:
public class CustomProcessQueueItemProcessor implements
ItemProcessor<Foo, Bar> {
@Autowired
private EmailService amazonEmailService;
@Override
public Bar process(Foo foo) throws Exception {
//generate email from Foo object//
String result = amazonEmailService.sendEmail(email);
//create Bar object from result//
return bar;
}
}
AmazonEmailServiceTest:
public class AmazonEmailServiceTest extends SpringTest{
@Autowired
private EmailService amazonEmailService;
@Test
public void testSendEmailSuccess() {
Email successEmail = MockObjectFactory.setTestSuccessEmail();
String emailResultId = amazonEmailService.sendEmail(successEmail);
assertNotNull("The returned emailResultId was null", emailResultId);
}
}
The SpringTest class is where I configure my unit tests to run in the Spring application context. My MockObjectFactory, is what its name suggests, a class containing static methods to generate test objects.
Batch Servlet:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:batch="http://www.springframework.org/schema/batch"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.2.xsd
http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-2.2.xsd">
<import resource="jobs/fill-queue-job.xml" />
<import resource="jobs/process-queue-job.xml" />
<batch:job-repository id="jobRepository"
data-source="dataSource" transaction-manager="transactionManager" />
<bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
<property name="taskExecutor" ref="defaultTaskExecutor"></property>
</bean>
<bean id="jobRegistry"
class="org.springframework.batch.core.configuration.support.MapJobRegistry" />
<bean id="jobRegistryBeanPostProcessor"
class="org.springframework.batch.core.configuration.support.JobRegistryBeanPostProcessor">
<property name="jobRegistry" ref="jobRegistry" />
</bean>
<bean id="jobLoader"
class="org.springframework.batch.core.configuration.support.DefaultJobLoader">
<property name="jobRegistry" ref="jobRegistry" />
</bean>
<bean id="jobExplorer"
class="org.springframework.batch.core.explore.support.JobExplorerFactoryBean">
<property name="dataSource" ref="dataSource" />
</bean>
<bean id="jobOperator"
class="org.springframework.batch.core.launch.support.SimpleJobOperator">
<property name="jobLauncher" ref="jobLauncher" />
<property name="jobRepository" ref="jobRepository" />
<property name="jobRegistry" ref="jobRegistry" />
<property name="jobExplorer" ref="jobExplorer" />
</bean>
<bean id="domainObjectIdQueue" class="java.util.concurrent.ConcurrentLinkedQueue" />
<bean id="mainQueue" class="java.util.concurrent.ConcurrentLinkedQueue" />
<bean id="notificationQueue" class="java.util.concurrent.ConcurrentLinkedQueue" />
<bean id="fillQueueItemReader"
class="au.com.mail.batch.itemreaders.CustomServiceItemReader"
scope="step">
<constructor-arg ref="emailTaskServiceImpl" />
</bean>
<bean id="fillQueueItemProcessor"
class="au.com.mail.batch.itemprocessors.CustomFillQueueItemProcessor"
scope="step" />
<bean id="fillQueueCompositeItemWriter"
class="org.springframework.batch.item.support.CompositeItemWriter">
<property name="delegates">
<list>
<bean id="fillQueueItemWriter"
class="au.com.mail.batch.itemwriters.CustomQueueItemWriter"
scope="step" />
<bean id="emailTaskItemWriter"
class="au.com.mail.batch.itemwriters.CustomServiceItemWriter"
scope="step">
<constructor-arg ref="emailTaskServiceImpl" />
</bean>
</list>
</property>
</bean>
<bean id="processQueueItemReader"
class="au.com.mail.batch.itemreaders.CustomQueueItemReader"
scope="step">
<constructor-arg>
<value type="java.lang.Class">au.com.mail.domainobject.messagewrappers.MainQueueMessageWrapper
</value>
</constructor-arg>
</bean>
<bean id="processQueueItemProcessor"
class="au.com.mail.batch.itemprocessors.CustomProcessQueueItemProcessor"
scope="step" />
<bean id="processQueueItemWriter"
class="au.com.mail.batch.itemwriters.CustomQueueItemWriter"
scope="step" />
<bean id="defaultTaskExecutor"
class="org.springframework.scheduling.quartz.SimpleThreadPoolTaskExecutor">
<property name="threadCount" value="5" />
</bean>
<bean id="processQueueTaskExecutor"
class="org.springframework.scheduling.quartz.SimpleThreadPoolTaskExecutor">
<property name="threadCount" value="1" />
</bean>
<bean id="customStepExecutionListener" class="au.com.mail.batch.CustomStepExecutionListener"
scope="step" />
<bean id="jobLauncherTestUtils" class="org.springframework.batch.test.JobLauncherTestUtils">
<property name="job" ref="fillQueue" />
<property name="jobRepository" ref="jobRepository" />
<property name="jobLauncher" ref="jobLauncher" />
</bean>
</beans>
Process Queue Job Definition:
<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/batch"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:beans="http://www.springframework.org/schema/beans"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.2.xsd
http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-2.2.xsd">
<beans:bean id="repeatQueueOpenTasklet"
class="au.com.mail.batch.CustomQueueRetryTaskletImpl" scope="step" />
<job id="processQueue" job-repository="jobRepository">
<step id="getQueue">
<tasklet ref="repeatQueueOpenTasklet" task-executor="processQueueTaskExecutor">
</tasklet>
<end on="FAILED" exit-code="NOOP" />
<next on="*" to="sendEmail" />
</step>
<step id="sendEmail">
<tasklet task-executor="processQueueTaskExecutor">
<chunk reader="processQueueItemReader" processor="processQueueItemProcessor"
writer="processQueueItemWriter" commit-interval="5" />
</tasklet>
<listeners>
<listener ref="customStepExecutionListener"></listener>
</listeners>
</step>
</job>
</beans:beans>
BIG BIG BIG UPDATE:
I removed my processQueueTaskExecutor
on the process queue tasklet and removed the defaultTaskExecutor
from the jobLauncher
and the Amazon service call succeeds. Now I just need to know why this is the case.