How much should an integration test contain? Where are side effects tested?

Question

Say I have a feature requirement that when a user successfully signs up for a paid account, several things happen:

1. The user's account is created
2. The user's credit card is billed, resulting in a new transaction record
3. A receipt is emailed to the user

Most of the rspec integration or cucumber tests I've seen focus pretty exclusively on what's displayed on screen, e.g. expect(page).to have_text(X) or Then I should see X

How much should an integration test cover here?

Should the integration test check...

• that an account record was created?
• that a transaction was created?
• that the transaction was for the correct amount?
• that the receipt email was sent?

Why or why not? If not, where/how should these types of side effects be tested?

Answer 1

If you develop outside in, most of your integration tests will be acceptance tests. These are about what the user sees, so they interact with the system from outside. They don't look in the database or otherwise go behind the scenes. They don't need to: if there was a reason to save something in the database, we'll know if it worked if it showed up on the screen. They do need to test that an email was sent or that an account was created in the third-party credit card processing system, because those are part of what the user sees.

After you've implemented all of your acceptance tests you might look at the system as an engineer and decide that some aspects of how classes in the system interact with one another or with external systems aren't adequately tested. First, decide whether these are telling you about requirements that should really be in the acceptance tests (which is often the case), and, if they are, improve the acceptance tests.

You might still feel the need to write some more integration tests that aren't acceptance tests, and they might need to look directly in the database or whatever. In my experience these are in the minority.

That said, sometimes it makes pragmatic sense for an acceptance test to stop in mid-scenario and just test side effects, because another acceptance test has already described the rest of the scenario. (For example, maybe there are 18 different ways to create an account in your app. Maybe they all result in the same account in the database in mid-scenario.) Similarly, you might want to fake up some database state and start in mid-scenario to avoid duplication; it's common to do this with user login.

After your application implements all of its acceptance tests and has good integration coverage, test the details with unit tests, which do test side effects. But if your acceptance/integration tests already test a given piece of code adequately, it doesn't need unit tests and its side effects might never be directly tested -- which is fine.