Acceptance Testing Rails using a "Testing API" for client side state control

Question

I am currently evaluating how to test a rather big and complex web application, based on Rails 4 on the server side and EmberJS on the client side. In our app, the client exclusively communicates through a restful JSON API with the server.

We did a lot of unit testing based on Konacha so far and are now willing to setup integration/acceptance tests too. We are not sure whether we should start writing end-to-end tests, so tests including a running instance of our server, or whether we should go for integration testing the API and the client side separately.

Our preferred choice at the moment is end-to-end testing, because we fear that in the case of integration testing API and client separately we have twice the effort of creating and maintaining tests and that there might be tiny, little specialities of the comunication between API and client which we could not catch.

Well, we like modern & fast testing frameworks like Konacha, so we don't really want to use Selenium. Not only because it feels a little bit old, but also because its performance is quite poor. Still you will need to control instantiation of mock data on the server and reset of the server, why we came up with the following approach:

We implemented a Testing API which conceptually is used to control the state of the server, e.g. it has the following methods:

GET /api/test/setup  # Simple bootstrapping of the database, e.g. populate table with ISO language codes etc...
GET /api/test/reset  # Reset the database, using `database_cleaner` gem

A konacha test case would then call setup and reset, before and after each test case respectively.

What do you think about this approach?

Answer 1

Not sure what I would call the test of the API and the client separately, but even if you think about running this kind of test, you still should got for the end-to-end test.

So yes, I think your idea of going for end-to-end testing is very good.

Your idea of setting up simple commands to allow test automation for your system (the setup and the reset commands) is very good as well. Be prepared to add more during automation - while an end-to-end-test is conceptually a black-box test, in reality it's often a grey-box test, i.e. you will need to access the internal state of your system. I would call this the "operation and maintenance interface" of the system under test.