Better Android Automated Test Performance using Expresso?

Question

I stumbled upon Espresso after researching automated testing frameworks for Android. It seemed to have everything that I wanted: reliable tests, minimal boilerplate code, increased performance.

I watched the GTAC 2013 presentation(s) demoing Espresso and was very excited to see how fast it ran the tests. Having actually implemented some tests, however, I must say that I don't notice much, if any performance benefit over using the standard Android testing framework. I have not done any sort of "official" benchmarking, but it was my understanding that Espresso blew away the standard Android testing framework.

The project that I am testing is the one described in the tutorial on developer.android.com. The tests that I am writing are very simple:

@Test
public void test_sendButton_shouldInitiallyBeDisabled() {
    onView(withId(R.id.button_send)).check(matches(not(ViewMatchers.isEnabled())));
}

@Test
public void test_sendButton_shouldBeEnabledAfterEnteringText() {
    String enteredText = "This is my message!";
    // Type Text 
    onView(withId(R.id.edit_message)).perform(ViewActions.typeText(enteredText));

    // Validate the Result
    onView(withId(R.id.button_send)).check(matches(ViewMatchers.isEnabled()));
}

@Test
public void test_enteringTextAndPressingSendButton_shouldDisplayEnteredText() {
    String enteredText = "This is my message!";
    // Type Text 
    onView(withId(R.id.edit_message)).perform(ViewActions.typeText(enteredText));

    // Click Button
    onView(withId(R.id.button_send)).perform(click());

    // Validate the Results
    onView(withId(R.id.display_message)).check(ViewAssertions.matches(ViewMatchers.withText(enteredText)));
}

I followed all the instructions on the Espresso website, paying particularly close attention that my Run Configuration used the GoogleInstrumentationTestRunner.

So what am I missing? Did I just miss something simple? Or is my premise about significantly improved performance completely wrong?

Answer 1

The fact that you don't notice a big difference vs standard instrumentation for very simple tests may be expected. For more complex tests (e.g. ones that span multiple activities) that run continuously (i.e. have to be stable), people often end up adding sleep/retry logic to guard against flakiness. With Espresso, you can remove that and enjoy a significant reduction in test runtime. Here is a G+ post that shows a comparison of Espresso vs Robotium tests.