Timeout for whole test in Robotium

Question

I have few classes that test my aplication. I'd like to fail test if it lasts more than 4 seconds. My code breaks the test but it doesn't execute next test class in some case.

When I write (it has nothing to do with timeout, but just an example of fail()):

public void testSmth() {
    fail("msg");
}

the Failure Trace is empty and it breaks the test and starts another. But when I want to make it like a timeout:

public void testSmth() {
    Timer timer = new Timer();
    timer.schedule(new TimerTask() {
        @Override
        public void run() {
            fail("msg");
        }
    }, 4000);

    // some tests (that even lasts more than 4 secons) like clickOnImage() etc.
}

it breaks the test but it doesn't execute next one and in the Failure Trace there is:

Test failed to run to completion. Reason: 'Instrumentation run failed due to ''junit.framework.AssertionFailedError''. Check device logcat for details

and in LogCat I got:

07-26 11:46:07.428: E/AndroidRuntime(6195): FATAL EXCEPTION: Timer-1

07-26 11:46:07.428: E/AndroidRuntime(6195): junit.framework.AssertionFailedError: msg

07-26 11:46:07.428: E/AndroidRuntime(6195): at junit.framework.Assert.fail(Assert.java:47)

07-26 11:46:07.428: E/AndroidRuntime(6195): at java.util.Timer$TimerImpl.run(Timer.java:284)

or maybe there's some another way to do what I want?

Thanks.

Answer 1

If you just want to fail test in case of reaching timeout this should be enough:

public void test1() throws Exception {
    long start = System.currentTimeMillis();
    solo.sleep(5000);
    if (System.currentTimeMillis() - start > 4000) {
        fail("msg");
    }
}

public void test2() throws Exception {
    long start = System.currentTimeMillis();
    solo.sleep(3000);
    if (System.currentTimeMillis() - start > 4000) {
        fail("msg");
    }
}

It's hard to break test during its execution, you can also check timeout after every command, however checking it will take time and your test methods will last longer.

Answer 2

You must override the runTest() method with something like this:

[EDIT] This code creates a new thread and runs the actual test on it. The Future class allows us to put a timeout on the execution of this thread, and when the timeout is reached, it is stopped. It also takes care of catching the exceptions. (Did I mentioned that the rest of the tests will still going to run?)

This is very helpful if you want to use this timeout to make sure the test does not remain 'hanged' somewhere in the tested code.

public class MyTestClass extends
                ActivityInstrumentationTestCase2<EditorActivity> {

@Override
public void runTest() throws Throwable {
    final Throwable[] exceptions = new Throwable[1];

    ExecutorService executor = Executors.newCachedThreadPool();
    Callable<Object> task = new Callable<Object>() {
        public Object call() throws Exception {

            try {
                doRunTest();
            }
            catch (Throwable t) {
                exceptions[0] = t;
            }

            return Boolean.TRUE;
        }
    };

    int timeOutMinutes = 10;
    String testCaseName = String.format("%s.%s", getClass().getName(), getName());

    Future<Object> future = executor.submit(task);
    try {
        future.get(timeOutMinutes, TimeUnit.MINUTES);
    } catch (TimeoutException ex) {
        Assertions.fail("[Test method timed out after " + timeOutMinutes + " minutes.]\n" + testCaseName);
    } catch (Throwable e) {
        throw e;
    } finally {
        future.cancel(true); // may or may not desire this
    }

    if (exceptions[0] != null) {
        throw exceptions[0];
    }
}


private void doRunTest() throws Throwable {
    super.runTest();
}

}