With Spoon, a test will not run twice if the first test passed. This is cause it believes if it runs the test again, it will pass and there is no reason for that. Bad design on Square's part if you ask me.
The solution is: gradle clean spoon
. clean
will regenerate the res
files (among others) and make spoon believe that it is essentially a different test. This makes running tests take longer than it should. But it works.