How to order test cases for Spoon Automated Testing in Android?

Question

I have added a suite() method to order my tests the way I want them and thus when I run it through Android JUnit they are executed accordingly. But then I noticed that when I use the Spoon execution, the one using cmd, my test cases are executed alphabetically which is the default order.

Why does this happen and how would you counter it without renaming my test cases?

Answer 1

I have the same issue as you; I require a specific order that my test need to be ran in. The app I am testing is too complicated to run in an unpredictable order. My solution was this:

Add this to your build.gradle:

spoon {
     if (project.hasProperty('spoonClassName')){
         className = project.spoonClassName
      }
}

Now, you can execute a specific class with a command like so:

gradle spoon -PspoonClassName=< com.your.pakage.ClassName>

Next, create a file at the root of your Android project: runAllTests.sh

Edit your .sh to look like this:

 #!/bin/sh
 date +%b-%dT%H.%M > timestamp.out

 sites="$HOME"/path/to/project/root

 timestamp="$(cat "$sites"/timestamp.out)"
 result_folder="$sites"/results
 destdir="$result_folder/Results-$timestamp"

 mkdir -p "$destdir"
 echo "Directory created: ${destdir##*/}"

 <---------- Here you start running the test --------------->

 echo "Starting Master Setup"
 gradle spoon -PspoonClassName=com.espresso.test.MasterSetup
 cp -r "$sites"/app/build/spoon "$destdir"/MasterSetup
 echo "Results saved to MasterSetup"

 echo "Starting WorkoutSchedule"
 gradle spoon -PspoonClassName=com.espresso.test.WorkoutSchedule
 cp -f "$sites"/app/build/spoon "$destdir"/WorkoutSchedule
 echo "Results saved to WorkoutSchedule"

 echo "Starting Setting.test"
 gradle spoon -PspoonClassName=com.espresso.test.Settings
 cp -r "$sites"/app/build/spoon "$destdir"/Settings
 echo "Results saved to Settings"

Then, give the script permissions

1. cd to the script
2. type chmod u+x runAllTest.sh

You're set. Now just cd to your root, then to execute your test, type . runAllTest.sh.

So, what this does:

1. First, it creates a timestamp.out. I use this so I can save my results to a file over and over without previous results being overwritten. You do not need this part.
2. Next, it creates a result folder in the root of your project if it is not already there.
3. Then, it will make a folder inside the results folder named Results-SOME-DATE.
4. Lastly, each test will run, saving the results to the normal spot on your project. (Inside build/spoon) Once test are complete it will copy the results to the results folder, and name each test result appropriately so it is easy to see all your tests ran.

NOTE: This script was wrote for MAC. If your on windows or anything else, this script may need modifications.

---

Additionally: You will find it is inconvenient to open in to each folder to get the index.html opened. So I wrote this script to add to your bash_profile:

function open-results () {
  # the browser to open up `index.html' in.
  browser='/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'

  # let the user know what directory we're looking in
  printf "looking in %s" "$(pwd)"
  echo ...

  for paths in $(find ./ -name 'debug' -type d); do
    for files in $(find "$paths" -name 'index.html'); do
     open -a "$browser" "$files"
    done
  done
  echo done
 } 

Now, cd in terminal to the Results-SOME-DATE, and type open-results. Again, this was written for terminal. You may need to modify depending on your OS. But the structure should be the same

I hope this helps.

Answer 2

The jUnit testing philosophy is that test cases should not depend on each other so order shouldn't be important. That's why you're finding it hard to do. You might want to consider using the "setUp" method to create initial conditions for your test cases rather than having them build on each other.