How to build dependent tests for regression testing

Question

I have an ASP.Net MVC project and I thought I could use a tool like MS Test or NUnit to perform regression testing from the controller layer down to the database, however I hit an issue where tests are not designed to run in order (You can use ordered tests in MS Test, but the tests still run concurrently) and the other problem is how to allow the data created from one test accessible to another?

I have looked at Selenium and WatiN but I just wanted to write something that is not dependent on the UI layer which is most likely going to change an increase the amount of work to maintain the tests.

Any suggestions? Is it just the wrong tool for the job? Should I just use Selenium/WatiN?

Answer 1

Update The link below (and I assume the project) is dead. Best option maybe using Selenium and the Page Object Model. See here: http://code.tutsplus.com/articles/maintainable-automated-ui-tests--net-35089

Old Answer The simplest solution I have found is Rob Conery's Qixote:

https://github.com/robconery/Quixote

It works by firing http requests and consuming responses. Simple to set up and use and provides integration testing. This tool also allows a series of tests to be executed in order to create test dependencies.

Answer 2

Tests should always be independent of each other, so that running order doesn't matter. If your tests depend on other tests you are losing control of what you are testing.

Answer 3

WatiN, and I'm assuming Selenium, won't solve your ordering problem. I use WatiN and NUnit for UI automation and the running order is not guaranteed, which initially posed similar problems to what you're seeing.

In the vein of what dskh answered, you want independent tests, and I've done this in two ways for Integration / Regression black-ish box testing.

First: In your test setup, have any precondition data values setup so you're at a known "good state". For system regression test automation, I've got a number of database scripts that get called to reset data to a known state; this adds some dependencies so be conscious of the design. Note: In straight unit testing, look at using mock objects to take out dependencies and get your test to be "testing one thing". Mock objects, stubbing method calls, etc is the way to go if you can, which based on your question sounds likely.

Second: For cases where certain things absolutely had to be setup in a certain way, and scripting them to test setup added a ridiculous amount of necessary system internal knowledge (eg: all users setup + all permissions setup + etc etc etc) a small number of "bootstrap" tests were setup, in their own namespace to allow easy running via NUnit, to bootstrap the system. Keeping the number of tests small and making sure the tests were very stable was paramount. Now, on a fresh install, the bootstrap tests are run first and serve as advanced smoke tests; no further tests are run if any of the bootstrap tests fail. It is clunky in some ways, but the alternatives were clunkier or more time/resource/whatever consuming.