Run PHPUnit Tests in Certain Order

Question

Is there a way to get the tests inside of a TestCase to run in a certain order? For example, I want to separate the life cycle of an object from creation to use to destruction but I need to make sure that the object is set up first before I run the other tests.

Answer 1

Maybe there is a design problem in your tests.

Usually each test must not depend on any other tests, so they can run in any order.

Each test needs to instantiate and destroy everything it needs to run, that would be the perfect approach, you should never share objects and states between tests.

Can you be more specific about why you need the same object for N tests?

Answer 2

PHPUnit supports test dependencies via the @depends annotation.

Here is an example from the documentation where tests will be run in an order that satisfies dependencies, with each dependent test passing an argument to the next:

class StackTest extends PHPUnit_Framework_TestCase
{
    public function testEmpty()
    {
        $stack = array();
        $this->assertEmpty($stack);

        return $stack;
    }

    /**
     * @depends testEmpty
     */
    public function testPush(array $stack)
    {
        array_push($stack, 'foo');
        $this->assertEquals('foo', $stack[count($stack)-1]);
        $this->assertNotEmpty($stack);

        return $stack;
    }

    /**
     * @depends testPush
     */
    public function testPop(array $stack)
    {
        $this->assertEquals('foo', array_pop($stack));
        $this->assertEmpty($stack);
    }
}

However, it's important to note that tests with unresolved dependencies will not be executed (desirable, as this brings attention quickly to the failing test). So, it's important to pay close attention when using dependencies.

Answer 3

The correct answer for this is a proper configuration file for tests. I had the same problem and fixed it by creating testsuite with necessary test files order:

phpunit.xml:

<phpunit
        colors="true"
        bootstrap="./tests/bootstrap.php"
        convertErrorsToExceptions="true"
        convertNoticesToExceptions="true"
        convertWarningsToExceptions="true"
        strict="true"
        stopOnError="false"
        stopOnFailure="false"
        stopOnIncomplete="false"
        stopOnSkipped="false"
        stopOnRisky="false"
>
    <testsuites>
        <testsuite name="Your tests">
            <file>file1</file> //this will be run before file2
            <file>file2</file> //this depends on file1
        </testsuite>
    </testsuites>
</phpunit>

Answer 4

If you want your tests to share various helper objects and settings, you can use setUp(), tearDown() to add to the sharedFixture property.

Answer 5

PHPUnit allows the use of '@depends' annotation which specifies dependent test cases and allows passing arguments between dependent test cases.

Answer 6

In my view, take the following scenario where I need to test creation and destroying of a particular resource.

Initially I had two methods, a. testCreateResource and b. testDestroyResource

a. testCreateResource

<?php
$app->createResource('resource');
$this->assertTrue($app->hasResource('resource'));
?>

b. testDestroyResource

<?php
$app->destroyResource('resource');
$this->assertFalse($app->hasResource('resource'));
?>

I think this is a bad idea, as testDestroyResource depends upon testCreateResource. And a better practice would be to do

a. testCreateResource

<?php
$app->createResource('resource');
$this->assertTrue($app->hasResource('resource'));
$app->deleteResource('resource');
?>

b. testDestroyResource

<?php
$app->createResource('resource');
$app->destroyResource('resource');
$this->assertFalse($app->hasResource('resource'));
?>

Answer 7

Alternative solution: Use static(!) functions in your tests to create reusable elements. For instance (I use selenium IDE to record tests and phpunit-selenium (github) to run test inside browser)

class LoginTest extends SeleniumClearTestCase
{
    public function testAdminLogin()
    {
        self::adminLogin($this);
    }

    public function testLogout()
    {
        self::adminLogin($this);
        self::logout($this);
    }

    public static function adminLogin($t)
    {
        self::login($t, 'john.smith@gmail.com', 'pAs$w0rd');
        $t->assertEquals('John Smith', $t->getText('css=span.hidden-xs'));
    }

    // @source LoginTest.se
    public static function login($t, $login, $pass)
    {
        $t->open('/');
        $t->click("xpath=(//a[contains(text(),'Log In')])[2]");
        $t->waitForPageToLoad('30000');
        $t->type('name=email', $login);
        $t->type('name=password', $pass);
        $t->click("//button[@type='submit']");
        $t->waitForPageToLoad('30000');
    }

    // @source LogoutTest.se
    public static function logout($t)
    {
        $t->click('css=span.hidden-xs');
        $t->click('link=Logout');
        $t->waitForPageToLoad('30000');
        $t->assertEquals('PANEL', $t->getText("xpath=(//a[contains(text(),'Panel')])[2]"));
    }
}

Ok, and now, i can use this reusable elements in other test :) For instance:

class ChangeBlogTitleTest extends SeleniumClearTestCase
{
    public function testAddBlogTitle()
    {
      self::addBlogTitle($this,'I like my boobies');
      self::cleanAddBlogTitle();
    }

    public static function addBlogTitle($t,$title) {
      LoginTest::adminLogin($t);

      $t->click('link=ChangeTitle');
      ...
      $t->type('name=blog-title', $title);
      LoginTest::logout($t);
      LoginTest::login($t, 'paris@gmail.com','hilton');
      $t->screenshot(); // take some photos :)
      $t->assertEquals($title, $t->getText('...'));
    }

    public static function cleanAddBlogTitle() {
        $lastTitle = BlogTitlesHistory::orderBy('id')->first();
        $lastTitle->delete();
    }

• In this way, you can build hierarchy of you tests.
• You can steel keep property that each test case is totaly separate from other (if you clean DB after each test).
• And most important, if for instance, the way of login change in future, you only modify LoginTest class, and you don'n need correct login part in other tests (they should work after update LoginTest) :)

When I run test my script clean up db ad the begining. Above I use my SeleniumClearTestCase class (I make screenshot() and other nice functions there) it is extension of MigrationToSelenium2 (from github, to port recorded tests in firefox using seleniumIDE + ff plugin "Selenium IDE: PHP Formatters" ) which is extension of my class LaravelTestCase (it is copy of Illuminate\Foundation\Testing\TestCase but not extends PHPUnit_Framework_TestCase) which setup laravel to have access to eloquent when we want to clean DB at the end of test) which is extension of PHPUnit_Extensions_Selenium2TestCase. To set up laravel eloquent I have also in SeleniumClearTestCase function createApplication (which is called at setUp, and I take this function from laral test/TestCase)

Answer 8

There really is a problem with your tests if they need to run in a certain order. Each test should be totally independent of the others: it helps you with defect localization, and allows you to get repeatable (and therefore debuggable) results.

Checkout this site for a whole load of ideas / information, about how to factor your tests in a manner where you avoid these kinds of issues.