Dynamically setting the $browsers static property in Sauce.io phpunit selenium tests

Question

I am setting up some browser testing using SauceLabs. I've been able to get tests running locally and via Sauce, so I am now trying to integrate it with my Jenkins install to trigger builds and browser tests automatically.

Most of this is all working, but I have one small issue. So that I can run my tests locally and via Sauce, I want to set the $browsers static property during the phpUnit setup() function, rather than hardcoding it. This doesn't seem possible.

I'm using the Sausage binding, my TestCase looks pretty similar to this demo: https://github.com/jlipps/sausage/blob/master/WebDriverDemo.php

I have tried in setUp() to update the $browsers array, but it never seems to take effect. eg.

public function setUp()
{   
    self::$browsers = array(
        'browserName' => getenv('SELENIUM_BROWSER'),
        'desiredCapabilities' => array(
            'version' => getenv('SELENIUM_VERSION'),
            'platform' => getenv('SELENIUM_PLATFORM'),
         )
     );
}

Is there a way to pass the browser details from Jenkins so the test cases are more flexible? I feel like I'm missing something simple here?

Answer 1

Ok, so I worked this out. Should anyone else have the same issue, here's how I resolved it.

In the ANT script that Jenkins runs, which runs PHPUnit in turn, I included a config.xml file. This sets a config (environment) variable called sauce to true

<phpunit> 
    <php>
        <env name="sauce" value="true" />
    </php>
</phpunit>

Now the trick is to not actually use the $browsers static array, but instead to use the setupSpecificBrowser method. So , now in my setUp() function of my tests, I just switch on the sauce env variable and if it exists then I know we are running from Jenkins and so I use the supplied variables from it.

    if( getenv('sauce') == true) {
        $browser =  array(
                'browserName' => getenv('SELENIUM_BROWSER'),
                'desiredCapabilities' => array(
                        'version' => getenv('SELENIUM_VERSION'),
                        'platform' => getenv('SELENIUM_PLATFORM'),
                )
        );
    } else {
        $browser =  array(
                'browserName' => 'firefox',
                'local' => true,
                'sessionStrategy' => 'isolated'
        );
    }
    $this->setupSpecificBrowser($browser);

AFAIK there doesn't seem to be any documentation for this, I just worked it from from looking at the code. Fun.

Answer 2

After spending quite a bit of time digging through the source, I found a solution to the "multiple browsers" scenario. chapmatic's assertions regarding multiple browsers and the given answer were indeed correct: it doesn't work for parallel testing and runs the same browser several times if you have multiple browsers defined in your $browsers array. The solution still uses environment variables, but you must still use your $browsers array.

So, first make sure you define your $browsers array in your abstract test class. Then, let's say you define the env var BROWSER and assign to it the browser you want to test. You can set up the following static function in your abstract test class that extends Sauce\Sausage\WebDriverTestCase:

    public static function browserSetup()
    {   
        switch (getenv('BROWSER')) {
            case 'firefox':
                self::$browsers = array(
                    array(
                        'browserName' => 'firefox',
                        'desiredCapabilities' => array(
                            'platform' => 'self::WIN_VERSION',
                            'version' => self::FIREFOX_VERSION,
                        )   
                    )   
                );  
                break;

            case 'safari':
                //safari desiredCapabilities
               break;

            case 'explorer':
                //ie desiredCapabilities
               break;

           case 'chrome':
             //chrome desiredCapabilities

           default: //This will just use the default $browsers array you defined
         return;
    }

Now that browserSetup() is defined, you have to make sure it is called before the test suite is set up so that the tests are are set to run only on the browser you specified in your BROWSER environment variable. Let's look at PHPUnit_Extensions_Selenium2TestCase, which is expended by Sauce\Sausage\WebDriverTestCase; PHPUnit_Extensions_Selenium2TestCase defines the following method:

public static function suite($className)
{   
    return PHPUnit_Extensions_SeleniumTestSuite::fromTestCaseClass($className);
} 

This method gets called to set up the test suite with all the browsers you specified in your $browsers array, so you need to override this method in your abstract test class, making sure to call browserSetup() before fromTestCaseClass() is called:

public static function suite($className)
{   
    self::browserSetup();
    return PHPUnit_Extensions_SeleniumTestSuite::fromTestCaseClass($className);
} 

Now, if you define the environment variable BROWSER with the browser you wish to test, you can kick off your test suite and your $browsers array will be properly overridden with the settings you specified for the single browser defined in your BROWSER environment variable. Make sure jenkins is properly setting this environment variable in the Build->Execute shell section, and you are good to go.

Answer 3

Just to note as well, that setting the browsers after the fact stops parallel testing. Moreover if you have three different browsers to start with (in $browsers static array) and set the browser to chrome then you will have three machines running chrome now.