Why do my tests fail when run together, but pass individually?

Question

When I write a test in Visual Studio, I check that it works by saving, building and then running the test it in Nunit (right click on the test then run).

The test works yay... so I Move on...

Now I have written another test and it works as I have saved and tested it like above. But, they dont work when they are run together.

Here are my two tests that work when run as individuals but fail when run together:

using System;
using NUnit.Framework;
using OpenQA.Selenium.Support.UI;
using OpenQA.Selenium;

namespace Fixtures.Users.Page1
{
    [TestFixture]
    public class AdminNavigateToPage1 : SeleniumTestBase
    {
        [Test]
        public void AdminNavigateToPage1()
        {
            NavigateTo<LogonPage>().LogonAsCustomerAdministrator();
            NavigateTo<Page1>();
            var headerelement = Driver.FindElement(By.ClassName("header"));

            Assert.That(headerelement.Text, Is.EqualTo("Page Title"));
            Assert.That(Driver.Url, Is.EqualTo("http://localhost/Page Title"));
        }

        [Test]
        public void AdminNavigateToPage1ViaMenu()
        {
            NavigateTo<LogonPage>().LogonAsCustomerAdministrator();
            Driver.FindElement(By.Id("menuitem1")).Click();
            Driver.FindElement(By.Id("submenuitem4")).Click();
            var headerelement = Driver.FindElement(By.ClassName("header"));

            Assert.That(headerelement.Text, Is.EqualTo("Page Title"));
            Assert.That(Driver.Url, Is.EqualTo("http://localhost/Page Title"));
        }
    }
}

When the second test fails because they have been run together

Nunit presents this:

Sse.Bec.Web.Tests.Fixtures.ManageSitesAndUsers.ChangeOfPremises.AdminNavigateToChangeOfPremises.AdminNavigateToPageChangeOfPremisesViaMenu: OpenQA.Selenium.NoSuchElementException : The element could not be found

And this line is highlighted:

var headerelement = Driver.FindElement(By.ClassName("header"));

Does anyone know why my code fails when run together, but passes when run alone?

Any answer would be greatly appreciated!

Answer 1

Two things you can try

1. put the break point between the following two lines. And see which page are you in when the second line is hit

2. Introduce a slight delay between these two lines via Thread.Sleep

   Driver.FindElement(By.Id("submenuitem4")).Click(); var headerelement = Driver.FindElement(By.ClassName("header"));

Answer 2

Such a situation normally occurs when the unit tests are using shared resources/data in some way.

1. It can also happen if your system under test has static fields/properties which are being leveraged to compute the output on which you are asserting.
2. It can happen if the system under test is being shared (static) dependencies.

Answer 3

look into the TestFixtureSetup, Setup, TestFixtureTearDown and TearDown.
These attributes allow you to setup the testenvironment once, instead of once per test.

Answer 4

Without knowing how Selenium works, my bet is on Driver which seems to be a static class so the 2 tests are sharing state. One example of shared state is Driver.Url. Because the tests are run in parallel, there is a race condition to set the state of this object.

That said, I do not have a solution for you :)

Answer 5

If none of the answers above worked for you, i solved this issue by adding Thread.Sleep(1) before the assertion in the failing test...

Looks like tests synchronization is missed somewhere... Please note that my tests were not order dependant, that i haven't any static member nor external dependency.

Answer 6

Are you sure that after running one of the tests the method

NavigateTo<LogonPage>().LogonAsCustomerAdministrator();

is taking you back to where you should be? It'd seem that the failure is due to improper navigation handler (supposing that the header element is present and found in both tests).

Answer 7

I think you need to ensure, that you can log on for the second test, this might fail, because you are logged on already?

-> putting the logon in a set up method or (because it seems you are using the same user for both tests) even up to the fixture setup -> the logoff (if needed) might be put in the tear down method

     [SetUp]
     public void LaunchTest()
     {
        NavigateTo<LogonPage>().LogonAsCustomerAdministrator();
     }

     [TearDown]
     public void StopTest()
     {
        // logoff
     }
     [Test]
     public void Test1()
     {...}
     [Test]
     public void Test2()
     {...}

If there are delays in the DOM instead of a thread.sleep I recommend to use webdriver.wait in combination with conditions. The sleep might work in 80% and in others not. The wait polls until a timeout is reached which is more reliable and also readable. Here an example how I usually approach this:

    var webDriverWait = new WebDriverWait(webDriver, ..);
    webDriverWait.Until(d => d.FindElement(By.CssSelector(".."))
        .Displayed))