Multiple Assert verification Handling

Question

In my test method below, this method has multiple asserts. I want it to run so that if all the asserts pass then it will log the status as "Passed" in my test management tool (we are integrating Test Link with Selenium)

But if any one assert fails than testNG generates an Assertion Error; in this situation I'm not able to update the status to "Failed" in my TMT, because this method's execution was stopped and the run has proceeded with the next method.

Please provide some help to solve this.

Test(dataProvider = "User_login")
public void StatusForm_Verification(String uname, String pwd)
        throws InterruptedException {
    NavigateToLogin();
    Dashboard RD = LoginAs_user(uname, pwd);
    Thread.sleep(2000);

        if (Integer.parseInt(ReviewedStatuscount) >= 1) {

            Assert.assertEquals("true",
                    revui.Btn_SaveReview.getAttribute("disabled"));
            Assert.assertEquals("true",
                    revui.Btn_submitReview.getAttribute("disabled"));
            Assert.assertEquals("true",
                    revui.Btn_Needmoreinfo.getAttribute("disabled"));
            status = TestLinkAPIResults.PASSED;

        } else {
            throw new SkipException(
                    "Test Skipping - Reviewed count is Zero");
        }
    }

My question is : if any one assert fails, than how can I update the status to Failed in the Testlink application (i.e status = TestLinkAPIResults.FAILED)?

Answer 1

You could use @after and check status after test execution

@After
public void statusCheck() {
    if (status!=TestLinkAPIResults.PASSED) {
        status = TestLinkAPIResults.FAILED;
    }
}

Answer 2

If you want to test all assert at once then you have to add softAssert in your code this will give you final results after check all the fields. If not when assert failed happens (Assume First name error is invalid) system will throw assertion error after it.

    SoftAssert softAssert = new SoftAssert();
    String ActualErrorMEssage = firstNameerrorXpath.getText;
    String ActualErrorMEssage2 = secondNameNameerrorXpath.getText;
    softAssert.assertEquals(ActualErrorMEssage,ExpectedErrorMEssage);
    softAssert.assertEquals(ActualErrorMEssage2,ExpectedErrorMEssage);
    softAssert.assertAll();

Answer 3

With Asserts you cannot. You would need to wrap your asserts in your custom methods which can catch the assertionerror and probably then update the testlink results, also marking the status of the testcase as failed. You can also try exploring the new testng asserts.

Answer 4

An alternative I like to use (with JUnit, but the idea should work with TestNG as well) is to extend a base PoorlyWrittenUnitTest class.

The following class allows a unit test to make multiple assertions without failing until the test is finished. All failures encountered will be logged to stderr with a stacktrace and after the test completes it will fail with a message stating how many failures were encountered (or pass if 0 errors were encountered).

This way you can just extend this class and run your test as usual making as many assertions as you like and the test won't fail until the end (see below for some caveats). This should allow your PASSED setting to be set and checked as the test won't fail until the @After method is invoked.

public abstract class AbstractPoorlyWrittenUnitTest
{
   private int m_assertionErrorCount = 0;

   @After
   public final void failTestIfErrorsEncountered ()
   {
      if( m_assertionErrorCount > 0 )
         Assert.fail( "Test error(s) encountered: " + m_assertionErrorCount );
   }

   protected final void assertTrue ( boolean condition )
   {
      try
      {
         Assert.assertTrue( condition );
      }
      catch( AssertionError failure )
      {
         logAssertionError( failure );
      }
   }

   protected final void assertFalse ( boolean condition )
   {
      try
      {
         Assert.assertFalse( condition );
      }
      catch( AssertionError failure )
      {
         logAssertionError( failure );
      }
   }

   protected final void assertEquals ( Object expected, Object actual )
   {
      try
      {
         Assert.assertEquals( expected, actual );
      }
      catch( AssertionError failure )
      {
         logAssertionError( failure );
      }
   }

   // include any other assertions you like to use (including message variants)

   private void logAssertionError ( @NotNull AssertionError failure )
   {
      System.err.println();
      ++m_assertionErrorCount;
      System.err.println( "Failure " + m_assertionErrorCount + ":" );
      failure.printStackTrace();
   }
}

Caveats:

If you assert something and go on as if the assertion were correct you may hit a RuntimeException which will cause the test to "error" out and no further assertions can be checked.

For instance, the 2nd assertion in the following code will throw NullPointerException. The test will still fail, but the last assertion will not be checked and the test will show 1 failure (the "NotNull" assertion) and 1 error (the NullPointerException).

Object myObject = null;
assertNotNull( myObject );
assertEquals( "Some text", myObject.toString() );
assertEquals( 4, 3 );

PS - I named this PoorlyWrittenUnitTest because most of our tests that need this code are indeed very poorly written legacy tests with multiple things-being-tested in a single test method and multiple assertions per thing-being-tested, and because most people I talk to consider tests with more than 1 assertion to be poorly written.