Test a specific exception type is thrown AND the exception has the right properties

StackOverflow https://stackoverflow.com/questions/23270078

Question

I want to test that MyException is thrown in a certain case. EXPECT_THROW is good here. But I also want to check the exception has a specific state e.g e.msg() == "Cucumber overflow".

How is this best implemented in GTest?

Was it helpful?

Solution 2

I mostly second Lilshieste's answer but would add that you also should verify that the wrong exception type is not thrown:

#include <stdexcept>
#include "gtest/gtest.h"

struct foo
{
    int bar(int i) {
        if (i > 100) {
            throw std::out_of_range("Out of range");
        }
        return i;
    }
};

TEST(foo_test,out_of_range)
{
    foo f;
    try {
        f.bar(111);
        FAIL() << "Expected std::out_of_range";
    }
    catch(std::out_of_range const & err) {
        EXPECT_EQ(err.what(),std::string("Out of range"));
    }
    catch(...) {
        FAIL() << "Expected std::out_of_range";
    }
}

int main(int argc, char **argv) {
  ::testing::InitGoogleTest(&argc, argv);
  return RUN_ALL_TESTS();
}

OTHER TIPS

A colleague came up with the solution by just re-throwing the exception.

The knack: no need of extra FAIL() statements, just the two EXPECT... calls that test the bits you actually want: the exception as such and its value.

TEST(Exception, HasCertainMessage )
{
    // this tests _that_ the expected exception is thrown
    EXPECT_THROW({
        try
        {
            thisShallThrow();
        }
        catch( const MyException& e )
        {
            // and this tests that it has the correct message
            EXPECT_STREQ( "Cucumber overflow", e.what() );
            throw;
        }
    }, MyException );
}

I had previously offered a macro to solve this in an older answer. However time has passed and a new feature was added to GTest, which allows for this without macros.

The feature is a set of matchers, e.g., Throws that can be used in combination with EXPECT_THAT(). However the documentation does not seem to have been updated, so the only information is hidden in this GitHub issue.

The feature is used like this:

EXPECT_THAT([]() { throw std::runtime_error("message"); },
    Throws<std::runtime_error>());

EXPECT_THAT([]() { throw std::runtime_error("message"); },
    ThrowsMessage<std::runtime_error>(HasSubstr("message")));

EXPECT_THAT([]() { throw std::runtime_error("message"); },
    ThrowsMessageHasSubstr<std::runtime_error>("message"));

EXPECT_THAT([]() { throw std::runtime_error("message"); },
    Throws<std::runtime_error>(Property(&std::runtime_error::what,
         HasSubstr("message"))));

Note that due to how EXPECT_THAT() works you need to put the throwing statement into something invokable without arguments. Hence the lambdas in the examples above.

Edit: This feature is included beginning with version 1.11.

Also note that this feature is not included in version 1.10, but it has been merged into master. Because GTest follows abseil's live at head policy there is are no new versions planned at the moment. Also they don't seem to follow abseil's policy to release specific versions for those of use who can't/won't live at head.

Jeff Langr describes a good approach in his book, Modern C++ Programming with Test-Driven Development:

If your [testing] framework does not support a single-line declarative assert that ensures an exception is thrown, you can use the following structure in your test:

    TEST(ATweet, RequiresUserNameToStartWithAnAtSign) {
        string invalidUser("notStartingWith@");
        try {
            Tweet tweet("msg", invalidUser);
            FAIL();
        }
        catch(const InvalidUserException& expected) {}
    }

[...] You might also need to use the try-catch structure if you must verify any postconditions after the exception is thrown. For example, you may want to verify the text associated with the thrown exception object.

    TEST(ATweet, RequiresUserNameToStartWithAtSign) {
        string invalidUser("notStartingWith@");
        try {
            Tweet tweet("msg", invalidUser);
            FAIL();
        }
        catch(const InvalidUserException& expected) {
            ASSERT_STREQ("notStartingWith@", expected.what());
        }
    }

(p.95)

This is the approach I've used, and have seen in practice elsewhere.

Edit: As has been pointed out by @MikeKinghan, this doesn't quite match the functionality provided by EXPECT_THROW; the test doesn't fail if the wrong exception is thrown. An additional catch clause could be added to address this:

catch(...) {
    FAIL();
}

A new feature was added to GTest master on 2020-08-24 (post v1.10) which I explained in a separate answer. However I'll leave this answer because it still helps if the version you're using doesn't support the new feature. Also this and similar solution is a bit easier to use.

Also see Bryant's solution which improves on this one. (no longer an exception_ptr polluting your scope)

As I need to do several of such tests I wrote a macro that basically includes Mike Kinghan's answer but "removes" all the boilerplate code:

#define ASSERT_THROW_KEEP_AS_E(statement, expected_exception) \
    std::exception_ptr _exceptionPtr; \
    try \
    { \
        (statement);\
        FAIL() << "Expected: " #statement " throws an exception of type " \
          #expected_exception ".\n  Actual: it throws nothing."; \
    } \
    catch (expected_exception const &) \
    { \
        _exceptionPtr = std::current_exception(); \
    } \
    catch (...) \
    { \
        FAIL() << "Expected: " #statement " throws an exception of type " \
          #expected_exception ".\n  Actual: it throws a different type."; \
    } \
    try \
    { \
        std::rethrow_exception(_exceptionPtr); \
    } \
    catch (expected_exception const & e)

Usage:

ASSERT_THROW_KEEP_AS_E(foo(), MyException)
{
    ASSERT_STREQ("Cucumber overflow", e.msg());
}

Caveats:

  • As the macro defines a variable in the current scope, so it can only be used once.
  • C++11 is needed for std::exception_ptr

I recommend defining a new macro based on Mike Kinghan's approach.

#define ASSERT_EXCEPTION( TRY_BLOCK, EXCEPTION_TYPE, MESSAGE )        \
try                                                                   \
{                                                                     \
    TRY_BLOCK                                                         \
    FAIL() << "exception '" << MESSAGE << "' not thrown at all!";     \
}                                                                     \
catch( const EXCEPTION_TYPE& e )                                      \
{                                                                     \
    EXPECT_EQ( MESSAGE, e.what() )                                    \
        << " exception message is incorrect. Expected the following " \
           "message:\n\n"                                             \
        << MESSAGE << "\n";                                           \
}                                                                     \
catch( ... )                                                          \
{                                                                     \
    FAIL() << "exception '" << MESSAGE                                \
           << "' not thrown with expected type '" << #EXCEPTION_TYPE  \
           << "'!";                                                   \
}

Mike's TEST(foo_test,out_of_range) example would then be

TEST(foo_test,out_of_range)
{
    foo f;
    ASSERT_EXCEPTION( { f.bar(111); }, std::out_of_range, "Out of range" );
}

which I think ends up being much more readable.

I found the original answer given by Brandlingo (the macro solution) to create a better experience than the EXPECT_THAT alternative, which is still limiting if you need to examine the content of the exception in a non-trivial way.

There were some caveats to Brandlingo's answer though that I've resolved here:

  1. Removed the unscoped exception_ptr that makes it annoying to use multiple times, by nesting try blocks instead and throwing to the outer block only if the expected exception occurs.
  2. Assume any unexpected exception derives from std::exception, and print the value of "what()". Useful for providing hints as to what happened. If the exception doesn't derive from std::exception, the test will still fail, you just won't get a line number. You could handle it by adding another catch, but that really shouldn't be happening.
  3. Pass in the name of the variable to use for the exception (for readability).
#define ASSERT_THROW_AND_USE(statement, expected_exception, exception_varname)                   \
    try {                                                                                         \
        try {                                                                                     \
            (statement);                                                                          \
            FAIL() << "Expected: " #statement " throws an exception of type " #expected_exception \
                      ".\n  Actual: it throws nothing.";                                          \
        } catch (const expected_exception&) {                                                     \
            throw;                                                                                \
        } catch (const std::exception& e) {                                                       \
            FAIL() << "Expected: " #statement " throws an exception of type " #expected_exception \
                      ".\n  Actual: it throws a different type, with message: "                   \
                   << std::quoted(e.what());                                                      \
        }                                                                                         \
    } catch (const expected_exception& exception_varname)

Then use it like this:

ASSERT_THROW_AND_USE(foo(), MyException, e)
{
    ASSERT_STREQ("fail message", e.MyMethod());
}

If the wrong exception is thrown, you'll get something like this:

Failed
Expected: foo() throws an exception of type MyException.
  Actual: it throws a different type, with message: "some other thing"

My version; it produces the same output as EXPECT_THROW and just adds the string test:

#define EXPECT_THROW_MSG(statement, expected_exception, expected_what)                    \
  try                                                                                     \
  {                                                                                       \
    statement;                                                                            \
    FAIL() << "Expected: " #statement " throws an exception of type " #expected_exception \
              ".\n"                                                                       \
              "  Actual: it throws nothing.";                                             \
  }                                                                                       \
  catch (const expected_exception& e)                                                     \
  {                                                                                       \
    EXPECT_EQ(expected_what, std::string{e.what()});                                      \
  }                                                                                       \
  catch (...)                                                                             \
  {                                                                                       \
    FAIL() << "Expected: " #statement " throws an exception of type " #expected_exception \
              ".\n"                                                                       \
              "  Actual: it throws a different type.";                                    \
  }

I like most of the answers. However, since it seems that GoogleTest provides EXPECT_PRED_FORMAT that helps facilitating this, I'd like to add this option to the list of answers:

MyExceptionCreatingClass testObject; // implements TriggerMyException()

EXPECT_PRED_FORMAT2(ExceptionChecker, testObject, "My_Expected_Exception_Text");

where ExceptionChecker is defined as:

testing::AssertionResult ExceptionChecker(const char* aExpr1,
                                          const char* aExpr2,
                                          MyExceptionCreatingClass& aExceptionCreatingObject,
                                          const char* aExceptionText)
{
  try
  {
    aExceptionCreatingObject.TriggerMyException();
    // we should not get here since we expect an exception
    return testing::AssertionFailure() << "Exception '" << aExceptionText << "' is not thrown.";
  }
  catch (const MyExpectedExceptionType& e)
  {
    // expected this, but verify the exception contains the correct text
    if (strstr(e.what(), aExceptionText) == static_cast<const char*>(NULL))
    {
      return testing::AssertionFailure()
          << "Exception message is incorrect. Expected it to contain '"
          << aExceptionText << "', whereas the text is '" << e.what() << "'.\n";
    }
  }
  catch ( ... )
  {
    // we got an exception alright, but the wrong one...
    return testing::AssertionFailure() << "Exception '" << aExceptionText
    << "' not thrown with expected type 'MyExpectedExceptionType'.";
  }
  return testing::AssertionSuccess();
}

I use Matthäus Brandl's macro with the following minor modification:

Put the line

std::exception_ptr _exceptionPtr;

outside (f.e. before) the macro definition as

static std::exception_ptr _exceptionPtr;

to avoid multiple definition of the symbol _exceptionPtr.

Expanding on previous answers, a macro that verifies that an exception of a given type was thrown and the message of which starts with the provided string.

The test fails if either no exception is thrown, the exception type is wrong or if the message doesn't start with the provided string.

#define ASSERT_THROWS_STARTS_WITH(expr, exc, msg) \
    try\
    {\
            (expr);\
            FAIL() << "Exception not thrown";\
    }\
    catch (const exc& ex)\
    {\
            EXPECT_THAT(ex.what(), StartsWith(std::string(msg)));\
    }\
    catch(...)\
    {\
            FAIL() << "Unexpected exception";\
    }

Usage example:

ASSERT_THROWS_STARTS_WITH(foo(-2), std::invalid_argument, "Bad argument: -2");

I would suggest to use

EXPECT_THROW(function_that_will_throw(), exception);

If your function is returning something void it:

EXPECT_THROW((void)function_that_will_throw(), exception);

The above answers were helpful, but I wanted to share a solution I am using that keeps each test short while still testing both the exception type and the 'what' value. It works for any function that throws an exception that is derived from std::exception, but could be modified (or templated) to catch other types if needed.

I originally tried using a perfect forwarding type wrapper function, but ended up just using a lambda as a wrapper. Using a lambda is really flexible for re-use, allows all the expected implicit type conversions that happen when calling functions directly, avoids passing pointer-to-member functions, etc..

The important point is to have the wrapper template function throw a custom exception for a 'what-mismatch' that has a type that would not be thrown from the function being tested. This causes EXPECT_THROW to print a nice error about the mismatch. I derived from std::runtime_error since that class has a constructor that accepts a std::string.

class WhatMismatch : public std::runtime_error {
public:
  WhatMismatch(const std::string& expectedWhat, const std::exception& e)
    : std::runtime_error(std::string("expected: '") + expectedWhat +
                         "', actual: '" + e.what() + '\'') {}
};

template<typename F> auto call(const F& f, const std::string& expectedWhat) {
  try {
    return f();
  } catch (const std::exception& e) {
    if (expectedWhat != e.what()) throw WhatMismatch(expectedWhat, e);
    throw;
  }
}

A test for a function foo that should throw a std::domain_error would look like:

EXPECT_THROW(call([] { foo(); }, "some error message"), std::domain_error);

If the function takes parameters, just capture them in the lambda like this:

EXPECT_THROW(call([p1] { foo(p1); }, "some error message"), std::domain_error);

You can try Boost lightweight test:

#include <boost/detail/lightweight_test.hpp>
#include <stdexcept>

void function_that_would_throw(int x)
{
  if (x > 0) {
    throw std::runtime_error("throw!");
  }
}

int main() {
 BOOST_TEST_THROWS(function_that_would_throw(10), std::runtime_error);
 boost::report_errors();
}
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top