Is it good practice to use C++ hierarchical namespace resolution to provide mock components for unit tests?

Question

A typical use case would be a component (e.g. testable::MyFolder in the example below) that uses boost::filesystem in its implementation. Unit-testing that component would require mocking parts of boost::filesystem. One approach for mocking boost::filesystem would be to implement the mock components inside a namespace that also contains MyFolder (e.g. inside the testable namespace in the example) and rely on the hierarchical namespace resolution to replace the boost::filesystem components with their mock counterparts at compile time.

For instance:

in file MyFolder.hh:

#include <boost/filesystem.hpp>
#include <boost/exception/all.hpp>

namespace testable
{

  struct SomeError: public std::exception {};

  struct MyFolder
  {
    MyFolder(const boost::filesystem::path &p)
    {
      if (!exists(p)) // must be resolved by ADL for unit-tests
      {
        BOOST_THROW_EXCEPTION(SomeError());
      }
    }
  };

} // namespace testable

in file MockFilesystem.hh:

#include <string>

namespace testable
{
  namespace boost
  {
    namespace filesystem
    {
      struct path
      {
        path(const std::wstring &) {}
      };

      bool exists(const path&)
      {
        return false;
      }

    } // namespace filesystem
  } // namespace boost
} // namespace testable

in file testMyFolder.cpp:

#include "MockFilesystem.hh" // provides boost::filesystem mocks for MyFolder
#include "MyFolder.hh"

#include <cppunit/ui/text/TestRunner.h>
#include <cppunit/extensions/HelperMacros.h>

class TestMyFolder : public CppUnit::TestFixture
{
  CPPUNIT_TEST_SUITE( TestMyFolder );
  CPPUNIT_TEST( testConstructor );
  CPPUNIT_TEST_SUITE_END();
private:
public:
  void setUp() {}
  void tearDown() {}
  void testConstructor();
};

const std::wstring UNUSED_PATH = L"";

void TestMyFolder::testConstructor()
{
  CPPUNIT_ASSERT_THROW(testable::MyFolder(testable::boost::filesystem::path(UNUSED_PATH)), testable::SomeError);
}

int main()
{
  CppUnit::TextUi::TestRunner runner;
  runner.addTest( TestMyFolder::suite() );
  runner.run();
}

The specific questions about this approach are:

• Are there any good reasons for not doing that?
• What are the most common pitfalls with this approach?
• What are the alternatives?
• In what context is this solution better or worse than the alternatives?
• How to improve the class MyFolder to make it easier to unit-test?

Answer 1

Unit testing is nothing more than executing a high-level component (the unit under test) with an alternative implementation of the low-level component (the mock component). From this point of view, any SOLID approach to decoupling high- and low-level components is acceptable. It is however important to notice that with unit testing the selection of the mock components is done at compile-time, as opposed to run-time patterns like Plugins, Service Locators, Dependency Injection, etc.

There are many different interface mechanisms to lower the coupling between high- and low-level components. In addition to language agnostic-ish approaches (hacks, compiler command line options, library paths, etc.), C++ offers several options including virtual methods, templates, namespace resolution and argument-dependent lookup (ADL). In this context, virtual methods can be seen as run-time polymorphism while templates, namespace resolution and ADL can be seen as compile-time flavors of polymorphism. All of the above can work for unit testing, from ed scripts to templates.

When the selection of low-level components is done at compile-time, I personally prefer using namespaces and ADL instead of interface classes with virtual methods to save the (some would argue minimal) overhead of defining the virtual interface and wiring the low level components into that interface. In fact, I would question the sanity of accessing any STL or boot component through a home-made virtual interface without a compelling reason. I am bringing up this example because a significant proportion of unit tests should test the behavior of high-level components when low-level STL or boost components meet specific conditions (memory allocation failure, index out of bound, io conditions, etc.). Assuming that you are systematic, strict and rigorous in your unit tests, and assuming that you always use abstract virtual classes as a mechanism for substituting mocks, then you would need to replace every single instance of a std::vector by a home made IVector, everywhere in your code.

Now, even though it is important to be strict and rigorous with unit testing, being systematic might be perceived as counterproductive: in most cases a std::vector will be used to implement a high-end component without any reason to be concerned about failing memory allocations. But what happens if you decide to start using your high-end component in a context where memory allocation is becoming a concern? Would you prefer modifying the code of the high-end component, replacing std::vector with a home made IVector for the sole purpose of adding the relevant unit test? Or would you prefer to add the missing unit test transparently - using namespace resolution and ADL - without changing any code in the high-level component?

Another important question is the number of different approaches you are willing to support for unit testing in your project. 1 seems like a good number, particularly if you decide to automate the discovery, compilation and execution of the unit tests.

If the previous questions led you to consider using namespaces and ADL, it is time to look into the possible limitations, difficulties and pitfalls before committing to a final decision. Let's use an example:

File MyFolder.hh

#ifndef ENCLOSING_MY_FOLDER_HH
#define ENCLOSING_MY_FOLDER_HH
#include <boost/filesystem.hpp>
#include <boost/exception/all.hpp>
namespace enclosing {
struct SomeError: public std::exception {};
struct MyFolder {
    MyFolder(const boost::filesystem::path &p);
};
} // namespace enclosing
#endif // #ifndef ENCLOSING_MY_FOLDER_HH

In file MyFolder.cpp:

#include "MyFolder.hh"
namespace enclosing {
MyFolder::MyFolder(const boost::filesystem::path &p) {
    if (!exists(p)) // must be resolved by ADL for unit-tests {
        BOOST_THROW_EXCEPTION(SomeError());
    }
}
} // namespace enclosing

If I want to test MyFolder constructor for the 2 obvious use cases, my unit test will look like this:

testMyFolder.cpp

#include "MocksForMyFolder.hh" // Has to be before include "MyFolder.hh"
#include "MyFolder.hh"
#include <cppunit/ui/text/TestRunner.h>
#include <cppunit/extensions/HelperMacros.h>
namespace enclosing {
class TestMyFolder : public CppUnit::TestFixture {
    CPPUNIT_TEST_SUITE( TestMyFolder );
    CPPUNIT_TEST( testConstructorForMissingPath );
    CPPUNIT_TEST( testConstructorForExistingPath );
    CPPUNIT_TEST_SUITE_END();
public:
    void setUp() {}
    void tearDown() {}
    void testConstructorForMissingPath();
    void testConstructorForExistingPath();
};
const std::wstring UNUSED_PATH = L"";
void TestMyFolder::testConstructorForMissingPath() {
    CPPUNIT_ASSERT_THROW(MyFolder(boost::filesystem::missing_path(UNUSED_PATH)), SomeError);
}
void TestMyFolder::testConstructorForExistingPath() {
    CPPUNIT_ASSERT_NO_THROW(MyFolder(boost::filesystem::existing_path(UNUSED_PATH)));
}
} // namespace enclosing
int main() {
    CppUnit::TextUi::TestRunner runner;
    runner.addTest( enclosing::TestMyFolder::suite() );
    runner.run();
}

With the mock paths implemented in MocksForMyFolder.hh:

#include <string>
namespace enclosing {
namespace boost {
namespace filesystem {
namespace MocksForMyFolder { // prevent name collision between compilation units
struct path {
    path(const std::wstring &) {}
    virtual bool exists() const = 0;
};
struct existing_path: public path {
    existing_path(const std::wstring &p) : path{p} {}
    bool exists() const {return true;}
};
struct missing_path: public path {
    missing_path(const std::wstring &p) : path{p} {}
    bool exists() const {return false;}
};
    inline bool exists(const path& p) {
        return p.exists();
    }
} // namespace MocksForMyFolder
using MocksForMyFolder::path;
using MocksForMyFolder::missing_path;
using MocksForMyFolder::existing_path;
using MocksForMyFolder::exists;
} // namespace filesystem
} // namespace boost
} // namespace enclosing

Finally, a wrapper is needed to compile MyFolder implementation with the mocks, WrapperForMyFolder.cpp:

#include "MocksForMyFolder.hh"
#include "MyFolder.cpp"

The main pitfall is that unit tests in different compilation units might implement mocks of the same low-level components (e.g. boost::filesystem::path) inside enclosing namespaces (e.g. enclosing::boost::filesystem::path). When linking all the unit tests with the test runner into a single test suite, depending on the situation the linker will either complain about the collision or, much worse, silently and arbitrarily select one of the implementations. The workaround is to enclose the implementation of the mock components in an inner unnamed namespace - or in a uniquely named namespace (e.g. namespace MocksForMyFolder) and then expose them with appropriate using clauses (e.g. using MocksForMyFolder::path).

This example shows that there are options to implement unit tests with configurable mocks (missing_path and existing_path). The same method would also enable deep testing of inner and hidden implementation aspects (e.g. private class members or inner implementation details of a method) but with significant limitations - which is probably a good thing.

When sticking to a strict definition of unit testing where the unit under test is a single compilation unit, things tend to stay reasonably simple as long as the design is reasonably SOLID: the single high-level component implemented in the compilation unit will include a small number of headers, each of them being a dependency to low-level components. When these dependencies are implemented in other compilation units, they are good candidates for mock implementations and that's where the header guards play a critical role.

With appropriate naming conventions, automation is trivial with just a few makefile recipes.

So, my personal summary is that namespace resolution and ADL:

• provide some forms of compile-time polymorphism well suited for unit test
• don't add anything to the interface or implementation of the high-level components
• are a very easy and convenient to implement mocks for low-level components like boost and STL
• can be used for any user-implemented lower level dependency

Some aspects that might be perceived as bad (or good) things:

• require careful encapsulation of the mocks to avoid namespace pollution
• require consistent and systematic header guards

I believe that important reasons for not using this method for unit tests would be legacy and personal preferences.

Answer 2

1. Instead of checking functionality of the MyFolder, you're also checking composition of 2 interfaces: public interface of MyFolder and public interface of boost::MyFolder As result you got more complex and brittle tests
2. brittle tests(at least from my experience) Your test depends on absence of the "/tmp" folder, which can be created at any moment
3. Using either simple types, or interfaces. Both categories can be easily mocked for testing needs.
4. I don't see any benefits from using your approach
5. see #3

As an example

class IPath
{
    virtual bool exists() const = 0;
}

struct MyFolder
{
    MyFolder(const IPath &p)
    {
        if (!p.exists()) // must be resolved by ADL for unit-tests
        {
            throw exception;
        }
    }
};

//TEST CODE
class CMockPath: public IPath
{
    CMockPath(string s) {};
    virtual bool exists() const { return false};
};
const wstring UNUSED_PATH = L"";

void TestMyFolder::testConstructor()
{
    CPPUNIT_ASSERT_THROW(CMockPath(UNUSED_PATH), testable::SomeError);
}

//PDN CODE
class CPath: public IPath
{
    ...
    boost::filesystem::path _p;
    bool exists() const { return _p.exists(); };
    ...
};

CPath path(L".....");
MyFolder folder(path);