CppUnit - advantages and disadvantages with the xUnit based design?

Question

CppUnit is designed to be similar to JUnit and other XUnit frameworks.

What are the advantages/disadvantages of this? When should it be selected for this reason?
And should a different framework be preferred if the similarity has no advantages. Is actual use when you get started easy and intuitive?

---

To start with advantages. Does it plat better with some tools? Tools for continuous integration for example?

Is it better in a mixed environment (Like C# and C++)?

Disadvantages: I know it is slightly more complex to build, and thus less cross-platform than other frameworks. Personally I have the impression it is easy (for a beginner) to get stuck in compile and link errors, it seems difficult to follow the actual code (In a debugger for example), and also it is not so natural to use with functions (C style interfaces).

Answer 1

Advantages:

The primary advantage of CppUnit is that it follows the XUnit patterns of Setup / Test / Teardown. The paradigm is understood by anyone with XUnit experience. Having to use macros instead of attributes or reflection (as you do with .Net or Java XUnit frameworks) is a bit of a workaround, but not really burdensome.

And CppUnit itself is portable. Although the TestRunner.DLL GUI is built entirely in MFC and is therefore Windows only, the command line version uses no Microsoft specific code, and executes on many platforms.

It's also very stable. The developer has done almost nothing with it for many years, mostly because its core needs nothing. It does its one thing very well.

Disadvantages:

The disadvantages are unfortunately that end user support is lacking. My biggest complaint is that there are no good tools that integrate CppUnit into an IDE. The reason I consider this most severe is that testing requires discipline. Anything that makes unit testing harder than it has to be will be used as an excuse to not do it.

You can manually add a format to output a test failure line that Visual Studio will properly interpret (so double clicking on it will take you to the failing ASSERTion in the test.) But this doesn't ship with the tool, you have to learn it externally on sites like this one.

You can write an IDE plugin that will generate unit test skeleton code for you, but I am aware of no such plugins being made publicly available. Again, this is something that needs to be easy for developers.

Because there is no integration, there is also no published guidance on incorporating a CppUnit project into your codebase. That makes it very hard to get started.

If you do manage to get it working, several people report problems with phantom CppUnit project check-in / check-out problems.

CppUnit doesn't support any particular mocking framework. As most mock frameworks use reflection to make developing mocks painless, C++ is always going to suffer a bit there.

Alternatives:

There are many, many C++ unit testing frameworks out there. Googletest gets good reviews from some people. CppUnitLite is praised by some, but I found it very confusing and unhelpful. CxxUnit requires a clumsy Python scanning and parsing step - a less elegant weapon for a less civilized age. And there are many commercial players in this space, offering their wares.

I'd recommend http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks#C.2B.2B for a very rich list of alternatives.

While I have not yet had the chance to use it, I understand that Visual Studio 2012 has incorporated Microsoft's own test suite tooling and made it available for native C++ code. Previously, it only worked through .Net integration, which was useless for most C++ code. I'm really looking forward to having a full suite of tools, including their code coverage integration, where it highlights your tested code, showing you gaps in your test coverage.

Bottom line: if I were starting over today, I would choose a different XUnit-compatible test framework, one that was fully integrated with the IDE. Back when we chose CppUnit, it was the best game in town for Visual Studio 6.0, but now we're just living with it.

Answer 2

Disadvantage:

While doing my research about CppUnit, I discovered for CppUnit Test, the path of installation is fixed (/usr/local) and cannot be modified. So this will be a problem when you will try to integrate this framework on your work machine where you don't have admin rights

MYMACHINE:cppunit-1.12.1 usrId$ make install
Making install in src
Making install in cppunit
test -z "/usr/local/lib" || ../../config/install-sh -c -d "/usr/local/lib"
 /bin/sh ../../libtool   --mode=install /usr/bin/install -c  'libcppunit.la' '/usr/local/lib/libcppunit.la'
/usr/bin/install -c .libs/libcppunit-1.12.1.0.0.dylib /usr/local/lib/libcppunit-1.12.1.0.0.dylib
install: /usr/local/lib/libcppunit-1.12.1.0.0.dylib: Permission denied
make[3]: *** [install-libLTLIBRARIES] Error 71
make[2]: *** [install-am] Error 2
make[1]: *** [install-recursive] Error 1
make: *** [install-recursive] Error 1
MYMACHINE:cppunit-1.12.1 usrId$

Even after changing multiple makefile, result is same

MYMACHINE:cppunit-1.12.1 usrId$ make install
test -z "/Users/username/lib" || ../../config/install-sh -c -d "/Users/username/lib"
 /bin/sh ../../libtool   --mode=install /usr/bin/install -c  'libcppunit.la' '/Users/username/lib/libcppunit.la'
/usr/bin/install -c .libs/libcppunit-1.12.1.0.0.dylib /Users/username/lib/libcppunit-1.12.1.0.0.dylib
(cd /Users/username/lib && { ln -s -f libcppunit-1.12.1.0.0.dylib libcppunit-1.12.1.dylib || { rm -f libcppunit-1.12.1.dylib && ln -s libcppunit-1.12.1.0.0.dylib libcppunit-1.12.1.dylib; }; })
(cd /Users/username/lib && { ln -s -f libcppunit-1.12.1.0.0.dylib libcppunit.dylib || { rm -f libcppunit.dylib && ln -s libcppunit-1.12.1.0.0.dylib libcppunit.dylib; }; })
/usr/bin/install -c .libs/libcppunit.lai /Users/username/lib/libcppunit.la
/usr/bin/install -c .libs/libcppunit.a /Users/username/lib/libcppunit.a
chmod 644 /Users/username/lib/libcppunit.a
ranlib /Users/username/lib/libcppunit.a
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: /Users/username/lib/libcppunit.a(BeOsDynamicLibraryManager.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: /Users/username/lib/libcppunit.a(ShlDynamicLibraryManager.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: /Users/username/lib/libcppunit.a(Win32DynamicLibraryManager.o) has no symbols
libtool: install: warning: remember to run `libtool --finish /usr/local/lib'
make[1]: Nothing to be done for `install-data-am'.