Matlab moving from XUnit to Matlab 2013 unit testing

Question

As many of you are aware as of the release of MatLab 2013a, xUnit a popular unit testing framework for MatLab is canceling further development.

Is MatLab's new and native unit testing framework comparable to xUnit? what features is it lacking when compared to xUnit? Is it better or worse than xUnit?

Answer 1

MATLAB xUnit has been an excellent contribution to the test focused development efforts of those writing MATLAB code. It has a solid implementation, it follows the xUnit paradigm very well, and has been invaluable as a file exchange contribution.

The MATLAB Unit Test framework has indeed learned from this submission as well as decades of requirements and test focused development for the MathWorks' internal code base. We have also learned and extended upon frameworks in other languages such as JUnit, NUnit, and python's unittest framework. As such there certainly are many more features in the R2013a-beyond framework, and it is designed to scale and extend.

There are too many other features to go into in a simple answer, but perhaps one way to describe some of the differences are that the 13a framework is what I loosely call an "xUnit 2.0" and the file exchange submissions is an "xUnit 1.0" framework. If you are familair with JUnit, this is like the difference between JUnit 3 and JUnit 4.

There are also other intangible or as yet unrealized benefits, such as:

1. The framework is included directly in MATLAB so you can share tests with others and know that they can run the tests even if they are not familiar with testing and do not want to download the file exchange framework.
2. The framework is under active development with a pipeline of additional features and capabilities in the works for future releases.

Hope that helps. I would be happy to go over any questions you have about specific functionality or features.

Answer 2

I don't believe MathWorks are planning at all to stop making xUnit available, so you can continue using it if you like. xUnit had not seen any large changes for quite a while in any case, and even though it won't be developed further in terms of features, it may receive an occasional fix if any are needed.

I have tried out the new framework quite a bit, but have not used it on any large projects yet. Previously I have used xUnit on large projects. However, I'm no expert on unit testing - so please read the following opinions in that context.

I'm pretty sure there's nothing you can do in xUnit that you can't do in the new framework. In general it's much more flexible and powerful than xUnit, providing additional features and a better way to organise and structure your tests. It's a lot easier to set up and tear down suites of tests, to manage and close resources (files, figure windows, database connections etc), and to carry out tricky tests such as checking that the right number of arguments are returned.

However, whereas a typical xUnit test was implemented as a fairly simple MATLAB function, tests in the new framework are typically implemented (in 13a, but see below for 13b) as classes using MATLAB's OO syntax, and if you're not comfortable with that it may seem like a big leap.

I should also add that although the documentation for the testing framework is excellent as reference material, I haven't found it to be great as a tutorial.

In 13b, the need to use classes has been offset a bit with the introduction of the functiontests command, which creates a test suite for you from a file containing tests implemented as local functions. That will make things much easier if you're not comfortable with class syntax. But I would think that if you want to take advantage of everything, you'd probably still want to use the main framework.

Hope my experience is of help - if you're lucky, perhaps @AndyCampbell will chime in...