Why the practice of writing unit tests in a different language isn't that popular?

Question

When Microsoft released Visual Studio 2008, there was a thing they were talking a lot about at the conferences and in their online tutorials: the idea of writing the actual code in one language, and the unit tests in a different language. For instance, unit tests for C# code can be written in Visual Basic.

Six years later, I don't know any open source project which uses this practice and no proprietary project I've seen uses it. The subject is not discussed any longer at Microsoft's conferences, blogs or tutorials.

The benefit of this practice is that doing the same type of error in two different languages is more difficult. For instance, if I don't know that in C#, 10/3 equals 3 and expect it to be 3.333..., a unit test written in Python or JavaScript will fail and lead me to the discovery in integer division.

.NET Framework, specifically, makes it possible to use any .NET-enabled language to test another one. While testing C# code with Visual Basic tests doesn't look particularly interesting, given the similarity of both languages, I would imagine that, for example, testing F# code with unit tests written in C# could be useful for people who are not that familiar with functional programming.

Personally, I have no experience writing unit tests in a different language. Therefore, I don't know if there are actual drawbacks in this technique. I often write system and acceptance tests in a different language (for instance in Python for a Node.js web app, because Python's Requests library is so great), but this doesn't count, since those tests are truly language-agnostic: they just do black box testing through HTTP: I can rewrite an app from Node.js to ASP.NET MVC, and the tests won't even notice the change.

So what could be the drawbacks, and why is this practice not that popular?

Answer 1

Unit tests are white-box tests and usually written by the developers of the thing being tested. The first means that you need first-class interaction with the APIs and objects and values of the language the actual product is written in, which rules out almost all combinations of languages. In fact, I think the CLR is the only agglomeration of languages that each can call into each other perfectly. And as you say, most CLR languages are so similar that none of the advantages you suggest apply.

That leaves pretty much only the combination of F# and C#/VB.NET. I can imagine several reasons this hasn't taken off:

• F# doesn't have that big of a community. Conversely, most C# developers don't know enough F# to even consider this.
• The people who do know F# are either busy writing both unit tests and actual code in F#, or in a team where they can't use F# at all because nobody else knows it (well). As mentioned before, unit tests are usually not written by a separate team, so people who write C# unit tests for F# code probably still have to know F#. Plus the whole white-box thing: If you don't know a thing about functional programming, even the data structures and APIs exposed by (idiomatic) F# will be unnatural for you once you delve deeper than very simple libraries of the form "put in a URL here, receive IEnumerable<UrlParameter>.
• There's no benefit (or people don't believe the benefit), but quite possibly some very real friction from switching languages constantly.

Answer 2

I do not recollect any such push. I imagine it was a demonstration of interoperability in the .NET space rather than a practical use of different languages for testing. Any benefits would have been made up just to underpin the virtues of this wonderful common language runtime environment.

I guess this could make some sense for library code. If you write and deploy a .NET library that will likely be used from other .NET languages than your own, you may fall into the trap of using features that only work well if both client and server code are the same language, like using language specific types rather than CLR types exclusively.

Answer 3

IMO the reason you don’t see this much anymore is because it was really just another way to put a nail in VB. The idea being promoted wasn’t that it’s good to write unit test in another language, but rather that you could write C# unit test for VB code.

This has several potential advantages. First, if your team doesn’t have a lot of c# experience, it’s a way to introduce it to them. Secondly, it’s a foot in the door for future projects. I.e, all of our code is VB, so the next project will be in VB, becomes, the last project (a unit test project, but hey, its still a project) was C#, so it’s OK to make the next project a C# project.

That was in 2008, by 2015, that just wasn’t necessary. Teams still using VB are going to continue using it for legacy reasons, but most new development will be in c#.