Error trying to run mstest on jenkins

Question

I´m trying to configure MSTest build plugin on jenkins, but I´m getting the following error:

Path To MSTest.exe: mstest.exe
Result file was not found so no action has been taken. file:/C:/Program%20Files%20(x86)/Jenkins/jobs/SoftwrenchvNext/workspace/TestResult.trx
FATAL: null
java.lang.NullPointerException
    at org.jenkinsci.plugins.MsTestBuilder.perform(MsTestBuilder.java:144)

The configuration simply specifies TestResult.trx as ResultFileName. This file is not versioned, and I expect it to be created on each build.

What needs to be done for that?

Answer 1

I too had this exact same error message!

My recommendation is to replace the "Run unit tests with MSTest" step. with an "Execute Windows batch command" step. This worked for me.

Command

del TestResults.trx
"C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\MSTest.exe" /testcontainer:Example.Tests\bin\Debug\Example.Tests.dll /resultsfile:TestResults.trx

Using this technique you may still use the "Publish MSTest test result report" step specifying...

Test report TRX file

TestResults.trx

Good luck!

Also, you can "simulate" the "Continue on failed tests" functionality with an "EXIT" call, seen below.

del TestResults.trx
"C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\MSTest.exe" /testcontainer:Example.Tests\bin\Debug\Example.Tests.dll /resultsfile:TestResults.trx /nologo
EXIT /B 0

Answer 2

After some investigation I was able to get MSTest to work in Jenkins by doing the following:

Select Manage Jenkins

From within the Manage Jenkins choose Configure System

In Configure System you will need to find MSTest and Add your MSTest configuration

Enter your configuration as you see fit. Mine appears as follows:

Save the configuration and then go to your build project and configure the appropriate items like shown below:

One thing to note is that I did not follow any of the steps listed on the author's site because they didn't seem to make any sense to me. I did however read the code and spent some time digging and realized that the missing piece was in the Configure System page of Jenkins.

Cheers!

Answer 3

In the end I gave up on configuring the MSTest through the easy panels - it would not find the test dll, so like the guy above I used the batch command. I have added my batch command below because it addresses some issues with previous answers that they do not take into account that you have to delete the .trx file before each run, or give it a unique name or tests will fail.

Here is my solution running in Jenkins as a batch command:

"C:Program Files (x86)/Microsoft Visual Studio 12.0/Common7/IDE/MSTest.exe" /resultsfile:"%WORKSPACE%/MyAppTest_%BUILD_NUMBER%.trx" /testcontainer:"%WORKSPACE%/Components/ebox2/Central/MyApp/MyAppWeb/trunk/MyAppTest/bin/Release/MyAppTest.dll" /nologo /category:Build

Note the inclusion of %BUILD_NUMBER% in the .trx file name, which gets round the problem of jobs failing because of duplicate .trx file.

Note also that /category:Build allows you to choose the tests you want. You set up the category in the tests themselves:

[TestCategory("Build"), TestMethod()] 

I use this because I have some Selenium tests which I have not figured out how to run in Jenkins yet, or if this is possible.

Answer 4

Another thing to notice: This does not work with .NetCore, just with the regular .Net Framework: (See https://github.com/Microsoft/vstest/issues/1213#issuecomment-338561792 and https://developercommunity.visualstudio.com/content/problem/238572/mstest-command-line-reports-no-tests-to-run.html)

I used a batch job in Jenkins with something like the following code:

dotnet test src\Folder\Project.csproj --logger:trx

Please find the full reference for dotnet test here: https://docs.microsoft.com/de-de/dotnet/core/tools/dotnet-test?tabs=netcore21