Visual studio 2010: Nunit test case fails in debug mode but passes in run mode

Question

I'm having a weird problem with Visual Studio 2010 Ultimate: One of my Nunit(2.6.2) test cases is failing in the debug mode but passing in the run mode, as if we had completely different code paths for the two scenarios.

Is this a known bug? or is there some option in vs I'm missing?

Please enlighten me! Thanks a lot.

EDIT - MORE INFORMATION My application submits some requests to a dll written by a group of people within the organization. The dll does some computation and returns the results back to me.

In a test case exploring the dll's behavior (e.g. submit a request having certain parameter, check the dll's output), running the Nunit test works fine, but debugging the test case gives me an error - an exception was thrown from within that dll.

IMPORTANT: Running/debugging another test case gives me consistent results.

So, for the weird test case: 1, either the dll is good, and something under the debug mode breaks the dll 2, or, the dll has a bug, which is triggered by something under the debug mode.

To my knowledge, the ONLY difference between running and debugging a piece of code in visual studio is that, when debugging, a pdb file is loaded while when running it's not. Essentially, a symbol table is loaded to identify code execution.

Then the issue doesn't make sense in the first place - why would loading a symbol table affect the dll's behavior? (It's unfair to ask anyone to give an explanation without seeing any code; however, since it's corporate prop. code, I can't show it here. Please, if you've ever encountered such things in your career, do share with me what happened in your case - let's hope my problem has the same cause so that I can actually know what went wrong. Thanks)

Answer 1

Thank you for your response. I've identified the reason: it's due to a false parameter which drives the dll nuts. My bad. It still doesn't answer the question why the behavior at debug time goes crazy, but good while running the test case.

However, I guess, since the parameter is wrong in the first place, I can't really blame the dll for going nuts. Anyway, when I passed in the correct param, all went good.

Thanks a lot guys.

Answer 2

Are you using code coverage? If so, try disabling it and run. It will probably work.

For more details, check: http://social.msdn.microsoft.com/Forums/en-US/aba3d58f-f19f-4742-b960-8ac2be29bb88/unit-test-passes-when-in-debug-but-fails-when-run

Answer 3

You may have run into a situation where you're taking the same code path, but the results are subtly different in debug vs non-debug due to optimizations. There are a couple of distinct possibilities here:

• Your code has a subtle bug, e.g. a race condition
• Your test is being overly specific, e.g. for floating point comparisons where you should use a tolerance

It's a pain not being able to debug, but I suggest you add logging throughout the method and the test so you can see what's going on. (And hope that the logging itself doesn't change the test result, which is also possible...)