Why are scenario outlines shown as 'External' in the VS 2012 Test Explorer?

Question

I have a scenario outline (a.k.a scenario template) and I wish to list my tests using the VS 2012's Test Explorer. The default test grouping is done by the "Project" name - if I stick to it, my tests coming from scenario outlines are in the group "External" and not under the group of my project.

Why is this and is there a solution/workaround to it?

My scenario is that I'd like to run all of my tests for a given project using the Test Explorer. Setting the same trait for all of them is silly and prone to human mistakes.

My solution has 10 such projects with the idea of more being added to it soon. Thus keeping a project's scenario outlined tests under "External" may cause anyone to miss them or to grab a scenario outline-generated test that belongs to another project.

UPDATE

GitHub user Shashi (https://github.com/shashiprabhakar) has logged an issue to the SpecFlow contributors: https://github.com/techtalk/SpecFlow/issues/275. Thanks a lot, mate! I've started tracking it so if a fix/workaround is proposed there, I'll post it as an answer here.

Answer 1

<specFlow>
  <unitTestProvider name="..." /> <!-- this you have already -->
  <generator allowDebugGeneratedFiles="true" /> <!-- this is the important setting -->
</specFlow>

This is the configuration Gaspar Nagy has recommended in the App.config file. The "allowDebugGeneratedFiles" seems to be the one ticket. More info on the tag and it's sub-tags can be found on the official SpecFlow configurations page. Setting it to TRUE is only the first step though.

As Gaspar suggests, the next step is to right-click on the project file in the Solution Explorer and to select the Regenerate Feature Files option.

NB: What actually happens when you set the allowDebugGeneratedFiles flag and then regenerate the .feature files, SpecFlow simply removes the #line hidden and #line number from the .feature.cs file.

Finally, I'd personally suggest to restart Visual Studio 2012 before or after rebuilding. When I tried the above steps, the Test Explorer showed the tests correctly only after restarting the Visual Studio.

PS: Gaspar noted that if the allowDebugGeneratedFiles flag is raised (=true), then using the "Run SpecFlow Scenarios" option won't work anymore. However, I was able to use that option by right-clicking on the project file. Using that option on a .feature wasn't working for me even before I tried Gaspar's suggestion so that must be the drawback that he was referring to.

PS #2: Gaspar also mentions that there won't be such a drawback will be removed in SpecFlow 1.9.2.

Answer 2

As Gaspar (https://github.com/gasparnagy) suggested in a comment to the GitHub issue (https://github.com/techtalk/SpecFlow/issues/275), right-clicking on the project file in the Solution Explorer and then selecting "Run SpecFlow Scenarios" is an excellent workaround for the Project grouping of the Test Explorer.

I can verify that the proposition is valid - tested it with SpecFlow 1.9.0.

However, that still leaves the problem with the Test Explorer and its weird "External" group of tests where the scenario-outline-generated tests are put.

I'll update this answer or post a new one if more details come to light.