https://stackoverflow.com/questions/16901453
italiano
english
français
española
中国
日本の
العربية
Deutsch
한국어
Português
RussianWell, I had a similar problem. I wanted to report some extra data/reports/counters from my tests in the final test result like Visual Studio does and I found a solution.
First, this cannot be done with the way you are trying. There is no direct link between the Load Test and the Unit Test where the TestContext exists.
Second, you have to understand how visual studio creates the reports. It collects data from the performance counters of the OS. You can edit these counters, remove those you don't want and add others you want.
How to edit the counters
The load test configuration has two basic sections regarding the counters. These are:
The
Counter Sets. These are sets of counters, for exampleagentwhich is added by default. If you open this counter set you will see that it collects counters such as Memory, Processor, PhysicalDisk e.t.c. So, at the end of the test you can see all these data from all your agents. If you you want to add more counters to this counter set you can double click on it (from the load test editor, see picture below) and selectAdd Counters. This will open a window with all the counters of your system and select those you want.The
Counter Set Mappings. Here you associate the counters sets with your machines. By default the[CONTROLLER MACHINE]and[AGENT MACHINES]are added with some default counter sets. This means that all the counters contained in the counter sets which are mapped to the[CONTROLLER MACHINE]will be gathered from your controller machine. The same applies for all your agents.
You can add more counters sets and more machines. By right clicking on the Counter Set Mappings --> Manage Counter Sets... a new window opens as below:
As you can see, I have added an extra machine with name db_1. This is the computer name of the machine and it must be at the same domain with the controller in order to have access to it and collect counters. I have also tagged it as database server and selected the sql counter set (default for sql counters but you can edit it and add any counter you want). Now every time this load test is executed, the controller will go to a machine with computer name db_1 and collect data which will be reported at the final test results.
Now the coding part
Ok, after this (big) introduction it's time to see how to add your data into the final test results. In order to do this you must create your own custom performance counters. This means that a new Performance Counter Category must be created in the machines you need to collect these data. In your case, in all of your agents because this is where the UnitTests are executed.
After you have created the counters in the agents, you can edit the Agents counter set as shown above and select your extra custom counters.
Here is a sample code on how to do this.
First create the performance counters to all your agents. Run this code only once on every agent machine (or you can add it in a load test plugin):
void CreateCounter()
{
if (PerformanceCounterCategory.Exists("MyCounters"))
{
PerformanceCounterCategory.Delete("MyCounters");
}
//Create the Counters collection and add your custom counters
CounterCreationDataCollection counters = new CounterCreationDataCollection();
// The name of the counter is Delay
counters.Add(new CounterCreationData("Delay", "Keeps the actual delay", PerformanceCounterType.AverageCount64));
// .... Add the rest counters
// Create the custom counter category
PerformanceCounterCategory.Create("MyCounters", "Custom Performance Counters", PerformanceCounterCategoryType.MultiInstance, counters);
}
And here the code of your test:
[TestClass]
public class UnitTest1
{
PerformanceCounter OverallDelay;
PerformanceCounter PerDelay;
[ClassInitialize]
public static void ClassInitialize(TestContext TestContext)
{
// Create the instances of the counters for the current test
// Initialize it here so it will created only once for this test class
OverallDelay= new PerformanceCounter("MyCounters", "Delay", "Overall", false));
PerDelay= new PerformanceCounter("MyCounters", "Delay", "Per", false));
// .... Add the rest counters instances
}
[ClassCleanup]
public void CleanUp()
{
// Reset the counters and remove the counter instances
OverallDelay.RawValue = 0;
OverallDelay.EndInit();
OverallDelay.RemoveInstance();
OverallDelay.Dispose();
PerDelay.RawValue = 0;
PerDelay.EndInit();
PerDelay.RemoveInstance();
PerDelay.Dispose();
}
[TestMethod]
public void TestMethod1()
{
// Use stopwatch to keep track of the the delay
Stopwatch overall = new Stopwatch();
Stopwatch per = new Stopwatch();
overall.Start();
for (int i = 0; i < 5; i++)
{
per.Start();
doAction();
per.Stop();
// Update the "Per" instance of the "Delay" counter for each doAction on every test
PerDelay.Incerement(per.ElapsedMilliseconds);
Sleep(1000);
per.Reset();
}
overall.Stop();
// Update the "Overall" instance of the "Delay" counter on every test
OverallDelay.Incerement(overall.ElapsedMilliseconds);
}
}
Now, when your tests are executed, they will report to the counter their data. At the end of the load test you will be able to see the counter in every agent machine and add it to the graphs. It will be reported with MIN, MAX and AVG values.
Conclusion
- I think (after months of research) that this is the only way to add custom data from your tests to the final load test report.
- It may seems too hard to do it. Well, If you understand the point it's not difficult to optimize it. I have wrap this functionality in a class to be easier to initialize, update and after all to manage the counters.
- It is very very useful. I can now see statistics from my tests that it would not be possible with the default counters. Such us, when a web request to a web service fails, I can catch the error and update the appropriate counter (e.g. Timeout, ServiceUnavailable, RequestRejected...).
I hope I helped. :)
