Can I exclude counts from failed webtests in a VS.Net 2010 Loadtest?

Question

I am using Visual Studio 2010 Ultimate to perform loadtests. These loadtest use recorded webtests.

When running a loadtest with an increasing number of concurrent users, some steps in my webtests will start to fail. The first error is often an internal server error 500. This will give a wrong impression of the average page_load, because these internal server errors are often returned very fast, in contrast to the generation a succesful response. So, when the load increases, the average page_load drops.

Of course, I need to attend to these internal server errors, but in the meantime, I would like to exclude failed webtests from my measurements.

Does anybody know if this can be done?

Thanks in advance.

Answer 1

It may be possible to run your own query on the test results database that ignores errors, but even that will be inaccurate.

Remember that the page return stats are really only useful when read in conjunction with the load on the hardware.

Essentially, the load test is recording the effect on your hardware of a given load. If you website is returning a large number of 500 error pages quickly, the load on the hardware will be affected and any page stats will reflect the change in server loading.

You will have to investigate the cause of the 500 errors and either fix the issue or report in your load testing results that once a load of 'x' is reached on the servers, the pages 'y' will give an internal server error 500 result instead of the requested page.

This gives the business owners of you app some information to make the decision whether to fix the problem or live with it.