Performance Testing fundamentals

Question

I have some basic questions around understanding fundamentals of Performance testing. I know that under various circumstances we might want to do - Stress Testing - Endurance Testing etc. But my main objective here is to ensure that response time is decent from application under a set of load which is towards a higher end or in least above average load.

My questions are as follows:

1. When you start to plan your expected response time of application; what do you consider. If thats the first step at all. I mean, I have a web application now. Do I just pull out a figure from air and say "I would expect application to take 3 seconds to respond to each request". and then go about figuring out what my application is lacking to get that response time?

2. OR is it the other way round, and you start performance test with a given set of hardware and say, lets see what response time I get now, and then look at results and say, well it's 8 seconds right now, I'd like it to be 3 seconds at max, so lets see how we can optimize it to be 3 seconds? But again is 3 seconds out of air? I am sure, scaling up machines only will not get response time up. It'll get response time up only when single machine/server is under load and you start clustering?

3. Now for one single user I have response time as 3 seconds but as the load increases it goes down exponentially; so where do I draw the line between "I need to optimize code further" (which has it's upper limit) and "I need to scale up my servers" (Which has a limit too)

4. What are the best free tools to do performance and load testing? I have used Jmeter a bit. But is there anything else, that is good and open source?

5. If I have to optimize code, I start profiling the specific flows which took lot of time responding to requests?

Basically I'd like to see how one goes about from end to end doing performance testing for their application. Any links or articles would be very helpful.

Thanks.

Answer 1

The Performance Testing Council is your gateway to freely exchange experiences, knowledge, and practice of performance testing.

Also read Microsoft Patterns & Practises for Performance testing. This guide shows you an end-to-end approach for implementing performance testing.

phoenix mentioned the Open Source tools.

Answer 2

First of all you can read

Best Practices for Speeding Up Your Web Site

For tools

Open source performance testing tools

performance: tools

Answer 3

This link and this show an example and method of performance tuning an application when the application does not have any obvious "bottlenecks". It works most intuitively on individual threads. I have no experience using it on web applications, although other people do. I agree that profiling is not easy, but I've always relied on this technique, and I think it is pretty easy / effective.

Answer 4

First of all, design your application properly.

Use a profiler, see where the bottlenecks in your application are, and take them away if possible. MEASURE performance before improving it.

Answer 5

I will try to provide basic step by step guide, which can be used for implementing Performance testing in you project.

1 - Before you start testing you should know amount of physical memory and amount of memory allocated for JVM, or whatever. DB size collect as much metrics as possible for your current environment. Know you environment

2 - Next step would be to identify common DB production size and expected yearly growth. You will want to test how your application will behave after year, two, five etc.,

3 - Automate environment setup, this is will help you a lot in future for regression testing and defect fix validation. So you need to have DB dumps for your tests. With current (baseline), one year, five year volume.

4 - Once you're done if gathering basic information - Think about monitoring your servers under load, maybe you already have some monitoring solution like http://newrelic.com/ this will help you to identify cause of performance degradation (CPU/Mem/Amount of threads etc.,) Some performance testing tools do have built in monitoring systems.

At this you are ready to move with tooling and load selection, there is already provided materials on how to do that so I will skip part with workload selection.

5 - Select tool I think that JMeter + http://blazemeter.com/ is what you need at this point, both do have a lot nice articles and education materials, for your script recording I would recommend to use blazemeters Chrom Extension instead of inbuilt JMeters solution. If you still think that you do lack knowledge on how things are done in JMeter I recommend to get this book - Performance Testing With JMeter 2.9 by Bayo Erinle

6 - Analyze results, review test plan and take corresponding actions.