Factors limiting concurrent outstanding requests

Question

I am trying to achieve a high number of webrequests per second.

With C#, I used multiple threads to send webrequest and find that no matter how many threads I created, the max number of webrequest is around 70 per second in the condition that a server responds quickly.

I tried to simulate timeout response using fiddler in order to make concurrent outstanding web requests to have a better understanding.

With whatever amount of threads, there are instantly fired 2x requests, afterward, the queued requests fired one by one very slowly although the previous requests were still getting response. Once there were finished requests, the queued requests fired faster to replenish the amount. Its like it takes time to initialize once the pre-initialized amount is reached. Moreover, the response is small enough that bandwidth problem could be neglected.

Below is the code.

I tried in window xp and window 7 in different network. Same thing happens.

public Form1()
{
System.Net.ServicePointManager.DefaultConnectionLimit = 1000;
for (int i = 0; i < 80; i++)
{
    int copy = i;
    new Thread(() =>
    {
        submit_test(copy);
    }) { IsBackground = true }.Start();
}
}
public void submit_test(int pos)
{
    webRequest = (HttpWebRequest)WebRequest.Create("http://www.test.com/");
    webRequest.Method = "GET";
    using (HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse())
    {
    }
}

Is it the network card limiting the instantly fired amount?

I know that a large server can handle thousands of incoming request concurrently. Isn't it the same as sending out requests ( Establishing connection )?

Please tell me if using a server helps solve the problem.

Update clue:

1) I suspect if the router limiting and unplugged it. No difference.

2) Fiddler show that one queued requests fired exactly every second

3) I used apache benchmarking tool to try to send concurrent timeout request and same thing happens.Not likely to be .Net problem.

4) I try to connect to localhost instead. No difference

5) I used begingetresponse instead and no difference.

6) I suspect if this is fiddler problem. I use wireshark as well to capture traffic. Sensibly, the held outgoing requests are emulated by fiddler and the response was received in fact. There are not outstanding requests actually. It seems that it is fiddler queuing the requests. I will edit/close the question after I find a better method to test

I had been stuck in this problem for a few days already. Please tell me any single possibility if you could think of.

Answer 1

Finally, I find that my test is not accurate due to an implementation of fiddler. The requests are queued after 2X outstanding requests for unknown reason.

I set up a server and limit its bandwidth to simulate timeout response.

Using wireshark, I can see that 150 SYN could be sent in around 1.4s as soon as my threads are ready.

Answer 2

There is a lot of overhead associated with creating Threads directly. Try using Task factory instead of Thread. Tasks use ThreadPool under the covers, which reuses threads instead of continuously creating them.

        for (int i = 0; i < 80; i++)
        {
            int copy = i;
            Task.Factory.StartNew(() =>
            {
                submit_test(copy);
            });
        }

Check out this other post on the topic:

Why so much difference in performance between Thread and Task?