CURL and DDOS Problems

Question

I need get some data from remote http server.Im using Curl Classes for multirequests. My problem is Remote Server's Firewall. Im sending 1000 between 10000 GET and POST requests. And Server bans me from DDOS.

İ used this measures.

• packages still contain header information

  curl_setopt($this->ch, CURLOPT_HTTPHEADER, $header);

• packages still contain random referer information

  curl_setopt($this->ch, CURLOPT_REFERER, $refs[rand(0,count($refs))]);

• packages still contain random user agents

  curl_setopt($this->ch, CURLOPT_USERAGENT, $agents[rand(0,count($agents))]);

• I send packages by using the function of sleep at random intervals.

  sleep(rand(0,10));

But bans access to the server each time for 1 hour.

Sorry for my bad english :) Thanks for all.

Answer 1

Sending a large number of requests in a short space of time to the server is likely to have the same impact as a DOS attack whether that is what you intended or not. A quick fix would be to change the sleep line from sleep(rand(0,10)); which means there is a 1 in 11 chance of sending the next request instantly to sleep(3); which means there will always be 3 seconds (approximately) between requests. 3 seconds should be enough of a gap to keep most servers happy. Once you've verified this works you can reduce the value to 2 or 1 to see if you can speed things up.

A far better solution would be to create an API on the server that allows you to get the data you need in 1, or at least only a few, requests. Obviously this is only possible if you're able to make changes to the server (or can persuade those who can to make the changes on your behalf).