Does ScraperWiki rate limit sites it is scraping?

Question

Does ScraperWiki somehow automatically rate limit scraping, or should I add something like sleep(1 * random.random()) to the loop?

Answer 1

There is no automatic rate limiting. You can add a sleep command written in your language to add rate limiting.

Very few servers check for rate limiting, and usually servers containing public data don't.

It is, however, good practice to make sure you don't overrun the remote server. By default, scrapers only run in one thread, so there is a built in limit to the load you can produce.