How can I limit Parallel.ForEach?

Question

I have a Parallel.ForEach() async loop with which I download some webpages. My bandwidth is limited so I can download only x pages per time but Parallel.ForEach executes whole list of desired webpages.

Is there a way to limit thread number or any other limiter while running Parallel.ForEach?

Demo code:

Parallel.ForEach(listOfWebpages, webpage => {
  Download(webpage);
});

The real task has nothing to do with webpages, so creative web crawling solutions won't help.

Answer 1

You can specify a MaxDegreeOfParallelism in a ParallelOptions parameter:

Parallel.ForEach(
    listOfWebpages,
    new ParallelOptions { MaxDegreeOfParallelism = 4 },
    webpage => { Download(webpage); }
);

MSDN: Parallel.ForEach

MSDN: ParallelOptions.MaxDegreeOfParallelism

Answer 2

You can use ParallelOptions and set MaxDegreeOfParallelism to limit the number of concurrent threads:

Parallel.ForEach(
    listOfwebpages, 
    new ParallelOptions{MaxDegreeOfParallelism=2}, 
    webpage => {Download(webpage);});     

Answer 3

Use another overload of Parallel.Foreach that takes a ParallelOptions instance, and set MaxDegreeOfParallelism to limit how many instances execute in parallel.

Answer 4

And for the VB.net users (syntax is weird and difficult to find)...

Parallel.ForEach(listOfWebpages, New ParallelOptions() With {.MaxDegreeOfParallelism = 8}, Sub(webpage)
......end sub)