Concurrent Collection, reporting custom progress data to UI when parallel tasking

Question

I have a concurrent collection that contains 100K items. The processing of each item in the collection can take as little as 100ms or as long as 10 seconds. I want to speed things up by parallelizing the processing, and have a 100 minions doing the work simultaneously. I also have to report some specific data to the UI as this processing occurs, not simply a percentage complete.

I want the parallelized sub-tasks to nibble away at the concurrent collection like a school of minnows attacking a piece of bread tossed into a pond. How do I expose the concurrent collection to the parallelized tasks? Can I have a normal loop and simply launch an async task inside the loop and pass it an IProgress? Do I even need the concurrent collection for this?

It has been recommended to me that I use Parallel.ForEach but I don't see how each sub-process established by the degrees of parallelism could report a custom object back to the UI with each item it processes, not only after it has finished processing its share of the 100K items.

Answer 1

The framework already provides the IProgress inteface for this purpose, and an implementation in Progress. To report progress, call IProgress.Report with a progressvalue. The value T can be any type, not just a number.

Each IProgress implementation can work in its own way. Progress raises an event and calls a callback you pass to it when you create it.

Additionally, Progress.Report executes asynchronously. Under the covers, it uses SychronizationContext.Post to execute its callback and all event handlers on the thread that created the Progress instance.

Assuming you create a progress value class like this:

class ProgressValue
{
    public long Step{get;set;}
    public string Message {get;set;}
}

You could write something like this:

IProgress<ProgressValue> myProgress=new Progress<ProgressValue>(p=>
{
    myProgressBar.Value=p.Step;
});

IList<int> myVeryLargeList=...;
Parallel.ForEach(myVeryLargeList,item,state,step=>
{
    //Do some heavy work
    myProgress.Report(new ProgressValue
    {
         Step=step,
         Message=String.Format("Processed step {0}",step);
    });
});

EDIT

Oops! Progress implements IProgress explicitly. You have to cast it to IProgress , as @Tim noticed.

Fixed the code to explicitly declare myProgress as an IProgress.