Parallel.For does not loop expected times

Question

I have this test:

public void Run()
    {
        var result = new List<int>();
        int i = 0;
        Parallel.For(0, 100000, new Action<int>((counter) =>
        {
            i++;
            if (counter == 99999)
            {
                Trace.WriteLine("i is " + i);
            }

        }));
    }

Now why does the output print seemingly random numbers in the range of 50000 to 99999? I expected the output to always be 99999. Have I missunderstood the parallel for loop implementation?

If I now run the loop 100 times then the program outputs 100, as expected. FYI I have a 8-core CPU

UPDATE: offcourse! I missed the thread safety aspect of it :) thanks! Now lets see which one is faster, using lock, declaring as volatile, or using Interlocked

Answer 1

Your problem is probably due to i++ not being thread-safe and your tasks being in a kind of 'race condition'.

Further explanation about i++ not being thread-safe you can find here: Are incrementers / decrementers (var++, var--) etc thread safe?

A quote of the answer given by Michael Burr in the aforementioned linked thread (upvote it there):

You can use something like InterlockedIncrement() depending on your platform. On .NET you can use the Interlocked class methods (Interlocked.Increment() for example).

A Rob Kennedy mentioned, even if the operation is implemented in terms of a single INC instruction, as far as the memory is concerned a read/increment/write set of steps is performed. There is the opportunity on a multi-processor system for corruption.

There's also the volatile issue, which would be a necessary part of making the operation thread-safe - however, marking the variable volatile is not sufficient to make it thread-safe. Use the interlocked support the platform provides.

This is true in general, and on x86/x64 platforms certainly.

The race to Trace.WriteLine()

Between the time you do ++i and output i, other parallel tasks might have changed/incremented i several times.

Imagine your first task, which is incrementing i so that it becomes 1. However, depending on your runtime environment and the weather of the day, i might be incremented twenty times more by other parallel tasks before the first task outputs the variable -- which would now be 21 (and not 1 anymore). To prevent this from happening use a local variable which remembers the value of the incremented i for a particular task for later processing/output:

int remember = Interlocked.Increment(ref i);
...
Trace.WriteLine("i of this task is: " + remember);

Answer 2

Because your code is not thread safe. i++ is "read modify write" which is not thread safe.

Use Interlocked.Increment instead, or get a lock around it.

Is the ++ operator thread safe?

Answer 3

Parallel.For means it runs all tasks in a parallel way, so for code, it doesn't means the it runs from 0 - 100000, it can start running the delegate function with 99999 first, so that is why you get an arbitrary value of i.

When a parallel loop runs, the TPL partitions the data source so that the loop can operate on multiple parts concurrently. Behind the scenes, the Task Scheduler partitions the task based on system resources and workload. When possible, the scheduler redistributes work among multiple threads and processors if the workload becomes unbalanced.