Is this PLINQ bug?

Question

Why PLINQ output is different than sequential processing and Parallel.For loop

I want to add sum of square root of 10,000,000 numbers.. Here is the code for 3 cases:

sequential for loop:

double sum = 0.0;
for(int i = 1;i<10000001;i++)
sum += Math.Sqrt(i);

Output of this is: 21081852648.717

Now Using Parallel.For loop:

object locker = new object();
double total ;

Parallel.For(1,10000001,
()=>0.0,
(i,state,local)=> local+Math.Sqrt(i),
(local)=>
{
  lock(locker){ total += local; }
}
);

Output of this is: 21081852648.7199

Now using PLINQ

double tot =  ParallelEnumerable.Range(1, 10000000)
                .Sum(i => Math.Sqrt(i)); 

Output of this is: 21081852648.72

Why there is difference between PLINQ output and Parallel.For and Sequential for loop?

Answer 1

I strongly suspect it's because arithmetic with doubles isn't truly associative. Information is potentially lost while summing values, and exactly what information is lost will depend on the order of the operations.

Here's an example showing that effect:

using System;

class Test
{
    static void Main()
    {
        double d1 = 0d;
        for (int i = 0; i < 10000; i++)
        {
            d1 += 0.00000000000000001;
        }
        d1 += 1;
        Console.WriteLine(d1);

        double d2 = 1d;
        for (int i = 0; i < 10000; i++)
        {
            d2 += 0.00000000000000001;
        }
        Console.WriteLine(d2);
    }
}

In the first case, we can add very small numbers lots of times until they become big enough to still be relevant when added to 1.

In the second case, adding 0.00000000000000001 to 1 always just results in 1 as there isn't enough information in a double to represent 1.00000000000000001 - so the final result is still just 1.

EDIT: I've thought of another aspect which could be confusing things. For local variables, the JIT compiler is able to (and allowed to) use the 80-bit FP registers, which means arithmetic can be performed with less information loss. That's not the case for instance variables which definitely have to be 64-bit. In your Parallel.For case, the total variable will actually be an instance variable in a generated class because it's captured by a lambda expression. This could change the results - but it may well depend on computer architecture, CLR version etc.