.NET rounding error in ToString(“f2”)

Question

Hello I have this code in C#:

float n = 2.99499989f;
MessageBox.Show("n = " + n.ToString("f2", CultureInfo.InvariantCulture));

And this code in C++:

float n = 2.99499989f;
printf("n = %.2f", n);

First one outputs 3.00.
Second one outputs 2.99.

I have no clue why this is happening.

Update:

I also tried Objective-C NSLog and the output is 2.99.

I needed to fix it fast so I used following method:

float n = 2.99499989f;
float round = (float)Math.Round(n, 2);
MessageBox.Show("round = " + round.ToString(CultureInfo.InvariantCulture));

This code shows 2.99, but computes round in double precision. I can't find Math.RoundF.

Answer 1

(I realize this is an old question, but I'm answering this as I am understanding some things myself...)

My suggestion is that "printf" is converting the float to a double before applying the formatting. I tried converting the float to a double, and then looking at the double's ToString("f2") result, which it did round in the opposite direction as the float's value was rounded.

Others re-enforce this idea about printf:

C automatically converts the float values to double (it is a standard conversion made when you call a function that takes variable arguments, such as int printf...

https://stackoverflow.com/a/7480244/119418

https://stackoverflow.com/a/6395747/119418

I believe the reason the float rounds up is because 2.995F doesn't convert perfectly into binary and is = 2.99499989F, and we'd expect 2.995 to round up.

I believe the reason the float copied into a double rounds down is because 2.995 as a double <> 2.99499989 as a double, and 2.995 actual double value is greater than that value (more precision closer to its true decimal value), and so that value we'd expect to round down.

It might seem wrong that 2.99499989F rounds up to 3.00 even though it is seemingly less than 2.995, but keep in mind that the 2.99499989 is a decimal, not a float, and you are "converting" it into a float, basically converting base-10 to base-2, and it is really 1's and 0's, then asking it to be rounded in base-10 terms, which means a conversion has to happen. Well, there are at least 2 base-10 values that I mentioned that can convert to that number as a float, and the simplest of those is 2.995.

Answer 2

Using BitConverter.GetBytes and printing out the actual bytes produced shows that this is not a compiler difference - in both cases, the actual float value stored is 0x403FAE14, which this handy calculator tells me is the exact value

2.99499988555908203125

The difference therefore must lie in differing behaviours of printf and ToString. More than that I cannot immediately say.

Answer 3

First one rounds the number incorrectly.

Second one only selects two digits after comma, I guess.

Maybe some floating-point-precission issues? 2.9949... in fact rounds to more than 2.995?

Answer 4

The float precision is 2^{-23} or about 0,0000001192 (relative error). Look, 2.995-2.99499989 = 0,00000011. Relative error is 3,6*10^{-8}.

If some compiler reads all digits of the 2.99499989f constant then the result is number greater than 2.995. But if another compiler reads only 2.994999 (because the precision is less than 2^{-23} and last digits are unimportant) then the result is number less than 2.995

Answer 5

Isn't that due to the fact that a 'float' isn't 'that' precise ?

Check this out, when you use a decimal:

        // float
        Console.WriteLine (2.99499989f.ToString ("f2"));
        // The above line outputs 3.00

        // decimal
        Console.WriteLine (2.99499989M.ToString ("f2"));
        // The above line outputs 2.99

Perhaps float can't represent 2.99499989 or whatever as 2.99.
Look also what happens when you do this:

// float
Console.WriteLine (2.98499f.ToString ("f2"));
// The above line outputs 2.99

Answer 6

The number 2.99499989f can't be presented exactly in IEEE754. Apparently printf and ToString handle this situation differently.