Question

I had divided 9501/100.0f expecting to get result of 95.01f, but for some deviant reason the result was 95.01000000002f.

I am aware of rounding errors and also that dividing two bigger floats can give improper result, but these two numbers are relative small, and they should not give bad answer.

I have changed floats to doubles, only to see the same result.

So my answer is, why am I seeing this false output?

And eventually workaround without copying number to string and back.

Was it helpful?

Solution 2

Floating point numbers are numbers represented in binary with limited precision.

The error between expected result and actual result is caused by the fact, that the number 95.01 is infinitely periodical in binary representation.

Double has only 51 binary digits, thus there has to be some rounding before the number is stored in the double precision. Single precision has only 23 digits.

It is not possible to represent 95.01 in finite precision floatin point number without any error. However, you may trust the first 6-9 decimal digits, thus you should format the number with some meaningfull format.

OTHER TIPS

Floating point numbers are not precise, and dealing with them has lots of idiosyncrasies.

What Every Computer Scientist Should Know About Floating-Point Arithmetic

I also enjoy Bruce Dawson's blog entries on floating point values.

Ahh good, another one of us has become a man in the church of programming :)

Floating points are not exact, the precision will vary from machine to machine. 1.0f != 1.00000000000000000000000000000000000 and so on, it's more like 1.0000001002003400011 and so on (I just picked arbitrary numbers here).

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top