Pergunta

Consider this code:

Console.WriteLine((197688 * 0.1) == 19768.800000000003); // True
Console.WriteLine((197688 * 0.1) == 19768.8); // False

is an incorrect calculation?

Java-> http://ideone.com/D3QsUo C# -> http://ideone.com/wp5pM9

Foi útil?

Solução

Floating-point types (and calculations) are imprecise in nature. They work in binary, not in decimal, and hence calculations yield “unexpected” decimal values.

Compare it with this:

Console.WriteLine((197688 * 0.1m) == 19768.800000000003m); // False
Console.WriteLine((197688 * 0.1m) == 19768.8m); // True

The result is as you'd expect, because decimal is used. As the name suggests, it's suited for decimal calculations because of its internal representation.

Side note: The rule of thumb is never use floating-point types for monetary calculations. Exactly becuase of precision issues with decical values.

Outras dicas

A double only has 15-16 digits of precision - the compiler will translate your literal to the closest value that can be represented by a double which is this case is 19768.8

Licenciado em: CC-BY-SA com atribuição
Não afiliado a StackOverflow
scroll top