Converting float to int in F#

Question

Can someone please explain why int (0.4 * 10.0) is 4 yet int ((2.4 - 2.0) * 10.0) is 3?

And then also please explain how to get 4. i.e. Do I really have to do int (System.Math.Round((2.4 - 2.0) * 10.0))?

It's just so ugly for something that should be so simple.

Answer 1

The double representation of 2.4 - 2.0 is 0.399999999999. When you convert a number to int the decimal part is truncated so 3.999999999 is truncated to 3.

Answer 2

And then also please explain how to get 4. i.e. Do I really have to do

int (System.Math.Round((2.4 - 2.0) * 10.0))

it's just so ugly for something that should be so simple.

Of course not, just define

let round x = int System.Math.Round(x)

Answer 3

Actually, in .Net there is a decimal data type which will help you with this problem, if your numbers can be represented exactly as decimal fractions.

2.4m - 2.0m == 0.4m
2.4 - 2.0   != 0.4

Answer 4

To covert a float number to an int use:

let i = Convert.ToInt32(f)

Answer 5

Floating-point numbers are not infinite precision. Some numbers can be represented perfectly (anything involving powers of two, e.g. 1, 1/2 = 0.5, 1/4 = 0.25, etc.), and others can't (e.g. 1/3 cannot be exactly decomposed into a finite sequence of powers of two). Often, chopping the end of the infinite series causes the encoded version of a number to be slightly smaller than its true value.

Combine this with the fact that the (int) operator truncates rather than rounds, and you can run into cases where an operation that should yield 4.0 results in 3.999999999991 instead, which becomes 3.

Any time you're trying to convert floats/doubles to ints, you want to think carefully about which operator you want to use -- truncating (rounding down), rounding up, or rounding-to-nearest.

Since this is an issue with floating-point representations, this is true not just in F#, but C# and (IIRC) Java and other languages.