Arithmetic in ruby

Question

Why this code 7.30 - 7.20 in ruby returns 0.0999999999999996, not 0.10?

But if i'll write 7.30 - 7.16, for example, everything will be ok, i'll get 0.14.

What the problem, and how can i solve it?

Answer 1

What Every Computer Scientist Should Know About Floating-Point Arithmetic

Answer 2

The problem is that some numbers we can easily write in decimal don't have an exact representation in the particular floating point format implemented by current hardware. A casual way of stating this is that all the integers do, but not all of the fractions, because we normally store the fraction with a 2**e exponent. So, you have 3 choices:

1. Round off appropriately. The unrounded result is always really really close, so a rounded result is invariably "perfect". This is what Javascript does and lots of people don't even realize that JS does everything in floating point.

2. Use fixed point arithmetic. Ruby actually makes this really easy; it's one of the only languages that seamlessly shifts to Class Bignum from Fixnum as numbers get bigger.

3. Use a class that is designed to solve this problem, like BigDecimal

To look at the problem in more detail, we can try to represent your "7.3" in binary. The 7 part is easy, 111, but how do we do .3? 111.1 is 7.5, too big, 111.01 is 7.25, getting closer. Turns out, 111.010011 is the "next closest smaller number", 7.296875, and when we try to fill in the missing .003125 eventually we find out that it's just 111.010011001100110011... forever, not representable in our chosen encoding in a finite bit string.

Answer 3

The problem is that floating point is inaccurate. You can solve it by using Rational, BigDecimal or just plain integers (for example if you want to store money you can store the number of cents as an int instead of the number of dollars as a float).

BigDecimal can accurately store any number that has a finite number of digits in base 10 and rounds numbers that don't (so three thirds aren't one whole).

Rational can accurately store any rational number and can't store irrational numbers at all.

Answer 4

That is a common error from how float point numbers are represented in memory.

Use BigDecimal if you need exact results.

result=BigDecimal.new("7.3")-BigDecimal("7.2")
puts "%2.2f" % result

Answer 5

It is interesting to note that a number that has few decimals in one base may typically have a very large number of decimals in another. For instance, it takes an infinite number of decimals to express 1/3 (=0.3333...) in the base 10, but only one decimal in the base 3. Similarly, it takes many decimals to express the number 1/10 (=0.1) in the base 2.

Answer 6

Since you are doing floating point math then the number returned is what your computer uses for precision.

If you want a closer answer, to a set precision, just multiple the float by that (such as by 100), convert it to an int, do the math, then divide.

There are other solutions, but I find this to be the simplest since rounding always seems a bit iffy to me.

This has been asked before here, you may want to look for some of the answers given before, such as this one: Dealing with accuracy problems in floating-point numbers