Question

Why exactly is Octave giving me the following results when given 1/3 or 0.1?

octave:1> (4/3-1)*3-1
ans = -2.2204e-16

octave:2> (10.1-10)*10-1
ans = -3.5527e-15

octave:3> (10.125-10)*8-1
ans = 0
Était-ce utile?

La solution

This is a consequence of floating point arithmetic. Your computer cannot fully accurately represent 4/3 as a floating point binary number hence you get a rounding error. However it can fully accurately represent 10.125 in binary and hence no error.

Said another way, consider the parts of your number after the decimal point. For 4/3 we have 0.333333... which is 0.010101010101... in binary. Your computer can't store an infinite series of numbers after and so it truncates to some precision. However for 10.125 consider what 0.125 is in binary which is simply 0.001. So only three "decimal" points in binary and thus no truncation and no rounding error.

Licencié sous: CC-BY-SA avec attribution
Non affilié à StackOverflow
scroll top