Pregunta

Why exactly is Octave giving me the following results when given 1/3 or 0.1?

octave:1> (4/3-1)*3-1
ans = -2.2204e-16

octave:2> (10.1-10)*10-1
ans = -3.5527e-15

octave:3> (10.125-10)*8-1
ans = 0
¿Fue útil?

Solución

This is a consequence of floating point arithmetic. Your computer cannot fully accurately represent 4/3 as a floating point binary number hence you get a rounding error. However it can fully accurately represent 10.125 in binary and hence no error.

Said another way, consider the parts of your number after the decimal point. For 4/3 we have 0.333333... which is 0.010101010101... in binary. Your computer can't store an infinite series of numbers after and so it truncates to some precision. However for 10.125 consider what 0.125 is in binary which is simply 0.001. So only three "decimal" points in binary and thus no truncation and no rounding error.

Licenciado bajo: CC-BY-SA con atribución
No afiliado a StackOverflow
scroll top