cancellation in numpy array operation including a scalar

Question

I'm using NumPy version 1.7.1. Now I came across a strange cancellation I don't understand:

>>> import numpy as np
>>> a = np.array([ 883,  931,  874], dtype=np.float32)

Mathematically a+0.1-a should be 0.1. Now let's calculate the value of this expression and absolute and relative error:

>>> a+0.1-a
array([ 0.09997559,  0.09997559,  0.09997559], dtype=float32)
>>> (a+0.1-a)-0.1
array([ -2.44155526e-05,  -2.44155526e-05,  -2.44155526e-05], dtype=float32)
>>> ((a+0.1-a)-0.1) / 0.1
array([-0.00024416, -0.00024416, -0.00024416], dtype=float32)

First question: This is a quite high absolute and relative error, this is just catastrophic cancellation, isn't it?

Second question: When I use an array instead of the scalar, NumPy is able to calculate with much more precision, see the relative error:

>>> a+np.array((0.1,)*3)-a
array([ 0.1,  0.1,  0.1])
>>> (a+np.array((0.1,)*3)-a)-0.1
array([  2.27318164e-14,   2.27318164e-14,   2.27318164e-14])

This is just the numerical representation of 0.1 I guess.

But why is NumPy not able to handle this the same way if a scalar is used instead of an array as in a+0.1-a?

Answer 1

If you use double precision the scenario changes. What you are getting is expected for single precision (np.float32):

a = np.array([ 883,  931,  874], dtype=np.float64)

a+0.1-a
# array([ 0.1,  0.1,  0.1])

((a+0.1-a)-0.1) / 0.1
# array([  2.27318164e-13,   2.27318164e-13,   2.27318164e-13])

Using np.array((0.1,)*3) in the middle of the expression turned everything to float64, which explains the higher precision in the second result.