Question

I'm looking at a c++ task which reads a file that contains decimal numbers and stores each number in a data structure as a single 4-byte float. Before changing the process (to use double) I want to know what is the average error in % between the number in the file and the number stored in the structure.

For example, if the file reads 19.5 and the data structure is 19.49999999 then the error is

abs(19.49999999 / 19.5 - 1) = 5.128205128205128e-8 %

If the distribution matters, most of the numbers in the file are positive and less than 1000.

Was it helpful?

Solution

32-bit floats have 23 bits of mantissa, which means a representation accurate to about 7-8 decimal digits. To be precise, the relative error is 2^-24, or about 6e-8.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top