comparing floating point numbers in C for unit testing

Question

So I'm using CUnit to do unit testing. I expect something like

float x;
x = atof("17.99");

And I want to test this with an assert; obviously with some small epsilon I could

CU_ASSERT(abs(x - atof("17.99")) < epsilon);

However I'm considering

r = atof("17.99");
CU_ASSERT(0 == memcmp(&x, &r, sizeof(float));

This does appear to work. I wish to use this to avoid having to manually set epsilon on each test based on the values. In the above case 1e-6 should be sufficient; however if the value is 1e-10 using epsilon of 1e-6 may not catch a problem. The more choices the developer has to make the more room for error.

My questions are: Should this technique be stable on posix systems? That is, if the two floating point numbers being compared are generated by exactly the same steps should their internal representation be exactly the same.

edit: More to the point I'd eventually like a CU_ASSERT_FLOAT_EQUAL macro.

Answer 1

So the answer appears to be no.

The reason is that even if you use an identical series of computations in computing each variable, if there is any code between the steps of computing the value you may induce different rounding errors, as noted by the note in @AProgrammer's response.

The issue is that while you might declare a n-bit floating point it may be stored in a larger register (The x87 uses an 80 bit register). If the value is pushed off the register and into memory to free the register for other operations, the value is then truncated (rounded? Where did my notes go...). When the value is brought back on to the register that lost precision carries through the rest of the computation.

On the other hand another piece of code may go through the exact same steps in computing the value; however if the value is not pushed off the register (or is pushed off at a different place...) then you get a different truncation when it is stored in memory again.

All this is IEEE approved according to the notes from the gcc mailing lists/bug reports.

Since I haven't touched a register since the 80386, I can only guess at what the modern x86 and amd_64 processors have; but I'm guessing without hints to gcc that for the x86 it's using a basic x87 register set or a basic SSE register set.

So the rule of thumb to use fabs(x-y) < epsilon; holds, which for CUnit is provided for in double format (and one could easily write a float version of the macro if one wanted to be as anal about things as I have a habit of getting) as noted by the post which @Martin Beckett commented on.

Answer 2

Comparing floating point values is hard. Mixing in strings isn't going to make things any better, and certainly doesn't introduce the small amount of leeway that epsilon would.

Have a look at this article: Comparing Floating Point Numbers, 2012 Edition. For my money, ULP is the way to go. There are some nasty edge cases, but you can probably ignore most of them.

Edit: 31 May Having thought about this a bit more, I think what you are asking is "If exactly the same steps are used to calculate a float in the test function and in the function under test, will the answer be exactly the same - so that I don't need to worry about +/- some small error?"

The answer is yes, they will be the same. But in that case you have rendered the unit test pointless because if there is a mistake in the code under test then the same mistake will necessarily be present in the test function.

By the way, don't get distracted by the use of memcmp(&x, &y, sizeof(x)). This just tests that exactly the same bits are set in both values. If that is true, then x == y will necessarily be true. Stick with x == y.

Answer 3

Beware of comparing a float to a double, as is done in the initial question. The float will necessarily have lost precision, so when it is compared to a full-precision double you may find that the numbers are not equal, even when the calculations are perfect.

See http://randomascii.wordpress.com/2012/06/26/doubles-are-not-floats-so-dont-compare-them/ for details.

Answer 4

float r, x;

r = atof("17.99");
CU_ASSERT(0 == memcmp(&x, &r, sizeof(float));

should have the same effect as

r = atof("17.99");
CU_ASSERT(x == r);

Note that atof returns a double, so

CU_ASSERT(x == atof("17.99"));

is different, but

CU_ASSERT(x == (float)atof("17.99"));

should also be the same.

Note also that gcc optimizer had a long standing bug with this on x86 when using the instructions inherited from the x87 (if I'm not mistaken, that doesn't happen on x86_64).

Answer 5

I think there is a different way of approaching this issue. The question is about unit testing. Unit testing should both test and document the unit under test (uut).

To this end when writing the test one should ask what are the acceptable tolerances for the uut. (This may need to be considered for each uut rather than across the test project).

This way the test can avoid testing for equality, test that the value is within the acceptable range as well as documenting the acceptable tolerance of the unit tests result.