Question

Has anybody got any ideas on this one?

When we run:

printf("%.0f", 40.5)

On a windows box the return is "41" but on our production ubuntu server we're getting "40"

Was it helpful?

Solution

Looks like a simple case of binary floating point imprecision. On one machine you get 40.499999999 which rounds to 40; on the other machine you get 40.500000000001 which rounds to 41.

If you need exact numbers then you should not use binary floating point. You can use fixed point decimal, or decimal floating point.

Edit: you're using BigDecimal, you say. Why not avoid any conversion to float by using #round and then #to_i? (Or #floor or #ceil instead of #round... it's not clear what your goal is.)

b = BigDecimal.new("40.5")
print b.round.to_i  # => 41

OTHER TIPS

How about using .round instead? Rails even enhances it so that you can specify the precision (see API doc).

use %.0g instead

ScottJ's answer does not explain your problem -- 40.5 is exactly representable in binary. What's probably happening is this: Windows' printf "rounds half up," and Ubuntu's printf "rounds half even."

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top