This is because 2/3 = 0 (integer division), try 2.0 / 3
Rounding of very large floats in Java
-
07-07-2023 - |
Question
Java:
float big = (float) 1e12;
float ulp = Math.ulp(big);
float result = (big + 2/3*ulp) - big;
result
is 0.0
, while I was expecting ulp
(65536.0
). Can somebody explain why?
Solution
OTHER TIPS
You may try this:
float result = (big + 2f/3*ulp) - big;
ie, you need to typecast the division values else integer/integer will result to zero.
Try typecasting either 2 or 3 to float and try, as 2/3=0, thats why result=0.0
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow