Question

I want to find the delta value of a number.
I tried to with this. I used Abs function. It is working fine for whole numbers.
But when i try to do the following,

var gh = Math.Abs(3.223 - 6.243);  

it returns 3.0200000000000005 . But i am expecting 3.02.
Why? how to get 3.02?

Was it helpful?

Solution

Because you are using float/double values, which are inherently not 100% precise (it's due to their binary representation). For example you can't even have precise 0.1 with floating-point numbers, because its binary representation is periodic 1.(1001). This is a very complete article on the matter: What Every Computer Scientist Should Know About Floating-Point Arithmetic

Tl;dr solution is to use decimal for precise decimal arithmetic operations

var gh = Math.Abs(3.223m - 6.243m); 

OTHER TIPS

Neither 3.223 nor 6.243 are exactly representable in binary floating point. Your desired answer of 3.02 is not exactly representable either.

If you want exact decimal arithmetic, you should use the decimal type. For example:

var gh = Math.Abs(3.223m - 6.243m);  

which will result in gh being of type decimal, and having the exact value that you desire. The m suffix on the literals is used to indicate values of type decimal.

Useful resources

Do you need to display the number like this, or you want to use it for further calculations?

For displaying it you can use string formatting. If you use it for further calculations you should leave it like this and apply truncation where you need to display numbers.

I am getting the right value. Try converting it to Decimal and maybe you will get the right precision:

decimal gh = Convert.ToDecimal(Math.Abs(3.223 - 6.243));

Good luck!

Can you maybe try this : var sgh= gh.ToString("0.00", CultureInfo.CurrentCulture);

it depends on how many digit you want to have, if you want to have decimal, then var gh = Convert.ToDecimal(Math.Abs(3.223 - 6.243));

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top