Pregunta

Let's consider a scenario: we have a double variable in our program representing percentage of byte value. After some action we are converting mentioned double to byte with simple range conversion out = 2.55*original. But what happens at 100% ?

double a = 100.0;
Console.WriteLine("original: {0}", a);
Console.WriteLine("original*2.55: {0}", (2.55*a));
byte b = (byte)(2.55*a);
Console.WriteLine("byte: {0}", b);
Console.WriteLine("wat -.-: {0}", (byte)(100.1 * 2.55));

>original: 100
>original*2.55: 255
>byte: 254  <------------------!!!
>wat -.-: 255

At the same time:

Console.WriteLine("{0}", (byte)(255.0));
Console.WriteLine("{0}", (byte)(255));

>255
>255

What is going on? Shouldn't representation of integer in double that is result of multiplication behave same as one that was not multiplied? What am I missing?

¿Fue útil?

Solución

The expression

2.55 * 100.0

is evaluated to

254.99999999999997

which means that truncation when casting to a byte yields 254.

Otros consejos

You should be able to convert the double to a byte by using

a = Convert.ToByte(b);

That's because 2.55 * 100 can not be represented as a float. Instead, it evaluates to 254.999999999, which, when truncated, results in 254.

Licenciado bajo: CC-BY-SA con atribución
No afiliado a StackOverflow
scroll top