Вопрос

When running bc -l

if you set ibase=obase=16, then ask for "0.2 * 5"

It equals .8. Why doesn't it equal the hex value, 1?

I originally asked this here (http://bit.ly/17o7RcK).

Это было полезно?

Решение

I think the problem is that you're doing input in hex, but it's doing internal calculations in decimal, and using the length of the hex input to decide how many decimal digits the number should be represented to. So when you input "0.2" (hex), it translates it to the nearest 1-digit decimal number, 0.1(dec). Multiply that by 5, and you get 0.5(dec) = 0.8(hex).

You can see this by forcing it to use more decimal digits. If you do "0.20 * 5", it translates 0.20(hex) into 0.12(dec), multiplying that by 12 gives 0.60(dec), which rounds to 0.99(hex) for output. If you do "0.200 * 5", it translates 0.200(hex) into 0.125(dec), multiplying by 5 gives 0.625(dec) which translates to 0.A00(hex), which is the "correct" result.

Другие советы

0.2 (decimal) --> 0.1 (hex)

0.5 (decimal) --> 0.8 (hex)

so what, I believe, is happening is:

hexadecimal input (0.1 * 5), which would output as 0.5 in decimal (base 10), outputs as (0.8) in hexadecimal (base 16)

Лицензировано под: CC-BY-SA с атрибуция
Не связан с StackOverflow
scroll top