When running bc -l

if you set ibase=obase=16, then ask for "0.2 * 5"

It equals .8. Why doesn't it equal the hex value, 1?

I originally asked this here (http://bit.ly/17o7RcK).

有帮助吗?

解决方案

I think the problem is that you're doing input in hex, but it's doing internal calculations in decimal, and using the length of the hex input to decide how many decimal digits the number should be represented to. So when you input "0.2" (hex), it translates it to the nearest 1-digit decimal number, 0.1(dec). Multiply that by 5, and you get 0.5(dec) = 0.8(hex).

You can see this by forcing it to use more decimal digits. If you do "0.20 * 5", it translates 0.20(hex) into 0.12(dec), multiplying that by 12 gives 0.60(dec), which rounds to 0.99(hex) for output. If you do "0.200 * 5", it translates 0.200(hex) into 0.125(dec), multiplying by 5 gives 0.625(dec) which translates to 0.A00(hex), which is the "correct" result.

其他提示

0.2 (decimal) --> 0.1 (hex)

0.5 (decimal) --> 0.8 (hex)

so what, I believe, is happening is:

hexadecimal input (0.1 * 5), which would output as 0.5 in decimal (base 10), outputs as (0.8) in hexadecimal (base 16)

许可以下: CC-BY-SA归因
不隶属于 StackOverflow
scroll top