Python: create fixed point decimal from two 32-bit ints (one for int portion, one for decimal)

Question

I have a 64-bit timestamp unpacked from a file with binary data, where the top 32 bits are the number of seconds and the bottom 32 bits are the fraction of the second. I'm stuck with how to actually convert the bottom 32 bits into a fraction without looping through it bit-by-bit.

Any suggestions?

For reference, the number 4ca1f350 9481ef80 translates to 1285682000.580107659

Edit: For context: the data comes from a packet capture device and the documentation I've seen says that it the fractional part has roughly nano-second precision (specifically it outputs 29 of the 32 bits, giving ~2ns).

Answer 1

To represent the sum of integral and fractional part with enough precision (32 + 29 = 61 bits), you need a Decimal (28 decimal digits by default, which is enough for 93 bits),

>>> from decimal import Decimal
>>> Decimal(0x9481ef80) / Decimal(2**32) + Decimal(0x4ca1f350)
Decimal('1285682000.580107659101486206')

or Fraction (exact),

>>> from fractions import Fraction
>>> Fraction(0x9481ef80, 2**32) + Fraction(0x4ca1f350)
Fraction(43140329262089183, 33554432)
>>> float(_)
1285682000.5801077

Note that a float uses "IEEE double format" so it can only hold 53 bits of precision:

>>> a = 0x9481ef80 / 2**32 + 0x4ca1f350
>>> b = 0x9481ef90 / 2**32 + 0x4ca1f350
>>> a == b

It is fine if you store the fractional part as its own variable, but if that's the case, why not just keep it as-is?

>>> 0x9481ef80 / 2**32
0.5801076591014862
>>> 0x9481ef90 / 2**32
0.5801076628267765

Answer 2

You can just divide the hex number by the maximum possible to get the correct ratio:

>>> float(0x9481ef80) / 0x100000000
0.58010765910148621

Answer 3

You didn't say seconds since when. It looks like it's since 1970-01-01. You can calculate a fudge factor that is the number of seconds between the epoch (1970-01-01) and your expected lowest value. Then you adjust each value ... vadj = float(hi32 - fudge) + lo32 / 2.0 ** 32

If the difference between max(hi32) and min(lo32) is less than about 6 days worth (should be enough for a packet capture exercise (?)), then you need only 19 bits for hi32 - fudge. 19 bits + 32 bits is 51 bits -- within the precision of a Python float IIRC.

It's late here so I'm not going to do a detailed analysis but the above should give you the picture.

Edit: why @unwind's answer doesn't work:

>>> a = 0x00000001/4294967296.0 + 0x4ca1f350
>>> b = 0x00000002/4294967296.0 + 0x4ca1f350
>>> b - a
0.0
>>>

Edit 2: What operations do you want to do on a timestamp apart from str(), repr(), timestamp_from_str()? Difference is about all that comes to mind. You can use something like this:

>>> class TS64(object):
...   def __init__(self, hi, lo):
...     self.hi = hi
...     self.lo = lo
...   def float_delta(self, other):
...     hi_delta = self.hi - other.hi
...     # check that abs(hi_delta) is not too large, if you must
...     return hi_delta + (self.lo - other.lo) / 4294967296.0
...
>>> a = TS64(0x4ca1f350, 1)
>>> b = TS64(0x4ca1f350, 2)
>>> b.float_delta(a)
2.3283064365386963e-10
>>> repr(_)
'2.3283064365386963e-10'
>>>

About my "if you must" comment: If the observations are more than 6 days apart, do you really need accuracy down to the last (second / 2 ** 32)??? IMHO, if you do float(difference(ts1, ts2)) instead of float(ts1) - float(ts2), you should be OK.

Edit 3: Ambiguity/inconsistency alert

Please edit your question to address the following issues:

You say in a comment that """the documentation I'm looking at says that it the fractional part has nano-second precision (specifically it outputs 29 of the 32 bits)""". Please provide a URL for that documentation.

There are 1000000000 (10**9) nanoseconds in a second. One would expect the fractional part to require math.log(10**9, 2) rounded up (i.e. 29.897352853986263 rounded up i.e. 30) bits, not 29. Please explain.

Please answer: Of the 32 bits available, which 29 or 30 bits contain the fractional part and which 3 or 2 bits are always zero?

Secondly one would expect to convert the nanoseconds to seconds by dividing by 10**9. However your statement in your question """the number 4ca1f350 9481ef80 translates to 1285682000.580107659""" is consistent with dividing by 2**32. In fact 0x9481ef80 is 2,491,543,424 which is greater than twice 10**9. Please explain. What is the source of the "translates to" statement? Do you have any other examples?