python conversion int into arbitrary number of bytes

Question

I am facing a little corner case of the famous struct.pack.

The situation is the following: I have a dll with a thin layer wrapper to python. One of the python method in the wraper accept a byte array as argument. This byte array the representation of a register on a specific hardware bus. Each bus has different register width, typically 8, 16 and 24 bits wide (alignement is the same in all cases).

When calling this method I need to convert my value (whatever that is) to a byte array of 8/16 or 24bits. Such conversion is relatively easy with 8 or 16bits using the struct.pack:

byteList = struct.pack( '>B', regValue )    # For 8 bits case
byteList = struct.pack( '>H', regValue )    # for 16 bits case

I am now looking to make it flexible enough for all three cases 8/16 & 24 bits. I could use a mix of the two previous line to handle the three cases; but I find it quite ugly.

I was hoping this would work:

packformat = ">{0}B".format(regSize)
byteList = struct.pack( packformat, regValue )

But it is not the case as the struct.pack expect an equal amount of arguments.

Any idea how can I convert (neatly) my register value into an arbitrary number of bytes?

Answer 1

You are always packing unsigned integers, and only big endian to boot. Take a look at what happens when you pack them:

>>> import struct
>>> struct.pack('>B', 255)
'\xff'
>>> struct.pack('>H', 255)
'\x00\xff'
>>> struct.pack('>I', 255)
'\x00\x00\x00\xff'

Essentially the value is padded with null bytes at the start. Use this to your advantage:

>>> struct.pack('>I', 255)[-3:]
'\x00\x00\xff'
>>> struct.pack('>I', 255)[-2:]
'\x00\xff'
>>> struct.pack('>I', 255)[-1:]
'\xff'

You won't get an exception now, if your value is too large, but it would simplify your code enormously. You can always add a separate validation step:

def packRegister(value, size):
    if value < 0 or value.bit_length() > size:
        raise ValueError("Value won't fit in register of size {} bits".format(size))
    return struct.pack('>I', value)[-(size // 8):]

Demo:

>>> packRegister(255, 8)
'\xff'
>>> packRegister(1023, 16)
'\x03\xff'
>>> packRegister(324353, 24)
'\x04\xf3\x01'
>>> packRegister(324353, 8)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in packRegister
ValueError: Value won't fit in register of size 8 bits