Structs and unpacking in Python

Question

I've been struggling to understand how unpacking works. More specifically this specific example in the documentation is confusing me the most because it won't give me the same output when I input it exactly the same. https://docs.python.org/2/library/struct.html#struct-examples

>>> from struct import *
>>> pack('hhl', 1, 2, 3)
'\x00\x01\x00\x02\x00\x00\x00\x03'
>>> unpack('hhl', '\x00\x01\x00\x02\x00\x00\x00\x03')
(1, 2, 3)
>>> calcsize('hhl')
8

8 byte size makes sense but when I do calcsize('hhl') it's returning 16. Storing pack('hhl', 1, 2, 3) as a variable and unpacking it works fine but using the hex values does not. I receive the error "unpack requires a string argument of length 16" , any idea what the problem is? Thanks

Answer 1

it's because when you do:

>>> pack('hhl', 1, 2, 3)
'\x00\x01\x00\x02\x00\x00\x00\x03'

it's not returning a string with [ \, x, 0, ...] but a bytearray containing each number given in hex:

>>> pack('hhl', 1, 2, 3)[0]
'\x00'

so, when you're doing:

>>> unpack('hhl', '\x00\x01\x00\x02\x00\x00\x00\x03')

you're actually trying to unpack a string like I was saying previously. Whereas if you do:

>>> s = pack('hhl', 1, 2, 3)
'\x00\x01\x00\x02\x00\x00\x00\x03'
>>> unpack('hhl', s)
(1, 2, 3)

it's working as expected.

Now as @falstru is telling, a way to convert a string with hex into a bytearray that struct can understand is to use binascii.unhexlify, which handles the endianness and the conversion. You can also do it yourself:

>>> unpack('hhl', str(bytearray([1,0,2,0,0,0,0,0,3,0,0,0,0,0,0,0])))
(1, 2, 3)

Answer 2

If you mean the hex string like '0001000200000003', you need to unhexlify it first using binascii.unhexlify (or binascii.a2b_hex)

>>> import struct
>>> import binascii
>>> struct.unpack('>hhl', binascii.unhexlify('0001000200000003'))
(1, 2, 3)