Unpacking data in python in struct library

Question

When I pack the data to fixed length and then while unpacking I am unable to retrieve the data with out mentioning the actual length of the data.

How do I retrieve only data without the \x00 characters without calculating the length in prior.

>>> import struct
>>> with open("forums_file.dat", "w") as file:
    file.truncate(1024)
>>> country = 'india'
>>> data = struct.pack('20s', country)
>>> print data
    india
>>> data
    'india\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
>>> print len(data)
    20
>>> unpack_data = struct.unpack('5s', country)
>>> unpack_data
    ('india',)

In the above code snippet I had mentioned the length of the data(5s) while unpacking.

Answer 1

Short answer: You can't do it directly.

Longer answer:

The more indirect solution is actually not that bad. When unpacking the string, you use the same length as you used for packing. That returns the string including the NUL chars (0 bytes).

Then you split on the NUL char and take the first item, like so:

result_with_NUL, = struct.unpack('20s', data)
print(repr(result_with_NUL))

result_string = result_with_NUL.split('\x00', 1)[0]
print(repr(result_string))

The , 1 parameter in split() is not strictly necessary, but makes it more efficient, as it splits only on the first occurrence of NUL instead of every single one.

Also note that when packing and unpacking with the goal to read/write files or exchange data with different systems, it's important to explicitly precede your format strings with "<" or ">" (or "=" in certain very special cases), both for packing and unpacking, since otherwise it will align and pad the structures, which is heavily system dependent and might cause hard to find bugs later.