Python: What's the best way to send status bits and timestamps with PySerial

Question

I have done a fair bit of searching and reading around but haven't found an exact clear answer. I am making python application which will communicate with another over the serial port. I have been using PySerial to accomplish this. The types of packets I want to send are status information and data readings. I am proposing to form of my packets as follows:

[ Start Byte ][ Length ][ Message ][ End Byte ][ Checksum ]

Although I don't think I need the end byte since have a length in there. Am I correct?

I am thinking of shaping the 'Message' part of the packet for status info like so:

[ MsgTypeID ][ PacketID ][ Status Bits ][ Timestamp ]

The message ID here is just to differentiate that this is a status packet rather than a data packet. The status information itself is made up of 16 bits, representing the state of a number of subsystems in the sending application. For some of these subsystems, a single bit will provide enough info ('0' to say the subsystem is off/low/false, '1' for on/high/true). Others will require 2 or 3 bits (representing states and such). Prototyping the system up, I have been simply constructing a string concatenating the bits up to form a something like '1001110101101100' and sending this out the serial port, with the checksum being a modulo 256 of the status bits.

I am pretty new to Python and serial communication but I know this is probably a waste of bandwidth. I know that PySerial has to send strings but representing each bit as a '0' or '1' like this is using a full string representation for each bit. I was wondering what is the best way to send these bits to reduce the bandwidth?

For example, would I take each 8 bits, convert them to hex and send concatenated hex bytes, as in:

'10011101' + '01101100'

represented as

'\x9d' + '\x6c'

or should I send them as ASCII? I've also seen mentions of the Struct module. Should I be going down that route instead?

The other thing I am wondering is how to represent the timestamp parts of the message.

Any help, suggestions you can give me would be greatly appreciated.

Thanks very much :)

Answer 1

Well, obviously it needs one byte per character sent, so if you can encode your data as an ascii string (str in Python 2, bytes in Python 3), that's most efficient.

The struct module is definitely a good idea: it packs Python data into a string (and unpacks it at the other end). That'll work for your timestamps; pack them as whatever type of int or long seems appropriate (see the linked documentation for your options).

struct doesn't do individual bits, though, so you'll have to do that yourself. I've built a dictionary before to translate characters into groups of eight True/False values:

To decode:

pot = [2**x for x in range(8)]  # Powers of 2 (bytes with one 1 and seven 0s)
bitvalues = {}
for x in range(256):
    bitvalues[chr(x)] = [(x & y) != 0 for y in pot]

To make a dictionary for encoding, replace the last line with:

bitvalues[tuple((x & y) != 0 for y in pot)] = chr(x)

If you want to encode/decode from strings of 1s and 0s instead, replace the bit generating the list/tuple with:

"".join("1" if (x&y) else "0" for y in pot)

Answer 2

How frequently are you expecting to send these updates? If update_frequency * length_of_message is far less than the data rate of the serial connection, then there's very little reason to worry about the efficiency of the data encoding, and you might even consider expanding things out to a more human-readable format, especially when you're getting started.