pySerial data received from XBee not properly displayed

Question

I am trying to get multiple XBees running as sensors and output devices to send their samples to a coordinator XBee hooked up as below and to turn things on and off on these remote XBees when instructed to. This 'received data' problem of mine seems similar to Stack Overflow question pySerial and reading binary data, but I do not consider it answered by that problem and its resolution.

So what steps will reproduce the problem?

1. Using python-xbee (ver 2.1.0, or 2.0.0), pySerial latest version 2.6.0, Python 2.7.3 on Raspberry Pi running Debian Wheezy (7.0).

2. Run the script below

3. All XBees are XB24-Z7-WIT-004 Series 2. The Co-ordinator is in API2 mode. I tried my two- sensor XBee's in both AT and API1 or API2 modes without any difference (notes on the firmware version is reflected as comments in the attached .py script)

What is the expected output? What do I see instead?

I expect the address received as a result of the script, or as the output from Minicom to be the correct 'source_addr_long', namely x13\xa2\x00\x40\x79\xe6\x5f, but instead I receive \x00\x13\xa2\x00@y\xe6. The 'source_addr' returns \xe3+. (The 'source addr' related stuff is handled / done by python_XBee libraries.) Here is my script:

#!/usr/bin/env python2.7

# NOTE - Not my own code - Abrie Willemse
# NOTE - I am not a programmer - Abrie Willemse

# I am using XBee XB24-Z7 WIT-004 for all devices
# Coordinator is running API
# SENSOR_1 and SENSOR_2 are Sensor Routers running AT (firmware XB24ZB 22A7) (I have tried API firmware XB24ZB 23A7) too)

import serial
from xbee import ZigBee
import time, sys, datetime

serial_port = serial.Serial('/dev/ttyAMA0', 9600)

zb = ZigBee(serial_port)


while True:
    try:
        data = zb.wait_read_frame() #Get data for later use
        print data # To check what comes in before processing / parsing (already buggered up)
        addr = repr(data ['source_addr_long']) # Working sort of, but with @y... issue in results
        file = open('/media/log/senslog.txt','a')
        value = float(((data['samples'])[0])['adc-0'])
        num = (value * 3.0) / 1023.0
        file.write(datetime.datetime.now().strftime('%d-%m-%Y %H:%M:%S') + ' ' + str(addr) + ' ' + str(value) + ' ' + str(num) + '\n')
        print str(datetime.datetime.now().strftime('%d-%m-%Y %H:%M:%S') + ' ' + str(addr) + ' ' + str(value) + ' ' + str(num) + '\n')
        file.close()

    except KeyboardInterrupt:
        break

serial_port.close()

This is the output:

{'source_addr_long': '\x00\x13\xa2\x00@y\xe6_', 'source_addr': '\xe3+', 'id': 'rx_io_data_long_addr', 'samples': [{'adc-0': 516, 'adc-3': 519, 'dio-6': False}], 'options': '\x01'}
18-06-2013 14:32:15 '\x00\x13\xa2\x00@y\xe6_' 516.0 1.51319648094

Note the problem starting at the @y above in the output. Note the correct data received, when I connect that very same co-ordinator to a Windows PC (this is the entire packet received from a remote XBee):

What versions am I using? On what operating system?

Using python-xbee (ver 2.1.0, or 2.0.0), PySerial latest version 2.6.0, Python 2.7.3 on Raspberry Pi Model B running Debian Wheezy (7.0) (upgraded to the latest version recently, in an attempt to address this matter.

Notes:

When I use ZigBee Operator software (Windows based, via an old-fashioned COM port), the addresses and entire messages are sent and received properly. Finally, it is quite possible that nothing is wrong with pySerial, I might just be buggering it up in code, although that does not explain why Miniterm already shows it wrong. Also, all serial port parameters have been checked, the XBee carefully decoupled or filtered between Vcc and ground pins, etc.

UPDATE, following further investigation, it would appear that if anything, the problem is possibly related to the pySerial library, and not with the python-XBee libraries. I base that on the following (referring to the expected results listed in the earlier part of my post, as well as the actual results, also listed above:

x40 = ascii @ and
x79 = ascii y and
xe6 = seems undefined in [ASCII][7], therefore seems to be coming through OK as xe6 and then finally,
x5f = ascii underscore (_)

Therefore, my theory is that for some reason, pySerial stops treating the stream / string (or whatever the correct technical term is) following the last x00 in \x00\x13\x2A\x00 and then start adding the ASCII characters equivelant to the hex characters / values instead. Using a terminal program which is reliant on the pySerial library (Miniterm), on the Raspberry Pi, I already receive the data wrong. This is before my script. (See comment to this post as the result of later discovery.)

It is important for me to receive the 'hardware address' correctly, since the MY address in XBee can change dynamically (I think the coordinator assigns it on the fly). This would be a problem when sending specific commands to a specific XBee module, obviously with a very specific outcome in mind. How can I fix this problem?

Answer 1

As a matter of fact @ is \x40. And y is \x79. So the "values" are correct...

>>> '\x13\xa2\x00\x40\x79\xe6\x5f' == '\x13\xa2\x00@y\xe6_'
True

If it is only a matter of formatting, you could use something like that to pretty display your addresses:

>>> value = '\x13\xa2\x00@y\xe6_'

>>> pretty_value = ':'.join("{:02X}".format(ord(c)) for c in value)
>>> print(pretty_value)
13:A2:00:40:79:E6:5F

---

On the other hand, it seems to me that you are off by one byte while accessing the address:

Expected:         \x13\xa2\x00\x40\x79\xe6\x5f
Actual value: \x00\x13\xa2\x00\x40\x79\xe6

One possible reason is that you missed tha fact that in API 2 mode some characters could be escaped. So changing the actual offset of the various fields in the data frame. Since you are using a library, are you sure it handle properly API 2 mode? Is it correctly configured to do so?

---

Concerning your frame:

7E 00 16 92 00 13 A2 00 40 79 E6 5F
DF 13 01 01 00 40 09 00 40 02 04 02
07 2E

Just decoding the first fields fields of the header:

• This is a 0x92 "IO Sample Rx" frame of 16 bytes long.
• The source address 64 bits is 00:13:A2:00:40:79:E6:5F
• The source address 16 bits is DF:13
• The packed was acknowledged (0x01)