How to read only the newest data from a socket, and/or empty a socket, in Python/Jython

Question

I have a bunch of data coming in over a socket connection, and for my application I only care about the most recent data. The messages are strings of text in a json format, separated by newlines. What I am trying to figure out is a clean and efficient way of just reading the latest message. Messages come in around every 30ms or so. One of my issues is the messages start coming before my code has loaded up, so there may be a lot of useless ones that I don't want it to go through.

Any suggestions?

I'm using jython, which is very similar to python, except for a few things that inconveniently don't work, but it has been okay for sockets.

Code snippets of what I have tried so far:

# ...
# This code is repeatedly called every 30 ms or so
in_file = sock_in.makefile( 'r' )
data_in = None
try:
  data_in = json.loads( in_file.readline() )
except JSONDecodeError:
  try:
    data_in = json.loads( in_file.readline() )
  except JSONDecodeError:
    print( "ERROR: malformed message, dropping data" )
    return
# do stuff with data_in
# ...

This basically reads from the socket, and if it happens to grab a chunk that is missing some of the message, it tries again and grabs the next one (which so far has worked every time). My issue with this is, if the messages are coming in too fast (which they are) it seems to be reading through the older ones first, while only the latest ones are useful. This approach seemed to work okay, but I have a feeling it could be a lot better.

another thing I tried was this:

   # ...
   self.buf = ""
   # ... This code is repeatedly called every 30ms or so
   self.read_socket( self.sock_in )
   data_in = json.loads( self.odom_str )
   # ...
   def read_socket( self, sock ):
     out = self.buf
     while True:
         try:
             data = sock.recv( 64 )
         except:
           return False
         if not data:
             break
         nl = data.find('\n')
         if nl >= 0:
             nl += 1
             out += data[:nl]
             self.buf = data[nl:]
             del data
             break
         out += data
     self.odom_str = out
     return True

This second approach looked nicer, but performed a lot worse.

Is there some way to empty the socket file after a successful read? Or an easy way to read from the end of it? Or is this actually reading from the end and I'm getting confused? I am new to using sockets, and a lot of the things I've been trying are probably horrible style, so any help would be greatly appreciated :)

Thanks!

Answer 1

The best way to do this that I've come up with so far, is to set the port to nonblocking, and then continuously read until there is no more data, and then use the last section of data that was read. The code looks something like this:

  self.sock_in.setblocking(0)

# ...

  while True:
    if self.read_socket( self.sock_in ):
      try:
        data_in = json.loads( self.odom_str )
      except JSONDecodeError:
        break
      self.sensor_data = data_in
    else:
      break