How to implement a python REPL that nicely handles asynchronous output?

Question

I have a Python-based app that can accept a few commands in a simple read-eval-print-loop. I'm using raw_input('> ') to get the input. On Unix-based systems, I also import readline to make things behave a little better. All this is working fine.

The problem is that there are asynchronous events coming in, and I'd like to print output as soon as they happen. Unfortunately, this makes things look ugly. The "> " string doesn't show up again after the output, and if the user is halfway through typing something, it chops their text in half. It should probably redraw the user's text-in-progress after printing something.

This seems like it must be a solved problem. What's the proper way to do this?

Also note that some of my users are Windows-based.

TIA

Edit: The accepted answer works under Unixy platforms (when the readline module is available), but if anyone knows how to make this work under Windows, it would be much appreciated!

Answer 1

Maybe something like this will do the trick:

#!/usr/bin/env python2.6

from __future__ import print_function

import readline
import threading

PROMPT = '> '

def interrupt():
    print() # Don't want to end up on the same line the user is typing on.
    print('Interrupting cow -- moo!')
    print(PROMPT, readline.get_line_buffer(), sep='', end='')

def cli():
    while True:
        cli = str(raw_input(PROMPT))

if __name__ == '__main__':
    threading.Thread(target=cli).start()
    threading.Timer(2, interrupt).start()

I don't think that stdin is thread-safe, so you can end up losing characters to the interrupting thread (that the user will have to retype at the end of the interrupt). I exaggerated the amount of interrupt time with the time.sleep call. The readline.get_line_buffer call won't display the characters that get lost, so it all turns out alright.

Note that stdout itself isn't thread safe, so if you've got multiple interrupting threads of execution, this can still end up looking gross.

Answer 2

Why are you writing your own REPL using raw_input()? Have you looked at the cmd.Cmd class? Edit: I just found the sclapp library, which may also be useful.

Note: the cmd.Cmd class (and sclapp) may or may not directly support your original goal; you may have to subclass it and modify it as needed to provide that feature.

Answer 3

run this:

python -m twisted.conch.stdio

You'll get a nice, colored, async REPL, without using threads. While you type in the prompt, the event loop is running.

Answer 4

look into the code module, it lets you create objects for interpreting python code also (shameless plug) https://github.com/iridium172/PyTerm lets you create interactive command line programs that handle raw keyboard input (like ^C will raise a KeyboardInterrupt).

Answer 5

It's kind of a non-answer, but I would look at IPython's code to see how they're doing it.

Answer 6

I think you have 2 basic options:

1. Synchronize your output (i.e. block until it comes back)
2. Separate your input and your (asyncronous) output, perhaps in two separate columns.