Building asynchronous socket tutorial: how do I connect to listening socket in python?

Question

python 2.6 Windows 7

I am trying to put together an as simple as possible tutorial of how to write cooperative multitasking programs. As an example application I've written a chat server with python's asyncore backend. I think this will be a valuable resource for the community. However, I have not gotten it to work yet, hence this post.

The structure is as follows. An instance of ChatServer runs on a remote computer. It's socket listens on REMOTE_PORT. When it detects an incoming connection, it spawns an instance of ChatHandler to mediate communication with that connection. Now, who is that connection? On the user's local machine we run an instance of ChatDaemon. This guy listens on LOCAL_PORT. When you connect to him like this

import socket
s = socket.socket()
s.connect(('localhost',LOCAL_PORT))

he detects the connection and spawns two things, a LocalListener and a Connection. The Connection connects to the server, answering our question from above. The LocalListener simply waits for data coming in from the user. If you send data

s.send("Hello, world!")

the LocalListener picks it up, and gives it to the Connection, which then sends it to the ChatServer. The server then puts the data in each ChatHandler's buffer to be sent out to all connected clients. When the Connection receives that data, it passes it to the Daemon who prints it to the screen.

(The Daemon layer seems overly complex but without it you have to do other complicated things to prevent hot loops in asyncore's select() loop whilst keeping latency for the user sending data low. I don't want to go down that road.)

The problem is that the connection to the Daemon does not seem to be made. My exact steps are

In one python session

d = ChatDaemon('localhost')
d.start()

When I do this I see the message "Chat daemon binding to 'localhost: 7668' as expected.

In another python session

import socket
s = socket.socket()
s.connect(('localhost',7668))

When I do this, I do not see the "Got new local connection" line printed.

I have edited my etc/hosts file to map 'localhost' to 127.0.0.1, and I installed the Microsoft Loopback adapter.

EDIT: I have found and fixed the problem. The code below should now be acceptable as a very simple chat implementation using asyncore.

Here is the source

import asyncore
import socket
import sys

LOCAL_HOST = 'localhost'
LOCAL_PORT = 7668
REMOTE_HOST = 'localhost'
REMOTE_PORT = 7667

class LocalListener(asyncore.dispatcher):
    """Receive data from user, putting into cxn's buffer"""
    def __init__(self, sock, cxn):
        self.cxn = cxn
        asyncore.dispatcher.__init__(self, sock)

    def writable(self):
        return False

    def readable(self):
        return True

    def handle_read(self):
        data = self.recv(4096)
        if data:
            self.cxn.buf = self.cxn.buf + data

class Connection(asyncore.dispatcher):
    """Mediates between user and server"""
    def __init__(self, host, port, master):
        asyncore.dispatcher.__init__(self)
        self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
        self.connect((host,port))
        self.buf=""

    def writable(self):
        return len(self.buf) > 0

    def readable(self):
        return True

    def handle_read(self):
        data = self.recv(4096)
        if data:
            self.master.newMessage(data)

    def handle_write(self):
        sent = self.send(self.buf)
        self.buf = self.buf[sent:]

class ChatDaemon(asyncore.dispatcher):
    """Listen for local connections and dispatch in/out data"""
    ADDRESS_FAMILY = socket.AF_INET
    SOCKET_TYPE = socket.SOCK_STREAM
    def __init__(self, remoteHost, remotePort=REMOTE_PORT,
                 localHost=LOCAL_HOST, localPort=LOCAL_PORT):
        self.remoteHost = remoteHost
        self.remotePort = remotePort
        self.localHost = localHost
        self.localPort = localPort
        self.buffer = ""
        asyncore.dispatcher.__init__(self)

    def writable(self):
        return False

    def readable(self):
        return True

    def newMessage(self, data):
        print data

    def start(self):
        """Listen for user connection on local port"""
        self.create_socket(self.ADDRESS_FAMILY, self.SOCKET_TYPE)
        print("Chat deamon binding to '%s': %s"%(self.localHost,self.localPort))
        self.bind((self.localHost,self.localPort))
        self.listen(1)
        asyncore.loop()

    def handle_accept(self):
        """Spawn local reader and remote reader/writer"""
        print "Got new local connection"
        (connSock, localAddress) = self.accept()
        print("New connection address is %s"%localAddress)
        #Make a server connection
        cxn = Connection(self.remoteHost, self.remotePort, self)
        #Connect to local user
        LocalListener(connSock, cxn)

### SERVER ###

class ChatHandler(asyncore.dispatcher):
    def __init__(self, sock, map, server):
        self.server = server
        self.buffer = ''
        asyncore.dispatcher.__init__(self, sock, map)

    def writable(self):
        return len(self.buffer) > 0

    def readable(self):
        return True

    def handle_read(self):
        """Notify server of any new incoming data"""
        data = self.recv(4096)
        if data:
            self.server.newMessage(data, self)

    def handle_write(self):
        """send some amount of buffer"""
        sent = self.send(self.buffer)
        self.buffer = self.buffer[sent:]

class ChatServer(asyncore.dispatcher):
    """Receive and forward chat messages

    When a new connection is made we spawn a dispatcher for that
    connection.
    """
    ADDRESS_FAMILY = socket.AF_INET
    SOCKET_TYPE = socket.SOCK_STREAM
    def __init__(self, host=REMOTE_HOST, port=REMOTE_PORT):
        self.map = {}
        self.address = (host,port)
        self.clients = []
        asyncore.dispatcher.__init__(self, map=self.map)

    def serve(self):
        """Bind to socket and start asynchronous loop"""
        self.create_socket(self.ADDRESS_FAMILY, self.SOCKET_TYPE)
        self.bind(self.address)
        self.listen(1)
        asyncore.loop(map=self.map)

    def writable(self):
        return False

    def readable(self):
        return True

    def newMessage(self, data, fromWho):
        """Put data in all clients' buffers"""
        for client in self.clients:
            client.buf = client.buf + data

    def handle_accept(self):
        """Deal with newly accepted connection"""
        print 'got new connection'
        (connSock, clientAddress) = self.accept()
        self.clients.append(ChatHandler(connSock, self.map, self))

Answer 1

The problem was that in ChatDaemon I forget the "return" keywords in readable and writable