asyncore.dispatcher in python: when are the handle_closed and handle_read executed?

Question

There are two files: server.py and client.py, both written with the help of asyncore.dispatcher

Server.py

import asyncore, socket

class Server(asyncore.dispatcher):
    def __init__(self, host, port):
        asyncore.dispatcher.__init__(self)
        self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
        self.bind(('', port))
        self.listen(1)
        print "Waiting for connection..."

    def handle_accept(self):
        socket, address = self.accept()
        print 'Connection by', address
        socket.send("Hello Server")

    def handle_read(self):
        print "Reading..."
        out_buffer = self.recv(1024)
        if not out_buffer:
            self.close()
        print out_buffer

    def handle_closed(self):
        print "Server: Connection Closed"
        self.close()

s = Server('0.0.0.0', 5007)
asyncore.loop()

Client.py

import asyncore, socket

class Client(asyncore.dispatcher):
    def __init__(self, host, port):
        asyncore.dispatcher.__init__(self)
        self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
        self.connect((host, port))
        print "Client Start..."

    def handle_close(self):
        print "Client: Connection Closed"
        self.close()

    def handle_read(self):
        data = self.recv(1024)
        if data:
            print "Received ", data
        self.send("Hello Client")

c = Client('127.0.0.1', 5007)
asyncore.loop()

Result:

Execute server.py:

Waiting for connection...

then client.py:

Client Start...
Received  Hello Server
Client: Connection Closed
Client: Connection Closed

Finally the client.py exited, and there is one more line displayed in the ouput window of server.py and the server keeps running:

Connection by ('127.0.0.1', 58197)

There are something that I cannot understand:

1. Why is the function handle_closed in client.py executed twice?

2. Why isn't the function handle_reading in server.py executed? The client.py has sent message("Hello Client"), but why cannot the server receive it?

3. Why isn't the function handle_closed in server.py executed? I want to execute some codes in server.py when the client exits, but it seems that it does nothing to do handle_closed in server.py?

Answer 1

Asyncore talk

The handle_read() in server.py will never be called.

But why?! It's a server class...

Yes, but Server class uses its socket for listening any non-established connections. Any reads on it go to handle_accept(), where actual channel sockets (connected to some endpoint) should be given to new instance of some dispatcher-inherited class (preferably). In your Server's handle_accept() method sockets got by accept() were local and thus deleted upon exiting this function, so: new connection was accepted, text was sent and after that socket was immediately killed.

Have some read on asyncore module and my answer in other question.

Server

You need to, like I said, make new class for connections in server.py:

class ClientHandler(asyncore.dispatcher):
    def handle_read(self):
        data = self.recv(1024)
        if not data:
            return
        print "Received:", data

    def handle_close(self):
        print "Server: Connection Closed"
        self.close()

Note here that reading don't need to manually close socket when null is recieved - asyncore takes care of properly closing connection.

Then you have to instantiate it in Server when connection is made:

    def handle_accept(self):
        ...
        ClientHandler(socket)

You also made spelling mistake in Server - method's proper name is handle_close. Though it wouldn't be useful. Everything client-connection related is in ClientHandler.

Client

In client.py you just need to modify handle_read():

    if data:
        print "Received ", data

Change to:

    if not data:
        return
    print "Received ", data

Why? Without this send() would be called even when socket is actually closed, resulting in handle_close() being called for second time, by asyncore. Like I said - asyncore takes care of this.

Notes

Now you can write more complex server-side class for connections. You could also learn how OS-level sockets work, so you won't get into trouble.

asyncore itself is a pretty nice wrapper for sockets, but if you want to do higher-lever things, like HTTP or SMTP processing in some event-driven environment, then Twisted library would interest you!

Answer 2

I was having a similar issue today. The handle_close class runs twice because the read function is returning a set of blank data that then causes the handle_write to trigger, when the handle_write sees the blank data, it closes the request a second time.

As stated above, you need to seperate the handle_read and handle_write functions into a seperate 'handler' class. Then, just check the buffer for blank data, if the data is blank just return.

In the code below, I am adding the length to beggining of each request. So, I pull that first in order to get the buffer size. If its blank, The function just returns

class Handler(asyncore.dispatcher_with_send):
  def __init__(self, conn_sock, client_address, server):
    self.SERVER = server
    self.CA = client_address
    self.DATA = ''
    self.out_buffer = ''
    self.BUFFER = 1024
    self.is_writable = False
    # Create with an already provided socket
    asyncore.dispatcher.__init__(self, conn_sock)

  def readable(self):
    return True

  def writable(self):
    return self.is_writable

  def handle_read(self):
    buffer = str(self.recv(8))
    if buffer != '': size = int(buffer)
    else: return
    data = self.recv(size)
    if data:
        self.DATA += data
        self.is_writable = True

  def handle_write(self):
    if self.DATA:
        self.RESPONSE = processRequest(self.DATA)
        dlen = "%08d" % (len(self.RESPONSE)+8,)
        sent = self.sendall(dlen+self.RESPONSE)
        self.DATA = self.RESPONSE[sent:]
        self.RESPONSE = ''
    if len(self.DATA) == 0:
        self.is_writable = False

  def handle_close(self):
    self.close()


class Server(asyncore.dispatcher):
  FAMILY = socket.AF_INET
  TYPE = socket.SOCK_STREAM

  def __init__(self):
    self.HANDLER = Handler
    #check for a specified host
    self.HOST =  "localhost"
    #check for a specified port
    self.PORT = 50007
    #check the queue size
    self.QUEUE = 5
    #set the reuse var
    self.REUSE = False
    #dispatch
    self.dispatch()

  def dispatch(self):
    #init the dispatcher
    asyncore.dispatcher.__init__(self)
    self.create_socket(self.FAMILY, self.TYPE)
    #check for address reuse
    if self.REUSE: self.set_reuse_addr()
    #bind and activate the server
    self.server_bind()
    self.server_activate()

  def server_bind(self):
    self.bind((self.HOST, self.PORT))

  def server_activate(self):
    self.listen(self.QUEUE)

  def fileno(self):
    return self.socket.fileno()

  def serve(self):
    asyncore.loop()

  def handle_accept(self):
    (conn_sock, client_address) = self.accept()
    if self.verify_request(conn_sock, client_address):
        self.process_request(conn_sock, client_address)

  def verify_request(self, conn_sock, client_address):
    return True

  def process_request(self, conn_sock, client_address):
    self.HANDLER(conn_sock, client_address, self.LOG, self.LOGFILE, self)

  def handle_close(self):
    self.close()

if __name__ == '__main__':
    server = Server()
    server.serve()

Answer 3

To have the handle_read() work , you SHOULD define the readable() function . This function must return either True or False . I think adding this function to your code will fix the problem :

def readable(self):
    return True

for some examples and more details visit this link: https://parijatmishra.wordpress.com/2008/01/04/writing-a-server-with-pythons-asyncore-module/