Why does asyncio.open_connection (or any other coroutine) prevent data_received from processing?

Question

I have the following code slightly modified from the asyncio documentation

import asyncio
asyncio.tasks._DEBUG = True


class EchoServer(asyncio.Protocol):
    def connection_made(self, transport):
        name = transport.get_extra_info('sockname')
        peername = transport.get_extra_info('peername')
        print('Connection:',name,'<--',peername)
        self.transport = transport

    def data_received(self, data):
        name = self.transport.get_extra_info('sockname')
        peername = self.transport.get_extra_info('peername')

        print('Got data:',name,'<--',peername,':',data.decode() )
        if name[1] == 8888:
            print("Making connection")
            reader, writer = yield from asyncio.open_connection('127.0.0.1', 8889, loop=asyncio.get_event_loop())
        else:
            self.transport.write(data)

        self.transport.close()


loop = asyncio.get_event_loop()
coro_1 = loop.create_server(EchoServer, '127.0.0.1', 8888)
coro_2 = loop.create_server(EchoServer, '127.0.0.1', 8889)
server_1 = loop.run_until_complete(coro_1)
server_2 = loop.run_until_complete(coro_2)
print('Serving on {}'.format(server_1.sockets[0].getsockname()))
print('Serving on {}'.format(server_2.sockets[0].getsockname()))

try:
    loop.run_forever()
except KeyboardInterrupt:
    print("^C caught, exiting")
finally:
    server_1.close()
    server_2.close()
    loop.close()

However, when I run this, it doesn't appear to even run the data_received function:

Serving on ('127.0.0.1', 8888)
Serving on ('127.0.0.1', 8889)
Connection: ('127.0.0.1', 8888) <-- ('127.0.0.1', 36580)

But when I comment out the yield from line, it runs the way I would expect.

Serving on ('127.0.0.1', 8888)
Serving on ('127.0.0.1', 8889)
Connection: ('127.0.0.1', 8888) <-- ('127.0.0.1', 36581)
Got data: ('127.0.0.1', 8888) <-- ('127.0.0.1', 36581) : Hi, gaise!
Making connection

I've tried adding:

@asyncio.coroutine
def do_a_thing():
    print("hey, it's a thing!")

def try_stuff():
    print('Trying')
    yield from asyncio.async(do_a_thing())

and calling try_stuff instead. This does output "Making connection", but not 'Trying'. By changing the yield from line to a simple call does output 'Trying'. And if I remove the coroutine decorator from do_a_thing then I get the output.

The important thing to me is that I'm trying to create an async connection to the 2nd server. But evidently, the presence of the yield from prevents any further processing (at least that I can tell - it certainly stops with the print).

So what am I doing wrong, and how can I fix this?

Answer 1

yield from must be used in a coroutine. As I can see, data_received can't be a coroutine because of some asyncio internals.

A solution is to wrap your code in a coroutine and call it using a asyncio.Task().

As an example you can have a look at my experiments

Answer 2

As gawel mentioned, yield from must be used in a coroutine, and despite the fact that the Server class is within some asyncio calls, the functions are not magically coroutines.

EchoServer needs to look like this:

class EchoServer(asyncio.Protocol):
    def connection_made(self, transport):
        name = transport.get_extra_info('sockname')
        peername = transport.get_extra_info('peername')
        print('Connection:',name,'<--',peername)
        self.transport = transport

    # This is where the magic happens
    @asyncio.coroutine
    def send_data(self, data):
        print("Making connection")
        reader, writer = yield from asyncio.open_connection('127.0.0.1', 8889)
        print("Connection made")
        writer.write(data)
        print("Data sent")
        # For some reason, not providing n caused this to not return anything
        resp = yield from reader.read(1024)
        print("Got response")
        self.transport.write(resp)
        print('Data returned:',resp)

    def data_received(self, data):
        name = self.transport.get_extra_info('sockname')
        peername = self.transport.get_extra_info('peername')

        print('Got data:',name,'<--',peername,':',data.decode() )
        if name[1] == 8888:
            print("making task")
            asyncio.Task(self.send_data(data))
            print("done making task")
        else:
            print("Sending response")
            self.transport.write(data[::-1])