How to solve twisted.internet.error.CannotListenError: Couldn't listen on any:8081: [Errno 98] Address already in use [duplicate]

Question

This question already has answers here:

Python: Binding Socket: “Address already in use” (12 answers)

Closed 5 years ago.

I have a twisted python server which runs on port 8080, and i have written different API's which runs on this server. so i want all these API's to run on a single port no. but when i try to use same port all API's for ex : 8081 and run at the same time using python interpreter. at that time i am getting this error : twisted.internet.error.CannotListenError: Couldn't listen on any:8081: [Errno 98] Address already in use. As i am new to twisted so don't know much of the things and there is no proper documentation on twisted . please someone guide me to solve this error :

Here is the Code snippets :

from twisted.internet import epollreactor

epollreactor.install()

from zope.interface import implements
from twisted.internet import reactor,interfaces
from functools import partial
from pymongo import Connection
import json
from bson.objectid import ObjectId
import server_1
import replacePlus_Space

global data, data_new, datadB, coll_auth, coll_person


class Login_user(server_1.HTTPEchoFactory):

    def __init__(self):
        server_1.HTTPEchoFactory.__init__(self,"testsite")
        server_1.HTTPEchoFactory.initResources(self)
        self.initResources()

    def initResources(self):
        print "in Login"
        self.responses["/login_user"] = partial(self.user_login)


    # To connect to DB and Check error cases and insert into mongoDB..!!!
    def user_login(self, client):
        # some functinality..!!!!

d = Login_user()

reactor.listenTCP(8081,d)

reactor.run()         

Second code snippet is :

from twisted.internet import epollreactor

epollreactor.install()

from zope.interface import implements
from twisted.internet import reactor,interfaces
from functools import partial
from pymongo import Connection
import json
from bson.objectid import ObjectId
import server_1
import replacePlus_Space

class AddFriendDB(server_1.HTTPEchoFactory):

    def __init__(self):
        server_1.HTTPEchoFactory.__init__(self,"testsite")
        server_1.HTTPEchoFactory.initResources(self)
        self.initResources()

    def initResources(self):
        print "in add friend"
        self.responses["/addFriend_DB"] = partial(self.addFriendDB)


    # To connect to DB and Check error cases and insert into mongoDB..!!!
    def addFriendDB(self, client):
        #some functionality here..!!!

d = AddFriendDB()

reactor.listenTCP(8081,d)

reactor.run()

Answer 1

Translating the question

Many of the specifics in your question don't make sense, so I'm translating your question into:

I have a twisted python standalone program which runs on port 8080, and i have written a different standalone program which runs on this server. I want both programs to run on a single port no. When i try to use same port for all programs ex : using port 8081 for both programs. I am getting this error :twisted.internet.error.CannotListenError: Couldn't listen on any:8081: [Errno 98] Address already in use. As i am new to twisted i don't know much of the things and there is no proper documentation on twisted. please someone guide me to solve this error.

Twisted has great documentation

You commented that:

[...] there is no proper documentation on twisted

That statement is blatantly false.

Twisted certainly has proper documentation, and as compared to most frameworks it has excellent documentation. To name just a few of the many high quality documentation sources:

1. Dave Peticola's (krondo) Twisted Introduction - an excellent and deep introduction to twisted that begins by explaining the technology Twisted is built upon. If you really want to understand twisted, this (long) introduction is the place to start
2. The high quality and extensive documentation on on Twisted's primary website twistedmatrix.com
3. The source itself. Twisted has well commented and surprisingly understand source, if the above documentation doesn't teach you what you need figure it out from the code.

What causes "Address already in use"

As previously answered by Bart Friederichs:

Only one process can listen on a certain port. If you start process 1 to listen on port 8081, and then start another process on that same port, you get this error.

It is an error from the TCP stack, not from python or twisted.

This is a fundament truth of TCP/IP stacks across all operating systems (or at least all process based operating systems that could run Python).

The error is your operating system reminding you that when data comes to a IP port the OS/IP-stack was designed to only forward to one process, a process that is implementing an application level protocol on that port. The error happens when a second program attempts to re-register a port some other program has already registered.

Work-arounds in general

Upon running into a port reuse issue like this you have to ask yourself:

1. Are the two programs even running the same application-level protocol?
2. If they are the same protocol, does the protocol have routing such that for any given transaction the correct sub-program/routine be identified?
3. Are the two programs at a different location in the routing?

If they aren't the same protocol (I.E. one was HTTP and the other is FTP) or they are the same protocol but its a protocol that doesn't have routing (I.E. two instances of NTP) then there is no easy way to mix them because your trying to break the laws of IP (and the way that application protocol implementations are built). The only solution will be to encapsulate one (or both) of the protocol(s) into a common protocol that also has application-level routing (I.E. encapsulating FTP inside of HTTP and using the URI to route)

If they are the same protocol, and the protocol provides for a per-transaction routing (I.E. the URI inside of HTTP transactions) and they aren't at the same routing location, then there are easier solutions, namely:

1. Merge the two application into one.

   If they are the same routable protocol but at a different location (I.E. HTTP protocol with the URI for the first program at /login and second program at /addfriend) it should be trivial to pull out the post-routing logic of the two programs/process and merge them into one program that does both functions

2. Front-end the programs with a redirector (This solution is only easy with HTTP because of the tools available).

   If you have HTTP protocol programs that have routing to separate locations, but for some reason its hard to merge the logic together (I.E. one program is written in Java, the other in Python) then you can look at front-ending both programs with a redirector like nginx

   The way you would use a redirector in a case like this is to run your two apps on two different unused ports (I.E. 12001, 12002), and then run the redirector on the port you want the service to be on, running it with a config file that will redirect to your two programs via their unique routing locations (via configs like SF: How to redirect requests to a different domain/url with nginx)

Code example

The following three programs illustrate the process of merging two programs into one, so both sets of logic can be accessed from the same port.

Two separate programs:

If you run the following code a web server will be started up on localhost:8081. If you then point your web browser at http://127.0.0.1:8081/blah the blah page will be displayed.

#!/usr/bin/python

from twisted.internet import defer, protocol, reactor # the reactor

from twisted.web.server import Site # make the webserver go
from twisted.web.resource import Resource

class BlahPage(Resource):
    idLeaf = True
    def render_GET(self, request):
        return "<html><body>Blah Page!</body></html>"

class ShutdownPage(Resource):
    def render_GET(self, request):
        reactor.stop()

webroot = Resource()
webroot.putChild("blah", BlahPage())
webroot.putChild("shutdown", ShutdownPage())

def main():
    # Register the webserver (TCP server) into twisted
    webfactory = Site(webroot)
    reactor.listenTCP(8081, webfactory)

    print ("Starting server")
    reactor.run()


if __name__ == '__main__':
  main()

This code will start a web server on localhost:8082. If you then point your web browser at http://127.0.0.1:8082/foo the foo page will be displayed.

#!/usr/bin/python

from twisted.internet import defer, protocol, reactor # the reactor

from twisted.web.server import Site # make the webserver go
from twisted.web.resource import Resource

class FooPage(Resource):
    idLeaf = True
    def render_GET(self, request):
        return "<html><body>Foo Page!</body></html>"

class ShutdownPage(Resource):
    def render_GET(self, request):
        reactor.stop()

webroot = Resource()
webroot.putChild("foo", FooPage())
webroot.putChild("shutdown", ShutdownPage())

def main():
    # Register the webserver (TCP server) into twisted
    webfactory = Site(webroot)
    reactor.listenTCP(8082, webfactory)

    print ("Starting server")
    reactor.run()


if __name__ == '__main__':
  main()

Merging the logic

This code is the merger of the two previous programs, as you can see it only required copying a small amount of code to glue both of the above into one that allows for access to http://127.0.0.1:8080/blah and http://127.0.0.1:8080/blah.

#!/usr/bin/python

from twisted.internet import defer, protocol, reactor # the reactor

from twisted.web.server import Site # make the webserver go
from twisted.web.resource import Resource

class BlahPage(Resource):
    idLeaf = True
    def render_GET(self, request):
        return "<html><body>Blah Page!</body></html>"

class FooPage(Resource):
    idLeaf = True
    def render_GET(self, request):
        return "<html><body>Foo Page!</body></html>"

class ShutdownPage(Resource):
    def render_GET(self, request):
        reactor.stop()

webroot = Resource()
webroot.putChild("foo", FooPage())
webroot.putChild("blah", BlahPage())
webroot.putChild("shutdown", ShutdownPage())

def main():
    # Register the webserver (TCP server) into twisted
    webfactory = Site(webroot)
    reactor.listenTCP(8080, webfactory)

    print ("Starting server")
    reactor.run()


if __name__ == '__main__':
  main()

Answer 2

Only one process can listen on a certain port. If you start process 1 to listen on port 8081, and then start another process on that same port, you get this error.

It is an error from the TCP stack, not from python or twisted.

Fix it either by choosing different ports for each process, or create a forking server.