Python Twisted Socket Server

Question

I've been using this Twisted Python TCP socket server for quite a while now and it's now starting to really get to me. My issue is that I have 350+ (give or take depending on the day and hour) clients on at once and sometimes I'll have major lag. By lag I mean sometimes when in a "room" (it's an avatar chat basically, users have inventories, friends, houses, etc etc) with any amount of users (usually like 10+ users in the room does it) there will usually be 2-5 second "freezes" every minute or so. By freezes I mean no packets are sent and nobody can see anyone move their character, talk, or anything. After it unfreezes it's like all the packets just send at once and you see everyone's messages and stuff.

My socket sever was based off this: https://github.com/jesstess/twisted-network-programming-essentials-examples/blob/master/Chapter11-Testing/chatserver.py

I'm wondering what I can do to prevent lag. I was thinking multi-threading, but how would I go about doing that in something such as the above example? The most clients I ever had online was 451 a few weeks ago, but I don't think that'll happen anytime soon.

If you're confused about my question - basically I'm asking how to make the server better handle ~350 clients and prevent lag/freezes to all clients in a room.

Edit: Unlike that example, I have added this at the bottom:

ServerCL = GameServer(sys.argv[1])
reactor.listenTCP(int(config.get('GameServer1_' + str(sys.argv[1]), 'port')), ServerCL)
reactor.listenTCP(int(config.get('GameServer2_' + str(sys.argv[1]), 'port')), ServerCL)
reactor.run()

Answer 1

These "freezes" could be happening for a variety of reasons. Most likely this is not something within Twisted itself, but something within your application which is taking that 2-5 seconds and preventing the server from doing anything else.

It sounds like you've got a large, complex application, so rather than guessing at what it might be, I suggest you use a tool like twisted_hang to identify where your application is getting stuck; that should give you some idea of what you could change.