Pyro4 does not allow more than two clients to access one URI

Question

I am creating a turn based strategy game in Python using pygame. I found writing sockets incredibly difficult, so I turned to Pyro for sharing the state of the game board. However, Pyro seems unable to support more than 2 connections at a time.

I am running a nameserver on localhost via

python -m Pyro4.naming

Test case 'server':

import Pyro4
class Testcase:
    def __init__(self):
        self.values = [1, 2, 3, 10, 20, 30]

    def askvalue(self, i):
        return self.values[i]


daemon = Pyro4.Daemon()
ns = Pyro4.locateNS()

uri = daemon.register(Testcase())
ns.register("thetest", uri)
daemon.requestLoop()

and the clients:

import Pyro4, time

ns = Pyro4.locateNS()

casetester = Pyro4.Proxy("PYRONAME:thetest")

while True:
    print "Accessing remote object:"
    print casetester.askvalue(1)
    print "staying busy"
    time.sleep(10)

Output from the first two clients:

/usr/local/lib/python2.7/dist-packages/Pyro4-4.14-py2.7.egg/Pyro4/core.py:155: UserWarning: HMAC_KEY not set, protocol data may not be secure
  warnings.warn("HMAC_KEY not set, protocol data may not be secure")
Accessing remote object:
2
staying busy
Accessing remote object:
2
staying busy

and repeats

Output from the third client:

/usr/local/lib/python2.7/dist-packages/Pyro4-4.14-py2.7.egg/Pyro4/core.py:155: UserWarning: HMAC_KEY not set, protocol data may not be secure
  warnings.warn("HMAC_KEY not set, protocol data may not be secure")
Accessing remote object:

and hangs.

Output from the fourth, fifth (and presumably all beyond) client:

/usr/local/lib/python2.7/dist-packages/Pyro4-4.14-py2.7.egg/Pyro4/core.py:155: UserWarning: HMAC_KEY not set, protocol data may not be secure
  warnings.warn("HMAC_KEY not set, protocol data may not be secure")

At this stage, I give the nameserver a ^C, and clients 3, 4, ... give this output and crash:

Traceback (most recent call last):
  File "client.py", line 3, in <module>
    ns = Pyro4.locateNS()
  File "/usr/local/lib/python2.7/dist-packages/Pyro4-4.14-py2.7.egg/Pyro4/naming.py", line 323, in locateNS
    raise Pyro4.errors.NamingError("Failed to locate the nameserver")
Pyro4.errors.NamingError: Failed to locate the nameserver

Meanwhile clients 1 and 2 stay busy.

However, breaking one of the active clients will let one of the hung up ones start to operate.

I have tried via "export PYRO_SERVERTYPE = multiplex" to switch away from threading, but this did not change the behavior. The setting for maximum connections seems to be 200. Setting it to 1000 did not resolve my issue either.

I've read that Pyro lacks scalability, but surely I will be able to get to at least 10 connections?

How can I connect more than two clients at a time to a Pyro4 object?

Answer 1

Answering my own question, hopefully this will appear on google swiftly!

This is not a "perfect" answer, but it is a working kludge. The server code stays the same, but the client code becomes:

import Pyro4, time


global ns
global casetester

def connect():
    global ns
    global casetester
    ns = Pyro4.locateNS()
    casetester = Pyro4.Proxy("PYRONAME:thetest")

def disconnect():
    global ns
    global casetester
    del ns
    del casetester


while True:
    print "Accessing remote object:"
    connect()
    print casetester.askvalue(1)
    disconnect()
    print "staying busy"
    time.sleep(3)

extra "Global" all over the place because never assume.

Why does this work? Because I make a connection, access the remote object, and then delete the connections.

I find this solution very ugly, but I will use it until I find the "right" way.

Answer 2

I had a similar problem. I independently found your global solution, but it did not help for long, the bug came back as application scale increased. After much trial and error, I think I may have found the problem now...user error.

According to http://pythonhosted.org/Pyro4/nameserver.html, the nameserver itself is a proxy. So I made sure to use them with a "With" statement like my other pyro proxies"

with Pyro4.locateNS() as ns:
  uri = ns.lookup('Object name')
with Pyro4.proxy(uri) as obj:
  obj.SomeMethod()

Also do the same thing on every reference to a nameserver like around your ns.register('thetest', uri) call

Answer 3

I have exactly the same consideration. But when I run similar code with you, I found that Pyro4 (2014-12) does not have such question.

I tested with 10 client calling it simultaneously. It works quite well.

I am writing this in case it be useful to someone like me.

Server code

import Pyro4
import time
class Testcase:
    def __init__(self):
        self.value = 1

    def askvalue(self, i):
        # Simulate doing some work.
        time.sleep(1)

        self.value += 1
        return self.value


daemon = Pyro4.Daemon()
ns = Pyro4.locateNS()

uri = daemon.register(Testcase())
ns.register("thetest", uri)
daemon.requestLoop()

Client

import Pyro4, time

ns = Pyro4.locateNS()

casetester = Pyro4.Proxy("PYRONAME:thetest")

while True:
    print("Accessing remote object:")
    print (casetester.askvalue(1))
    print ("staying busy")