KeyError: 'pop from an empty set' when connecting to a Mongodb replica set (motor, pymongo, Tornado)

Question

I'm connecting to a 3-node replica set running mongoDB in the following manner :

DB = motor.MotorReplicaSetConnection("mongodb://user:pass@ip_address:27017", replicaSet='repl_set_name').open_sync().collection_name

My replica set works nicely when PRIMARY goes down, another one takes over. But when I do this, my Tornado server throws a KeyError: 'pop from an empty set' error upon re-connection. It appears that Tornado/Motor is not able to reconnect to the new PRIMARY node, I'm not sure why. Even if I add/replace host:port to the connection string, I still get no results. Would this have to do with read_preference?

Answer 1

L-R and I have been debugging this privately, I think the strange error message is a consequence of this Tornado bug:

https://github.com/facebook/tornado/issues/651

... which I've worked around with this commit to Motor:

https://github.com/ajdavis/mongo-python-driver/commit/c7c1614bbf27c3a02ac8c582a8815dee59c39280

Based on your tests it sounds like MotorReplicaSetConnection can in fact reconnect to the new primary shortly after it becomes primary, please email me again if that's not the case.