MongoDB Java Client Automatic Failover Failing

Question

I have a 3 node replica set setup on localhost:

mongod --port 27017 --dbpath C:/data/repset/rsdb0 --replSet rs0
mongod --port 27018 --dbpath C:/data/repset/rsdb1 --replSet rs0
mongod --port 27019 --dbpath C:/data/repset/rsdb2 --replSet rs0

In my Java client I'm connecting to the replica set:

List<ServerAddress> addrs = new ArrayList<>();
addrs.add( new ServerAddress( "HOST" , 27017 ) );
addrs.add( new ServerAddress( "HOST" , 27018 ) );
addrs.add( new ServerAddress( "HOST" , 27019 ) );

MongoClient mongo = new MongoClient( addrs );
System.out.println(mongo.getReplicaSetStatus());

All works fine unless I take down (stop) the 3rd Secondary mongodb instance (the one on port 27019). This is to simulate a server failure.

Then when I run the above java code I get:

Feb 17, 2014 10:51:18 PM com.mongodb.ConnectionStatus$UpdatableNode update
WARNING: Server seen down: HOST/192.168.0.5:27019
java.net.ConnectException: Connection refused: connect
   at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)

Which seems to mean that the replica set is not failing over. I would expect the client to carry on reading and writing until I bring the 'failed' server back up and then I would expect a re-syncing.

I guess I must be doing something wrong, because automatic failover is fundamental to MongoDB, but can anyone tell me what it is?

Thanks!

Answer 1

Which seems to mean that the replica set is not failing over.

It's not the replica set that should fail over but the driver.

And what tells you it doesn't? The warning just informs you that one of the nodes is in DOWN state. You haven't done any operation yet, hence nothing to fail over from.

Did you try to read or write beyond connecting to the set? I'm pretty much convinced you can.

I guess I must be doing something wrong

No you don't, except for understanding the meaning of the warning

Answer 2

You'll get that error if you use a stale connection. When the driver detects that failed connection, it will repopulate the pool with new connections. If you retry in a few seconds (once any primary election has been sorted out by the server), you'll see that the java driver reconnects to the new primary and is ready for writing.