Using the memcached protocol and library to interface with kestrel

Question

EDIT

I've moved the questions to the top. Ill leave the description of the problem for search help and any background info that someone might want.

If you use a memcached library with kestrel, won't the memcached hashing algorithm possibly always look in the wrong place if you use 2+ servers in the cluster and leverage the reliable read features (or any other features)? Do you have to change the hashing algorithm in the memcached library? Am I missing something? Does anyone have any insight?

BACKGROUND INFO

Kestrel users boast that you can use any memcached library to connect to your kestrel cluster to pop and push items off and on the queue. After thinking about this, it seems flawed. Memcached works in a cluster without inter-server communication because the client determines where the key is stored or where to store it, based on the consisting hashing algorithm.

The kestrel documentation talks about how kestrel is "mostly fair" because the client connects to a random kestrel node to read from or write to the queue. If you use a memcached client, your client would always look in the same spot for the queue, because memcached's client uses a consistent hashing algorithm. Obviously, if you're only using a single kestrel server I your cluster, it doesn't matter, there's only once place to look. Even if you use multiple nodes, it might be okay because you're accessing a static queue name, so the hashing algorithm always looks in the same place.

However, extra features are exposed with kestrel that are interacted with by modifying the queue name that you're accessing from the client (a reliable read is initiated by adding /open, and concluded with /close). This should theoretically cause the client to consistently look in the wrong location for the queue, and would never retrieve a queue object, because they're written to a single node, consistently, and being read from a Different node, consistently.

Thanks!

Answer 1

Disclaimer: I didn't work with kestrel but it sounds good and I did some digging around for your question. Quoting from the kestrel docs:

Clients have a list of all servers in the cluster and pick one at random for each operation

It looks like you are right, you should probably just create your own server picking algorithm (this can easily be done in an custom class) and just connect to one server and use that one for one operation(set/get/open+close/monitor+confirm) then switch to another one if you want. If the memcached client would switch server for a /open and /close this would really screw up your transaction so it's not really safe, you have to make sure you use the same server inside a kestrel operation and not a memcached one. If you want to use monitor and confirm you will have to write your own client class anyway so it doesn't really matter what the memcached client library dose.