What are the factors that affect eventual consistency over long-haul networks?

Question

I have looked up quite a few resources on eventual consistency and they all speak about why it is important and about Paxos and Byzantine General problem and stuff like that. What I am more interested is in knowing about actually implementing or ensuring eventual consistency over a long haul network - what needs to be done, what the challenges are, possible ways to circumvent and prevent and safeguard against anything that might prohibit eventual consistency. Anything from your personal experiences to links that you might know of would be really useful. Thanks!

Answer 1

Most challenge: keep in mind what you want to achieve. In my experience ec systems has three challenges:

1. Unknown replication level of each key at any point in time. To solve this, various techniques adopted: read-repair, merkle-trees. In case of long haul networks this point can be challenging.

2. In case of strong quorum requirements you will write to distant datacenter, so you write performance will be constrained by link throughput to distant datacenter. And in case of long delays it becomes very likely to have 'lost writes': when client thinks, that write not performed, but actually it is. So it is easy to get situation, when you failed to write to quorum, but other client will see data, considered by you 'not written'

3. Data stailness if you will read with quorum=1 (for better read performance). But here some clever solutions like Probabilistically Bounded Staleness for Practical Partial Quorums exist