Hyperledger Fabric: Does it really come with consensus?

Question

As I understand, consensus is the mechanism that will ensure the ledger is kept in sync across nodes. Is that correct? Also its nothing new and wasn't something that was invented with the advent of blockchain. It is an integral part of distributed databases and has been around for decades. Many algorithms exist to drive consensus - Paxos, Raft, (P)BFT etc.

I am trying to understand if Hyperledger Fabric really comes with consensus protocol/mechanism. The docs say the orderer establishes consensus and currently its implemented via Apache Kafka.

But when I think about it a little deeper, Apache Kafka is not something that provides consensus. It is a pub-sub service. I have never heard of Kafka in the context of consensus. When people talk of consensus they mention terms like BFT, Paxos, Raft etc. not Kafka. Kafka is powered by Zookeeper which indeed is a technology to provide consensus. But the consensus Zookeeper will provide is among the Kafka nodes, not the nodes of the blockchain!

Indeed quoting this paper p.3 Section 1:

An ordering service atomically broadcasts state updates to peers and establishes consensus on the order of transactions

I agree with this - consensus is established on the order of transactions. But where is the consensus to ensure the ledger is in sync across peer nodes? Is that not needed? And if so then why are Fabric developers working to implement Raft consensus [1] - what is the reason and motivation behind it?

Answer 1

But where is the consensus to ensure the ledger is in sync across peer nodes?

That follows from the fact that if all the committing nodes are seeing the same messages and coming in the same exact order, then they will execute the same actions/commits and so will be in sync. Thus assuming:

1. All the committing peers start from the same state .
2. They see the same messages .
3. All of them are executing the same predictable code where input uniquely determines the output

It follows then that they will be in sync given all of them see same messages in same order.

Is that not needed?

Yes, no special additional sauce needed. See above.

And if so then why are Fabric developers working to implement Raft consensus [1] - what is the reason and motivation behind it?

refer to the work item for details. Raft still keeps us in CFT-land and as such, it is still not the optimal solution (BFT is, and that's also coming), but –

It allows us to have an ordering service without Kafka/ZK depedencies, and It helps with the BFT development as well, since it gives us a chance to hook into Fabric a leader-based consensus protocol and carry those lessons over when writing the BFT plugin.