ZooKeeper: Wait-free coordination for Internet-scale systems

buzzword

• wait-free property
• distributed lock services
• FIFO client ordering
• asynchronous linearizability
• linearizable write
• atomic broadcast protocol
  1. Zab
  2. dominated by reads
  3. read locally
• Znode
  1. Regular
  2. Ephemeral
  3. sequential flag
• use zookeeper to implement more powerful primitives
• Herd effect
• Fetching service based on zookeeper
• linearizability vs. serializability

Summary

This paper describes the ZooKeeper as a coordination kernel which provides high-performance service can be used to implement more powerful primitives. The ZooKeeper is inspired by chubby from Google which is a distributed lock manager and provides a similar file system style interface while the Zookeeper differs from it with wait-free property and order guarantees for coordination. The wait-free property and order guarantees enable asynchronous operations and so on makes the client requests pipelined. Moreover, the Zookeeper introduces a leader-based atomic broadcast protocol called Zab which is much like Raft with some tiny differences. The Zookeeper also reduces the read overheads by performing read locally for their workflow in which read is ten to hundreds more than writes. In addition, the paper introduces the watch mechanism which gives a nice interface for building higher abstraction above ZooKeeper and also provides nice performance for delay notifying and powerful functionality.

Above all, the paper first proposes a wait-free coordination service with relax consistency guarantees and introduces some interesting mechanisms like the watch, file system style API and two basic guarantees which supports linearizability and FIFO client order. Then the paper introduces some examples of primitives involving blocking and no-blocking ones and applications based on ZooKeeper. After that, it gives the implementation of request processor for idempotency, atomic broadcast based on Zab and replicated database which much like the MIT 6.826 lab3 while supports locally reads and asynchronous operations.

Strength

• The paper gives us an idea about how to fasten the distributed system and provides a nice consistency. Just like the wait-free property and Sync operations.

• The paper gives us the idea of watches which simplify the efforts of software programmers and also provides a much powerful feature.

• The paper introduces the term universal object, which can be used to build more powerful systems. And it is an important idea in the hierarchy of system architecture.

• Nice combination of wait-free property and ordering guarantees like watches, sessions. It gives a new idea of consistency implementation.

• The fuzzy snapshot is interesting and used with the idempotency.

Weakness

• No proper introduction to their Zab algorithm.

• No introduction to how ordering guarantees are implemented

Improvement

• More introduction to their new algorithm and the differences between Paxos, Raft and Zab

Lesson learned

We can compromise some of the accuracies for the performance just like the Zookeeper. And after that, we should design some mechanisms to solve the rare inconsistency. We can expect the system themselves solve these problems like the Zookeeper uses the ordering guarantees or application code to solve them or like the Gfs which depends on the application code to remove duplications. Moreover. the idea of generic coordination kernel is impressive.