kazoo

Kazoo is a high-level Python client for Apache ZooKeeper that reimplements the ZooKeeper wire protocol in pure Python rather than wrapping the notoriously buggy native C bindings. Built and maintained by teams originally from the Nimbus Project, Mozilla, and Zope Corporation over more than a decade, it gives Python applications a stable, typed interface to ZooKeeper’s low-level znode API.

Beyond the base client, kazoo ships implementations of distributed locks, leader election, counters, barriers, party membership, and partitioners, so teams building distributed systems don’t have to re-derive these primitives from ZooKeeper’s raw watch model. Its pluggable handler abstraction lets the same client run on plain threads, gevent, or eventlet, making it usable in both blocking and cooperative-concurrency codebases.

What You Get

• KazooClient with automatic reconnection, retry handling, and connection-state listeners
• Distributed recipes: Lock, ReadLock, WriteLock, Semaphore, Election, Barrier, DoubleBarrier, Counter, Party, SetPartitioner, and Queue
• Pluggable handler backends (threading, gevent, eventlet) selectable per client
• DataWatch and ChildrenWatch decorators for reactive znode monitoring
• Full type hints (py.typed) validated under mypy strict mode

Common Use Cases

• Coordinating leader election across a fleet of worker processes
• Building a distributed mutual-exclusion lock for a scheduled job
• Watching configuration znodes for live config reloads
• Distributing work across a dynamic set of consumer nodes with a partitioner

Under The Hood

Architecture KazooClient (kazoo/client.py) sits atop a ConnectionHandler (kazoo/protocol/connection.py) that implements the ZooKeeper wire protocol directly, exchanging requests serialized in kazoo/protocol/serialization.py, with session/connection state modeled by the KazooState/KeeperState enums in kazoo/protocol/states.py and dispatched to pluggable handler backends in kazoo/handlers/{threading,gevent,eventlet}.py, each of which runs three separate sequential queues (session, watch, completion callbacks) to prevent deadlocks when application callbacks re-enter the client. The recipes package (kazoo/recipe/*) builds coordination primitives like Lock, Election, and SetPartitioner purely from watches and znode operations exposed by KazooClient, staying decoupled from any specific handler implementation via the shared interface in interfaces.py — a change to the wire-protocol layer would ripple through every recipe and the retry/backoff logic in retry.py, but the recipes themselves would not need to know which handler backend is in use.

Tech Stack The library is pure Python (99% of the codebase) and depends only on typing-extensions for the base install; optional extras pull in pytest, pytest-cov, pytest-timeout, gevent, eventlet, pyjks, and pure_sasl for SASL/Kerberos-authenticated testing. Documentation is built with Sphinx and sphinx-autodoc-typehints and published to Read the Docs; typing is enforced with mypy in strict mode (configured in pyproject.toml), alongside black and flake8 for style. The package is built with setuptools, version-sourced from kazoo/version.py, and CI (.github/workflows/testing.yml) exercises Python 3.8 through 3.14 plus PyPy against multiple real ZooKeeper server versions via ensure-zookeeper-env.sh.

Code Quality kazoo/tests/ contains roughly two dozen test modules covering nearly every subsystem (test_client.py, test_connection.py, test_lock.py, test_election.py, test_partitioner.py, test_sasl.py) plus handler-specific suites (test_threading_handler.py, test_gevent_handler.py, test_eventlet_handler.py), backed by a testing harness (kazoo/testing/harness.py) that spins up real ZooKeeper server instances for integration tests rather than mocking the protocol. CI runs a required validate job (black, flake8, mypy strict) before tests execute, the package ships py.typed with comprehensive annotations throughout client.py and the recipes, and errors are surfaced through a dedicated exceptions.py hierarchy (NoNodeError, NodeExistsError, ConnectionLoss, SessionExpiredError) rather than generic exceptions.

API Design KazooClient exposes a clean surface — zk.start(), zk.get(), zk.ensure_path(), zk.Lock(path) — that hides ZooKeeper’s raw znode/watch model behind recipe classes usable directly as context managers (with zk.Lock(path):), and DataWatch/ChildrenWatch work either as decorators or plain function calls, so most application code never touches protocol/serialization.py directly. The trade-off is a real learning curve: correctly using the recipes still requires understanding ZooKeeper’s session/watch semantics, but extensive module-level docstrings, a documented implementation-details guide, and full Sphinx docs on Read the Docs substantially lower that barrier once the underlying model is understood.