redis-py-cluster

redis-py-cluster is a Python client library for talking to a Redis Cluster deployment (Redis 3.0+). It is built directly on top of redis-py, subclassing its Redis client so existing redis-py call patterns keep working, while adding the cluster-specific machinery redis-py itself didn’t originally provide: CRC16-based hash-slot routing (with hash-tag support for multi-key operations), automatic MOVED/ASK redirection handling as the cluster’s slot map changes, and periodic slot-map rediscovery via a dedicated NodeManager.

On top of that base it layers a cluster-safe pipeline implementation that batches commands per-node rather than per-connection, a cluster-aware pub/sub client, and optional read-only routing to replica nodes. The project’s own README now describes it as end-of-life: its functionality was ported upstream into redis-py itself starting with redis-py 4.1.0 in December 2021, and the maintainers recommend migrating there. It remains useful as a reference implementation and for projects still pinned to redis-py’s 3.x line.

What You Get

• A RedisCluster client with the same call surface as redis-py’s Redis, so most existing redis-py code ports over with minimal changes
• Automatic cluster slot-map discovery and refresh via NodeManager, plus CRC16 hash-slot key routing with hash-tag support for multi-key commands
• Transparent MOVED/ASK redirection and cluster-down handling through a dedicated exception hierarchy and custom response parser
• A cluster-safe pipeline (ClusterPipeline) that batches and dispatches commands per-node instead of per-connection
• Cluster-aware pub/sub and optional read-only routing to replica nodes for read scaling

Common Use Cases

• Connecting a Python application to a sharded Redis Cluster deployment instead of a single Redis instance
• Migrating an existing redis-py-based codebase to a clustered Redis backend with minimal API changes
• Running pipelined batches of commands safely across multiple cluster nodes
• Building services that need read-scaling by routing reads to cluster replica nodes

Under The Hood

Architecture The library is organized as a thin cluster-topology layer over redis-py: RedisCluster (client.py) subclasses redis-py’s Redis class, delegating command execution to a ClusterConnectionPool (connection.py) whose connections are opened against nodes tracked by NodeManager (nodemanager.py). NodeManager builds and periodically refreshes a 16384-slot map using CRC16 hashing (crc.py) so each key can be routed to its owning node, and a custom ClusterParser/exception hierarchy (exceptions.py) intercepts MOVED, ASK, TRYAGAIN, and CLUSTERDOWN responses to trigger redirection or slot-map reinitialization. ClusterPipeline (pipeline.py) reimplements redis-py’s pipelining to batch commands per target node rather than per single connection, and ClusterPubSub (pubsub.py) extends pub/sub to work across the cluster’s nodes. The layering is straightforward and traceable, though it depends tightly on redis-py 3.x’s internal shapes (Encoder, DefaultParser, _compat helpers), which is exactly the coupling that made upstream merging into redis-py 4.x feasible.

Tech Stack Written for Python 2.7 and 3.5+ (per setup.py’s python_requires), with a single pinned runtime dependency, redis>=3.0.0,<4.0.0, and an optional hiredis extra for the faster C response parser. Packaging uses classic setuptools/distutils via setup.py, tests run under pytest (declared in dev-requirements.txt) driven by tox.ini across environments, and CI runs on Travis CI against real Redis Cluster nodes rather than a mocked backend. There is no async/await support and no type-hint coverage; the code targets the pre-4.0 redis-py generation throughout.

Code Quality The tests/ directory mirrors redis-py’s own suite closely, with cluster-specific counterparts for each area (test_commands_cluster.py, test_pipeline_cluster.py, test_pubsub_cluster.py, test_lock_cluster.py, test_scripting_cluster.py, test_multiprocessing_cluster.py, and more), run as integration tests against real cluster nodes (conftest.py accepts a --redis-url option rather than relying purely on mocks). Error handling is explicit and typed through a dedicated exception hierarchy (RedisClusterException, MovedError, AskError, ClusterDownError, SlotNotCoveredError) rather than swallowing failures. Naming conventions closely track upstream redis-py. There is no static type checking and only a minimal pycodestyle style configuration in setup.cfg — reasonable for its era, but dated by current standards.

API Design The public API deliberately mirrors redis-py’s own Redis client — you construct a RedisCluster from a list of startup_nodes (or a single host/port) and call the same command methods you’d use on a single-node client, which keeps the learning curve low for anyone already using redis-py. Extensive Sphinx documentation (docs/) covers cluster setup, pipelines, pub/sub, read-only mode, and upgrading between versions, and the examples/ directory has runnable scripts for the common connection patterns (plain, password-protected, ElastiCache, from-URL). The one piece of friction is that its README now opens with an end-of-life notice directing users to redis-py’s native cluster support instead, so new adopters need to weigh that against the library’s otherwise low integration friction.