pymemcache

pymemcache is Pinterest’s pure-Python client for memcached, built to be a complete, fast, and dependency-free alternative to older clients like python-memcached and the C-backed pylibmc. It implements the full memcached text protocol over UNIX sockets or TCP (IPv4/IPv6/TLS), with configurable connect and send/recv timeouts, a noreply fast-write mode, and pluggable serialization for storing arbitrary Python objects.

Beyond the basic Client, the library ships a tiered set of wrappers for production use: PooledClient for thread-safe connection pooling, HashClient for distributing keys across a cluster of memcached servers with consistent (rendezvous) hashing, RetryingClient for wrapping any client with configurable retry policy, and FallbackClient for cascading between servers. An ext module adds a purpose-built AWSElastiCacheHashClient for AWS ElastiCache clusters. Since Django 3.2 the built-in Django cache backend is powered by pymemcache, and it remains one of the most widely deployed memcached clients in the Python ecosystem.

What You Get

• A complete implementation of the memcached text protocol, including all standard commands (get/set/add/replace/append/prepend/cas/delete/incr/decr/touch/stats/flush_all)
• Connections over UNIX sockets or TCP (IPv4/IPv6), with optional TLS via ssl.SSLContext and configurable socket keepalive
• PooledClient for thread-safe connection pooling and HashClient for consistent-hash distribution across a memcached cluster
• RetryingClient and FallbackClient wrappers for configurable retry policy and cascading failover between clients
• Pluggable serialization (PickleSerde, CompressedSerde) so values can be arbitrary Python objects, not just bytes
• An AWSElastiCacheHashClient for connecting directly to AWS ElastiCache configuration endpoints

Common Use Cases

• Dropping in as the cache backend for a Django, Flask, or other Python web app that needs a shared session/object cache
• Sharding reads and writes across a cluster of memcached nodes with consistent hashing to minimize cache churn on node changes
• Wrapping a client in RetryingClient so transient network errors during a deploy or node restart don’t bubble up as request failures
• Connecting to an AWS ElastiCache memcached cluster without hand-rolling endpoint discovery and hashing
• Using the noreply flag on high-volume write paths (e.g. counters, rate limiters) where write acknowledgement isn’t needed

Under The Hood

Architecture pymemcache separates cleanly into a transport layer, a distribution layer, and a resilience layer. pymemcache/client/base.py defines the low-level Client (raw socket I/O, protocol framing, STAT_TYPES/VALID_STORE_RESULTS response parsing) and PooledClient (built on pymemcache/pool.py’s generic ObjectPool). pymemcache/client/hash.py’s HashClient composes multiple Client instances behind a RendezvousHash (pymemcache/client/rendezvous.py) to route keys across a cluster, while pymemcache/client/retrying.py’s RetryingClient and pymemcache/fallback.py’s FallbackClient wrap any client to add retry and cascading-failover behavior without touching the underlying transport. pymemcache/client/ext/aws_ec_client.py subclasses HashClient for AWS ElastiCache-specific endpoint handling. This wrapper-composition pattern means resilience and distribution concerns are additive and swappable rather than baked into the base client, so swapping RendezvousHash for a custom hasher or adding a new wrapper doesn’t require touching Client itself.

Tech Stack The library has zero required runtime dependencies, relying only on the Python standard library (socket, ssl.SSLContext, threading, pickle, zlib, collections). It ships full type hints with a py.typed marker and is checked with mypy. Tooling is standard for a mature PyPI library: setup.cfg/setup.py for packaging, tox for running the matrix across Python 3.9–3.13 plus PyPy, flake8 for linting, black for formatting, and Sphinx (docs/) for documentation, published to Read the Docs.

Code Quality The pymemcache/test/ directory is extensive relative to the library’s size — test_client.py alone covers the bulk of Client/PooledClient behavior, with dedicated suites for hashing (test_client_hash.py), retry logic (test_client_retry.py), rendezvous hashing (test_rendezvous.py), serialization (test_serde.py, test_compression.py), and a benchmark suite (test_benchmark.py). CI (.github/workflows/ci.yml) runs unit and integration tests against real memcached and TLS-memcached Docker services across six Python interpreters, plus separate lint (flake8) and mypy tox environments, with CodeQL security scanning on every push. Errors are raised as a typed hierarchy (MemcacheClientError, MemcacheServerError, MemcacheUnknownError, etc.) rather than swallowed.

API Design pymemcache’s tiered client design (Client → PooledClient → HashClient, each wrappable in RetryingClient or FallbackClient) lets a caller opt into exactly the amount of complexity their deployment needs — a single-server script uses Client directly, a clustered production service composes HashClient with RetryingClient — without a divergent API between tiers, since wrappers preserve the same method surface as the client they wrap. The library documents itself explicitly against alternatives (pylibmc, python-memcached, memcache_client) in its README, and a published migration guide eases moving off python-memcached, which lowers the switching cost for the ecosystem’s most common prior client.