async-lru

async-lru is a small, focused port of Python’s built-in functools.lru_cache for asyncio coroutines. Decorating a coroutine function with @alru_cache gives it the same maxsize/typed cache semantics as the stdlib decorator, but adapted for async code: concurrent calls with the same arguments share a single in-flight task instead of triggering duplicate work, and awaiters all receive the same result once it completes.

Beyond the basic port, it adds features functools.lru_cache doesn’t have: TTL-based expiration with optional jitter to avoid synchronized cache-stampede expiry, a pluggable key callable for deriving cache keys from a subset of arguments, explicit cache_invalidate()/cache_contains() methods, and an event-loop-affinity guard that safely resets the cache instead of silently corrupting state if it’s reused across event loops. Maintained by the aio-libs organization, the same group behind aiohttp.

What You Get

• An @alru_cache decorator with the same maxsize/typed interface as functools.lru_cache, usable on plain functions, bound methods, partials, and partialmethods
• Automatic de-duplication of concurrent calls with identical arguments — all callers share one underlying task and its result via asyncio.shield()
• Optional ttl and jitter parameters for time-based expiration that spreads out simultaneous cache-entry invalidations
• A custom key callable option so cache keys can be derived from a subset of arguments instead of the full call signature
• cache_invalidate(), cache_contains(), cache_clear(), cache_info(), and cache_parameters() for explicit cache introspection and control
• An async cache_close() to cancel or drain in-flight cached tasks on shutdown, plus a loop-change guard that clears and warns instead of corrupting state

Common Use Cases

• Caching results of async HTTP calls or database queries within a single event loop’s lifetime
• Deduplicating bursts of concurrent requests for the same resource (e.g. many coroutines awaiting the same PEP lookup or API call at once)
• Adding TTL-based caching to async lookups where staleness needs to be bounded, without a separate cache service
• Memoizing expensive async computations keyed on a subset of arguments via the custom key callable

Under The Hood

Architecture The entire library lives in a single file, async_lru/__init__.py (492 lines). The core class _LRUCacheWrapper implements __call__, wraps a coroutine function, and holds cache state in an OrderedDict[Hashable, _CacheItem], where _CacheItem is a @dataclasses.dataclass(slots=True) bundling the backing asyncio.Task, an optional TTL later_call handle, and a waiter count. The descriptor protocol (__get__) branches into a separate _LRUCacheWrapperInstanceMethod for bound methods, delegating back to the wrapper rather than duplicating logic. The alru_cache() public factory uses @overload to support both bare (@alru_cache) and parameterized (@alru_cache(maxsize=32, ttl=5)) usage, funneling both into _make_wrapper. The trickiest data flow is in __call__ and _shield_and_handle_cancelled_error: concurrent callers with the same key await the same task via asyncio.shield(), and cancellation is handled so only the last waiter actually cancels the underlying task and evicts the cache entry. A _check_loop guard detects event-loop changes and safely clears state rather than reusing tasks bound to a dead loop. If this method’s task/waiter bookkeeping broke, every caller’s dedup/eviction/cancellation behavior would break with it, since there’s no seam between it and consumers.

Tech Stack Pure Python with no runtime dependencies beyond typing_extensions>=4.0.0 for Python <3.11 (per setup.cfg). Supports Python 3.10 through 3.14 per its trove classifiers, built on stdlib asyncio, collections.OrderedDict, dataclasses(slots=True), and directly reuses CPython’s own functools._make_key/_CacheInfo internals rather than reimplementing key hashing. Packaged with setuptools (setup.py + setup.cfg, packages=find:) and ships a py.typed marker for PEP 561 typing support. Dev tooling includes pytest with pytest-asyncio (asyncio_mode=auto), coverage reporting, mypy --strict across both the library and its test suite, flake8/isort enforced via pre-commit, and CodSpeed for continuous performance-regression benchmarking. CI (GitHub Actions) runs the test matrix across Python 3.10-3.14 plus PyPy 3.11, across Ubuntu/macOS/Windows, alongside a dedicated CodeQL security-scanning workflow.

Code Quality The test suite spans 17 files and roughly 1,674 lines covering basic caching behavior, cache_clear/cache_contains/cache_info/cache_invalidate, cancellation semantics, cache_close shutdown behavior, deferred (PEP 649) annotation support, exception propagation, internal cache state, custom key callables, partialmethod support, execution of the README’s own examples, maxsize eviction, thread-safety, and TTL/jitter — thorough for a library this size. pytest is configured with filterwarnings = error (warnings promoted to failures) and xfail_strict = true, which raises the bar for silent regressions. Typing is strict: mypy --strict runs over both async_lru and tests, with extensive use of Generic[_R], TypedDict, @final, @overload, and __slots__ on both wrapper classes. Error handling is deliberate rather than incidental — for example, the task-done callback reads the private task._exception attribute instead of calling .exception() specifically to avoid asyncio’s traceback-logging side effect. Linting (flake8, isort) is enforced in CI via a dedicated lint job that also runs twine check on the built distribution.

API Design The public surface mirrors functools.lru_cache closely by design, so anyone familiar with the stdlib decorator can adopt it with almost no new API to learn — the only unfamiliar additions are ttl, jitter, and key, each documented with runnable README examples. Beyond a bare port, it adds real behavior: concurrent-call de-duplication so simultaneous callers with the same key share one in-flight coroutine, last-waiter-cancels-the-underlying-task semantics on cancellation, TTL expiration with jitter to avoid synchronized cache-stampede invalidation, a pluggable key callable so cache keys can be derived from a subset of arguments, a non-mutating cache_contains() for membership checks, and an explicit event-loop-affinity guard that clears and warns on loop reuse instead of silently corrupting state (a common asyncio footgun in test suites that spin up multiple loops). The README’s candid “Security considerations” section calling out cross-tenant cache-key leakage risk is an unusually thorough piece of documentation for a caching library.