requests-cache

requests-cache is a persistent HTTP cache for the popular Python requests library. It provides a drop-in replacement for requests.Session that transparently stores responses and serves subsequent identical requests from cache, dramatically cutting latency and load on remote APIs.

It works with multiple storage backends including SQLite, Redis, MongoDB, DynamoDB, and plain files, and supports standard HTTP Cache-Control headers, custom expiration schedules, conditional requests, and fine-grained request matching so you can tune freshness against performance.

What You Get

• A drop-in CachedSession that behaves exactly like requests.Session but caches responses
• Global patching to add transparent caching to all requests calls without touching existing code
• Multiple storage backends: SQLite, Redis, MongoDB, DynamoDB, GridFS, and filesystem
• Cache-Control header support plus custom expiration schedules and per-request TTLs
• Fine-grained request matching by method, headers, and parameters, with redaction of secrets

Common Use Cases

• Speeding up scripts and web scrapers that repeatedly hit slow or rate-limited endpoints
• Reducing API quota consumption by serving cached responses within their freshness window
• Adding offline-friendly, resilient behavior with stale-if-error fallbacks
• Sharing a cache across processes or machines using Redis, MongoDB, or DynamoDB

Under The Hood

Architecture - The library centers on CachedSession (requests_cache/session.py), which wraps requests.Session and intercepts send() to consult a pluggable backend before hitting the network. Cache keys are computed in cache_keys.py from the request method, URL, headers, and body; responses flow through models/ (typed attrs data classes) and serializers/ (cattrs-based pipelines) before being stored. The policy/ package governs expiration, Cache-Control directives, and matching actions, cleanly separating caching policy from storage.

Tech Stack - Pure Python (>=3.8) built on requests, urllib3, attrs, cattrs, platformdirs, and url-normalize. Optional extras pull in boto3 (DynamoDB), pymongo (MongoDB/GridFS), redis, itsdangerous (signing), and pyyaml. Packaging and tooling use pyproject.toml with uv, nox, and pre-commit.

Code Quality - The codebase is fully type-annotated (Typing :: Typed) and organized into focused subpackages (backends, models, serializers, policy). It ships an extensive tests/ tree split into unit and integration suites with a docker-compose stack for backend integration tests, indicating strong coverage across storage layers.

API Design - The public API is deliberately minimal and ergonomic: CachedSession mirrors requests.Session so no new mental model is needed, and install_cache()/uninstall_cache() offer zero-code global patching. Sensible defaults work out of the box, while keyword options (expire_after, cache_control, allowable_methods, ignored_parameters) expose deep customization without boilerplate.