Moka

Moka is a fast, concurrent caching library for Rust inspired by the Caffeine library for Java. It provides both synchronous (thread-safe, shareable across OS threads) and asynchronous (futures-aware) cache implementations built on top of concurrent hash maps, with near-optimal hit ratios via a TinyLFU-based admission/eviction policy combining LFU admission with LRU eviction.

Caches can be bounded by entry count or total weighted size, support time-to-live and time-to-idle expiration plus per-entry variable expiration, and offer atomic per-key insertion (get_with), eviction listeners, and a lock-free concurrent iterator — the kind of feature depth needed by high-traffic production services (the crates.io registry itself uses Moka to maintain ~85% cache hit rates on its download endpoint).

What You Get

• Thread-safe synchronous cache (moka::sync::Cache) shareable across OS threads via cheap cloning
• An asynchronous, futures-aware cache (moka::future::Cache) for use under Tokio, async-std, or actix-rt
• TinyLFU-inspired admission (LFU) and eviction (LRU) policy for near-optimal hit ratios
• Size-aware eviction — bound a cache by entry count or total weighted size of entries
• Time-to-live, time-to-idle, and per-entry variable expiration policies
• Atomic per-key insertion methods (get_with, try_get_with) and an eviction listener callback

Common Use Cases

• Reducing load on a backing database or API by caching frequently-accessed query results in-process
• Building a high-throughput concurrent cache shared across many worker threads or async tasks
• Caching with automatic expiration (TTL/TTI) for session data, rate-limit counters, or short-lived computed values
• Embedding a cache in resource-constrained environments (e.g. home routers, embedded Linux) needing predictable memory bounds via size-aware eviction

Under The Hood

Architecture — Moka’s core lives in src/common (shared data structures and the TinyLFU-derived admission/eviction machinery), with parallel sync and future modules (src/sync, src/future) implementing the same conceptual cache over, respectively, a blocking/thread-safe API and an async/futures-aware API. A cht module implements the underlying concurrent hash table Moka builds its caches on top of, and notification/policy/ops modules handle eviction listener callbacks, cache-wide policy configuration, and cache operations respectively.

Tech Stack — Nearly pure Rust (99.6% of the codebase), feature-gated between sync and future cache variants so consumers only compile the concurrency model they need; the async cache integrates with any futures-aware runtime (Tokio, async-std, actix-rt) without hard-coding a single one. Moka explicitly does not support WebAssembly/WASI or no_std targets, focusing instead on full-std, multi-threaded server and embedded-Linux environments (tested down to 32-bit MIPS/ARMv5TE).

Code Quality — tests/ includes runtime-specific integration suites (runtime_tokio.rs, runtime_actix_rt2.rs, entry_api_sync.rs, entry_api_tokio.rs), stress tests for concurrent eviction correctness (eviction_stall_sync.rs, eviction_stall_future.rs, and_compute_with_race.rs), and a compile_tests directory verifying certain misuse patterns fail to compile. Codecov integration and an openspec/ directory of design specifications point to disciplined engineering practice; the README’s own comparison table against sibling projects (Mini Moka, Quick Cache) shows the maintainers are candid about Moka’s tradeoffs (larger dependency tree, more overhead) versus lighter-weight alternatives.

API Design — The sync::Cache/future::Cache split cleanly separates the two concurrency models while keeping a broadly consistent method surface (insert, get, get_with, invalidate) between them, and cache cloning is deliberately cheap (an Arc-based handle) so sharing across threads/tasks needs no extra wrapping. The v0.12 release introduced deliberate breaking changes (documented in a dedicated MIGRATION-GUIDE.md) to simplify internal behavior, which is a reasonable tradeoff for a caching library still refining its eviction internals.