foyer

foyer is a hybrid caching library for Rust that combines an in-memory cache with a disk-backed cache behind a single unified API. Inspired by Facebook’s CacheLib and the Java caffeine library, it lets applications tier hot data in memory while spilling less-frequently accessed entries to local disk, all through one HybridCache handle.

The library ships multiple interchangeable eviction algorithms (LRU, LFU, FIFO, S3-FIFO, SIEVE) and storage engines (block-based device I/O, no-op passthrough), so teams can tune cache behavior for their workload without rewriting integration code. Built-in Prometheus, OpenTelemetry, and Jaeger support gives production observability in a single configuration call, and the crate is used in projects like RisingWave, Chroma, and SlateDB for high-throughput caching in front of object storage or database engines.

What You Get

• A HybridCacheBuilder API for wiring together an in-memory cache and a disk-backed cache with independent memory and storage configuration.
• Five interchangeable in-memory eviction algorithms — LRU, LFU, FIFO, S3-FIFO, and SIEVE — selectable per cache instance.
• A block-based disk cache engine with configurable I/O throttling (read/write IOPS and throughput limits), LZ4/Zstd compression, and tombstone-log recovery modes.
• Built-in metrics integration for Prometheus, OpenTelemetry, and Jaeger, wired up in a single builder call.
• A get_or_fetch cache-aside pattern with async fetch closures for populating misses directly from a remote source.

Common Use Cases

• Fronting object storage - services like SlateDB and Percas use foyer’s disk tier to cache reads from S3-compatible storage and cut latency and cost on repeated fetches.
• Query engine intermediate caching - RisingWave uses foyer as a hybrid cache layer for stream-processing state that exceeds available memory.
• Embedding and vector lookups for LLM apps - Chroma uses foyer to cache embedding data across memory and disk tiers.
• Local read-through cache for object storage requests - Cachey uses foyer as a read-through cache in front of object storage.

Under The Hood

Architecture The project is a Cargo workspace split into layered crates: foyer-common holds cross-cutting primitives (a typed Error/ErrorKind with context and backtraces, metrics traits, hashing, buffer utilities), foyer-memory implements the in-memory eviction layer (raw.rs, cache.rs, an indexer with a sharded hash table, and pluggable eviction pickers under eviction/ for FIFO, LFU, LRU, S3-FIFO, and SIEVE), foyer-storage implements the disk-backed tier (store.rs, a block-based engine under engine/, an io/ abstraction, and serde.rs for on-disk encoding), and foyer-tokio abstracts the async runtime so the workspace can build against either real tokio or madsim-tokio for deterministic concurrency testing. The top-level foyer crate composes these into a single HybridCache via hybrid/{builder,cache,writer}.rs, using a fluent HybridCacheBuilder that configures the memory and storage layers independently before wiring them together through a pipe.rs write-back path — a design that lets eviction policy or disk engine be swapped without touching the calling code.

Tech Stack Written in Rust (edition 2024, MSRV 1.91) as a multi-crate workspace. Async execution runs on tokio (or madsim-tokio for simulation testing), with parking_lot for locking, hashbrown and twox-hash for hash tables, bincode/serde for optional serialization, and lz4/zstd for disk-tier compression. Disk I/O supports both a psync engine and an io_uring engine. Observability integrates prometheus, opentelemetry, opentelemetry-otlp, and fastrace; benchmarking uses criterion and a dedicated foyer-bench CLI crate.

Code Quality The workspace enforces workspace-level Clippy and rustc lints (missing_docs = warn, allow_attributes = warn), runs typo checking and ShellCheck in CI, and ships dedicated fuzzy-test suites (hybrid_cache_fuzzy_test.rs, storage_fuzzy_test.rs) alongside unit tests embedded per module. Errors are modeled as a typed Error/ErrorKind enum carrying context and captured backtraces rather than being swallowed or stringified, and a dependabot config plus a license-check workflow round out the CI setup.

API Design The public API favors a chained builder (CacheBuilder for memory-only, HybridCacheBuilder for hybrid) so a minimal in-memory cache can be created in one line while every disk-tier knob — block size, flushers, reclaimers, admission/reinsertion filters, throttling — remains reachable through the same fluent chain. A get_or_fetch cache-aside helper and automatic Code trait derivation for serde-compatible types reduce boilerplate for common population and serialization patterns.