Murr

Murr (Murrdb) sits between batch data pipelines and inference applications as a caching layer specifically tuned for ML/AI data serving patterns, which differ meaningfully from typical web-app caching: large batch reads and writes over columnar data rather than many small per-key operations. It uses tiered storage — hot data in memory, cold data on disk with S3-based replication — on the premise that keeping only genuinely hot data in expensive RAM is the right trade-off in 2026’s memory pricing.

Its core differentiator is native batch-in, batch-out semantics over columnar storage (e.g., dumping 1GB Parquet/Arrow files) with no per-row overhead, rather than treating batch operations as a loop over single-key reads/writes the way Redis does. Built on RocksDB for the underlying storage engine, Murr targets zero-copy reads and writes for latency-sensitive inference serving.

Apache-2.0 licensed and written in Rust, the project publishes benchmarks directly comparing itself against alternatives, and its README is explicit that it’s mostly human-written (used AI only for grammar/syntax checking) — a notable transparency choice in an era of AI-generated documentation.

What You Get

• A caching layer sitting between batch data pipelines and inference apps, tuned for ML/AI serving patterns
• Tiered storage — hot data in memory, cold data on disk, with S3-based replication for cold storage
• Native batch reads and writes over columnar formats (like Parquet/Arrow) with no per-row overhead
• RocksDB as the underlying storage engine, optimized for zero-copy, low-latency access

Common Use Cases

• Caching feature data between a batch ML pipeline and a real-time inference service
• Replacing Redis for AI/ML workloads where batch columnar reads/writes dominate over single-key operations
• Reducing RAM costs by tiering cold data to disk and S3 instead of keeping everything in memory
• Serving large Parquet/Arrow datasets to inference applications with minimal per-record overhead

Under The Hood

Architecture Murr layers a tiered-storage cache on top of RocksDB: hot keys stay resident in memory, cold data moves to local disk, and S3 provides durable replication for cold storage — a design specifically shaped around AI inference’s access pattern of mostly-batch reads with a smaller set of hot keys, rather than the uniformly-random access pattern typical caching layers like Redis assume. Batch reads/writes operate directly over columnar data rather than being implemented as a loop over individual key operations, which is what enables the project’s zero-copy, no-per-row-overhead claims.

Tech Stack Rust for the core implementation, RocksDB as the embedded storage engine, with S3 integration for cold-tier replication and native support for columnar formats like Parquet and Arrow for batch data movement.

Code Quality The project runs CI on every change and publishes a dedicated benchmarks section comparing performance directly, which is a stronger signal of engineering rigor than unverified performance claims; the README’s explicit disclosure that it’s mostly human-written (AI used only for grammar checking) is a notable transparency practice.

What Makes It Unique General-purpose caches like Redis are optimized for many small, independent key operations; Murr is specifically built around AI/ML serving’s batch-heavy, columnar access pattern with tiered memory/disk/S3 storage, targeting a different cost and latency profile than a Redis deployment would achieve for the same workload.

Self-Hosting

Licensing Model Apache-2.0 licensed — fully open source with no license key.

Self-Hosting Restrictions Not applicable; Murr runs as a self-hosted caching layer within your own infrastructure, with S3 as an optional cold-storage backend you configure yourself.

License Key Required No.