badger

Badger is an embeddable, persistent key-value store written entirely in Go, with no cgo dependency. It underlies Dgraph, a distributed graph database, and is designed as a performant, pure-Go alternative to bindings like RocksDB. Its storage engine is based on the WiscKey paper, which separates keys from values: keys and value pointers live in a compact LSM tree, while values themselves sit in an append-only value log, dramatically cutting write amplification versus a conventional LSM design.

Beyond raw storage, Badger exposes concurrent ACID transactions with serializable snapshot isolation, versioned keys with a 3D key-value-version access model via its Iterator API, TTL support, and online backup/restore. A nightly Jepsen-style bank test run with the Go race detector continuously validates its transactional guarantees, and it has been used in production to serve datasets spanning hundreds of terabytes.

What You Get

• An embedded LSM-tree-plus-value-log storage engine with no external database process or cgo dependency
• Concurrent ACID transactions with serializable snapshot isolation (SSI) via db.View/db.Update/db.NewTransaction
• A fluent Options builder (DefaultOptions, LSMOnlyOptions, and dozens of With* setters) for tuning memtables, compaction, caching, and encryption
• Multi-versioned keys with a 3D key-value-version access model exposed through the Iterator API
• Built-in TTL support, online backup/restore, and a badger CLI tool for offline maintenance
• Optional encryption-at-rest and block/index caching backed by Ristretto

Common Use Cases

• Embedded storage engine for a distributed database or search index that needs sorted key-value access without running a separate DB server
• Local persistent cache or metadata store for a Go service, CLI tool, or edge/IoT application
• Backing store for a message queue, job queue, or session manager that needs durable, transactional writes
• High-throughput key-value workloads on SSD-backed hosts where write amplification from a traditional LSM design becomes a bottleneck

Under The Hood

Architecture Badger separates concerns cleanly across db.go (lifecycle: Open, Close, write batching via doWrites/writeRequests, memtable flushing), txn.go (an oracle providing MVCC timestamps, conflict detection, and serializable snapshot isolation for Txn), levels.go/level_handler.go/compaction.go (LSM level management and compaction), and dedicated subpackages: skl/ (an arena-backed skiplist used as the in-memory memtable), table/ (SSTable building, reading, and merge iteration), and y/ (checksums, bloom filters, watermarks, encryption). Writes flow from sendToWriteCh through the write-ahead memtable (skiplist) into L0 SSTables, which compaction.go then merges down through levels tracked in manifest.go. Large values are written to a separate value log (value.go) rather than inline in the LSM tree, the core WiscKey separation that most alternatives don’t make. Changing that key/value-separation boundary would ripple through the write path, compaction, and the iterator/GC logic simultaneously, which is the main cost of this design’s cohesion.

Tech Stack Badger is a Go 1.24 module with a deliberately small dependency surface: dgraph-io/ristretto/v2 for block and index caching, cespare/xxhash/v2 for hashing, klauspost/compress for value/block compression, google/flatbuffers and google.golang.org/protobuf for internal encoding, and go.opentelemetry.io/otel for metrics/tracing. spf13/cobra powers the bundled badger CLI (badger/cmd) used for offline backup, restore, and maintenance. There is no external database process or network protocol — Badger runs entirely in-process against local disk, and is built and released via a Makefile plus a dedicated CI/CD GitHub Actions pipeline.

Code Quality The project ships 26 _test.go files, several of them extensive (db_test.go, levels_test.go, and value_test.go each tens of thousands of lines), built on stretchr/testify. A dedicated nightly “Jepsen-style bank test” workflow runs for hours with the Go race detector specifically to validate transactional (SSI) guarantees under concurrency, alongside separate CI workflows for the core test suite, LSM benchmarks, and downstream Dgraph compatibility. Errors are explicit, named sentinel values (ErrKeyNotFound, ErrConflict, ErrEmptyKey, etc.) documented inline rather than swallowed, and comment density across core files like options.go and db.go is high, consistent with a project treated as critical infrastructure.

What Makes It Unique Badger’s defining choice is implementing the WiscKey design in a widely-used production system: separating keys from values to convert random writes into mostly-sequential ones, tuned specifically for SSDs rather than the spinning-disk assumptions behind LevelDB/RocksDB’s lineage. It further exposes multi-version key access (retrieve or scan prior versions of a key up to a configured retention) and serializable snapshot isolation for concurrent transactions directly through its public API, capabilities that most embedded pure-Go key-value stores don’t offer at all.