gofakes3

gofakes3 is a Go library that spins up an in-process HTTP server implementing a large subset of the AWS S3 API, so applications and test suites that talk to S3 can run against a local, disposable double instead of a real bucket. It plugs directly into Go’s httptest.NewServer, which makes it a natural fit for unit and integration tests of Lambda functions, uploaders, and any service that reads or writes S3 objects.

The library ships three interchangeable storage backends (in-memory, BoltDB-backed, and afero filesystem-backed) behind a single Backend interface, so tests can pick fast, ephemeral storage or something that persists across runs without changing calling code. It also supports a growing slice of real S3 semantics — bucket versioning, CORS, multipart uploads, range requests, and conditional PUT/If-Match handling — and returns the same XML error contract the real S3 API does, so error-handling code paths can be exercised honestly.

What You Get

• An S3-API-compatible HTTP server you can start with one line via gofakes3.New(backend) and faker.Server()
• Three pluggable storage backends — in-memory (s3mem), BoltDB (s3bolt), and filesystem via afero (s3afero) — behind one Backend interface
• Support for bucket versioning, CORS preflight handling, multipart uploads, byte-range requests, and conditional PUT/If-Match semantics
• AWS-compatible XML error responses, so client error-handling paths behave the same as against real S3
• A standalone cmd/gofakes3 binary and Docker image for running it outside of Go test code (e.g. against browser-based direct uploads)
• Path-style and virtual-hosted-style bucket addressing, including hostname-based bucket routing

Common Use Cases

• Unit/integration testing AWS Lambda functions or services that read and write S3 objects, without hitting real AWS
• Local development of S3-dependent applications when no AWS credentials or network access are available
• Testing browser-based direct-to-S3 uploads (presigned POST policies) against a local server
• CI pipelines that need deterministic, network-free S3 behavior for test suites

Under The Hood

Architecture gofakes3 is organized around a small, well-defined Backend interface (backend.go) that any storage engine must satisfy — list/get/put/delete objects, manage bucket versioning, and report metadata — while gofakes3.go implements the HTTP-facing S3 protocol layer on top of whichever backend is supplied. routing.go handles path-style, virtual-hosted-style, and custom hostname-base bucket addressing before dispatching to handler methods, uploader.go and chunk.go implement multipart/streaming upload assembly, and error.go translates internal failures into the exact XML error shape real S3 clients expect. This separation means the three shipped backends (s3mem, s3bolt, s3afero) are fully interchangeable without touching the request-handling code, and a consumer could add a fourth backend by implementing just the Backend interface.

Tech Stack Built for Go 1.24 with no non-test runtime dependencies beyond spf13/afero (filesystem abstraction for the s3afero backend) and go.etcd.io/bbolt (embedded KV store for the s3bolt backend); the AWS SDK v2 packages in go.mod are used only by the test suite and examples to exercise the fake server as a real S3 client would. The library exposes itself as a standard net/http.Handler via faker.Server(), so it composes directly with httptest.NewServer for in-process tests or can be run standalone through the cmd/gofakes3 binary and an accompanying Docker image.

Code Quality The project carries a comprehensive test suite — roughly one test file per source file, plus dedicated suites for CORS, conditional PUT, prefixes, ranges, and routing — and CI runs go test with the race detector enabled and reports coverage to Codecov on every build. Errors are modeled as a typed S3Error value that maps directly to AWS’s error-code vocabulary rather than being swallowed or stringified, and naming/structure follow idiomatic Go conventions throughout.

API Design The public API is deliberately small: construct a backend (s3mem.New(), s3bolt.New(path), or s3afero.New(fs)), wrap it with gofakes3.New(backend), and hand faker.Server() to httptest.NewServer — three calls to a working fake S3 endpoint. The README backs this with copy-paste examples for both AWS SDK v1 and v2 in Go, plus SDK v3/v2 examples for JavaScript Lambda consumers, and functional options (option.go) let callers tune behavior (fixed clock, auto-bucket-creation, CORS strictness) without changing the core call shape.