fake-gcs-server

fake-gcs-server is an emulator for the Google Cloud Storage API, distributed as an importable Go package (fakestorage) and as a standalone binary/Docker image. Instead of mocking individual calls, it spins up a real HTTP (and gRPC) server that speaks the same JSON, XML, and upload APIs as GCS, so code written against the official cloud.google.com/go/storage client works against it unchanged.

The library is aimed squarely at Go test suites: fakestorage.NewServer() boots an in-process server pre-loaded with buckets and objects and hands back a ready-to-use *storage.Client. The same core also ships as a Docker container for integration tests in any language, with a pluggable storage backend (in-memory or filesystem) and optional Pub/Sub-style object notifications.

What You Get

• In-process Go server - fakestorage.NewServer() returns a server pre-seeded with buckets/objects and a working *storage.Client for direct use in tests.
• Standalone binary and Docker image - fsouza/fake-gcs-server runs the same emulator outside Go, for use from any language’s GCS SDK or gsutil.
• Pluggable storage backends - in-memory (fast, ephemeral) or filesystem-backed (-filesystem-root, with a /data preload directory), sharing one backend.Storage interface.
• HTTP and gRPC on one port - the server multiplexes the JSON/XML/upload HTTP API and a gRPC storage API on the same listener, switching on the request’s Content-Type.
• Object notifications - an internal Pub/Sub-style notification package can emit bucket-change events, for testing notification-driven pipelines.
• Signed-URL and config support - accepts -public-host/-external-url so client-generated signed URLs resolve back to the emulator instead of storage.googleapis.com.

Common Use Cases

• Unit/integration testing Go services - import fakestorage directly in _test.go files to assert upload/download/list behavior without a real GCS project.
• Language-agnostic CI pipelines - run the Docker image as a service container so Python/Node/Java/C++ test suites can point their GCS client at it via -public-host.
• Local development against GCS-backed apps - point an app’s STORAGE_EMULATOR_HOST-style config at the local emulator instead of provisioning cloud credentials.
• Regression-testing signed URL flows - validate application code that generates or consumes GCS signed URLs, using the emulator’s external-URL rewriting.

Under The Hood

Architecture The entry point (main.go) parses CLI flags and FAKE_GCS_* environment variables via internal/config, then constructs a fakestorage.Server backed by an implementation of the backend.Storage interface (internal/backend/memory.go for in-memory state, fs.go for a filesystem-backed store using extended attributes for metadata). fakestorage/server.go wires that backend into a gorilla/mux router serving the GCS JSON API, an XML API for public object access, and a resumable/multipart upload path (upload.go), while internal/grpc/server.go exposes a parallel gRPC surface over the same backend; both are multiplexed on one TCP port by inspecting each request’s Content-Type. An internal/notification package layers optional Pub/Sub-style event emission on top of object mutations, and the whole server is also embeddable directly from Go tests via fakestorage.NewServer, which returns a pre-configured *storage.Client.

Tech Stack Written in Go (module targets Go 1.26), the project depends on the official cloud.google.com/go/storage client for its own Server.Client() helper, gorilla/mux and gorilla/handlers for HTTP routing and middleware, google.golang.org/grpc plus generated storage protobufs for the gRPC surface, golang.org/x/oauth2/google for auth-token handling, github.com/pkg/xattr for filesystem-backend metadata storage, and bmatcuk/doublestar for glob-based object matching. It ships a multi-arch Docker image (Dockerfile, pushed via a GitHub Actions docker-push workflow using buildx/qemu) alongside the plain Go binary built from main.go.

Code Quality Tests live alongside their packages (fakestorage/*_test.go, internal/backend/*_test.go, internal/grpc/*_test.go) and lean on stretchr/testify for assertions, with the project’s own CLAUDE.md documenting go test -race -vet all -mod readonly ./... as the standard invocation and CI running dedicated docker-tests workflows on every PR. Public-facing behavior is additionally verified through runnable Example functions in fakestorage/example_test.go, which double as compiled documentation. A .golangci.yaml config plus staticcheck enforce linting, and expected failure states (bucket-not-found, precondition-failed) are modeled as a typed Error string rather than ad hoc panics or swallowed errors.

What Makes It Unique Unlike libraries that mock individual GCS client calls, fake-gcs-server runs an actual protocol-compatible server — JSON, XML, and gRPC surfaces together, multiplexed on a single port by Content-Type — so it can back test suites in any language, not just Go, via the same Docker image. The examples/ directory demonstrates this breadth directly, with working client snippets across C++, .NET, Java, Node, Python, Scala, gsutil, and gcloud, alongside a pluggable memory/filesystem backend abstraction that lets tests choose between throwaway speed and persisted, inspectable state.