Watermill

Watermill is a Go library that provides a single, consistent way to work with message streams and build event-driven applications, regardless of which underlying Pub/Sub technology you use. It defines Publisher/Subscriber interfaces and a Message type with per-message Ack/Nack semantics, so handlers written against the core message package can run unchanged against Kafka, RabbitMQ, NATS, Google Cloud Pub/Sub, AWS SNS/SQS, PostgreSQL/MySQL, Redis Streams, HTTP, or an in-memory Go channel implementation — swapping transports is a configuration change, not a rewrite.

On top of that core, Watermill ships a Router with composable middleware (retry, circuit breaker, throttling, deduplication, poison-message handling, correlation IDs) and a CQRS component that adds command and event buses, typed handlers, and pluggable marshalers (JSON, Protobuf, gogo/protobuf). The project has been production-stable since v1.0.0, with the public API frozen against breaking changes until the next major version, and its Pub/Sub implementations are validated against a shared stress-test suite that runs the full test matrix 20x in parallel with the race detector enabled.

What You Get

• A transport-agnostic Message/Publisher/Subscriber interface with per-message Ack/Nack
• A Router with a composable middleware stack (retry, circuit breaker, throttle, deduplicator, poison queue, correlation ID propagation)
• A CQRS component with command/event buses, typed processors, and JSON/Protobuf marshalers
• An in-memory GoChannel Pub/Sub for local development and testing, plus a shared conformance test suite used by every backend implementation
• Companion packages for Kafka, RabbitMQ, NATS JetStream, Google Cloud Pub/Sub, AWS SNS/SQS, SQL, Redis Streams, and HTTP

Common Use Cases

• Event-driven microservices — publish domain events from one service and consume them in others without coupling to a specific broker
• CQRS/event sourcing systems — separate command handling from event projection using the CQRS component’s command and event buses
• Reliable webhook delivery — receive and forward webhooks through the Router with retry and circuit-breaker middleware
• Exactly-once style processing pipelines — combine deduplication middleware with transactional outbox patterns for at-least-once delivery guarantees
• Swapping message brokers without rewriting handlers — prototype on the in-memory GoChannel Pub/Sub, then move to Kafka or RabbitMQ in production

Under The Hood

Architecture Watermill’s core is the message package (message.go, pubsub.go), which defines the Message type and the Publisher/Subscriber interfaces that everything else builds on. The Router (message/router.go) sits above these interfaces, dispatching messages to HandlerFunc callbacks and wrapping them in a composable HandlerMiddleware chain (message/router/middleware/*, e.g. retry.go, circuit_breaker.go, deduplicator.go, poison.go). Higher-level components — CQRS (components/cqrs), delay, fanin, forwarder, metrics, requestreply, requeuer — are built entirely on top of the Publisher/Subscriber contract rather than reaching into transport internals, so they work identically regardless of backend. internal/ holds shared test helpers (channel.go, publisher/subscriber) used both by the bundled GoChannel implementation and by the conformance suite in pubsub/tests, which every backend — including third-party ones like watermill-kafka or watermill-amqp — runs against before being considered production-ready. Because the public API centers on a handful of small interfaces, changing Message or Publisher/Subscriber would ripple across every satellite Pub/Sub package; the project treats that surface as frozen since v1.0.0.

Tech Stack Watermill targets Go 1.25 and keeps its core dependency footprint deliberately small: cenkalti/backoff for retry timing, sony/gobreaker for the circuit-breaker middleware, prometheus/client_golang for the metrics component, google/uuid, lithammer/shortuuid and oklog/ulid as interchangeable ID generators, pkg/errors for wrapped error chains, and three separate protobuf stacks (golang/protobuf, gogo/protobuf, google.golang.org/protobuf) to support multiple CQRS marshaler formats. Database, broker, and cloud-provider clients live in separate companion repositories (watermill-kafka, watermill-sql, watermill-nats, etc.) rather than in this module, keeping the core library free of heavyweight transport dependencies. Builds run through a Makefile (test, test_race, test_stress, test_codecov targets) and a reusable GitHub Actions workflow with Codecov integration.

Code Quality Roughly a quarter of the module’s Go files are test files, using testify for assertions alongside a hand-written shared conformance suite (pubsub/tests/test_pubsub.go) that every Pub/Sub implementation must pass, run with the race detector enabled and, in stress mode, 20x in parallel per the README. Error handling consistently wraps errors via pkg/errors rather than swallowing them, exported types carry extensive godoc comments, and formatting is enforced via go fmt/goimports in the Makefile. No dedicated golangci-lint config was found in the repo, so static-analysis enforcement beyond gofmt/vet is not evident from the source alone.

API Design The library’s signature design choice is reducing a message handler to a single function shape, func(*Message) ([]*Message, error), and letting everything else — acknowledgement, retries, circuit breaking, deduplication — attach as middleware around that one function, mirroring the net/http middleware pattern but for asynchronous messaging. This keeps onboarding low: a developer who understands one HandlerFunc can read and compose the built-in middleware stack without learning a different API per broker. The CQRS component extends the same ergonomics to commands and events with typed handlers and pluggable marshalers, and the bundled GoChannel Pub/Sub plus an extensive _examples directory let newcomers run a first working handler without standing up any external infrastructure.