cmux

cmux is a connection multiplexer for Go that lets a single TCP listener serve multiple protocols — gRPC, HTTP/1, HTTP/2, TLS, and arbitrary application protocols — by sniffing the first bytes of each incoming connection before handing it off to the right handler. Instead of requiring a separate port per service, cmux inspects the initial payload against a set of registered matchers and routes the connection to a dedicated net.Listener, so existing servers (net/http, grpc-go, net/rpc, and others) can run unmodified behind the same listener.

Under the hood, cmux buffers the sniffed bytes so they remain readable by the downstream handler once a match is found, and evaluates matchers in the order they were registered, so earlier calls to Match take priority when protocols could overlap. It ships built-in matchers for HTTP/1, HTTP/2 (including header-field matching for things like gRPC’s content-type), and TLS, plus a byte/string prefix matcher backed by a patricia tree for building custom protocol detectors.

What You Get

• Protocol-based connection routing via cmux.New(listener) plus Match()/MatchWithWriters()
• Built-in matchers for HTTP/1 (fast and strict variants), HTTP/2 (including header-field matching), and TLS handshakes
• A prefix matcher backed by a patricia tree for custom byte- or string-based protocol detection
• Non-destructive connection sniffing via a buffered reader that replays consumed bytes to the eventual handler
• Configurable error handling and read timeouts for the sniffing phase

Common Use Cases

• Serving gRPC and a regular HTTP/REST API from the same port
• Layering TLS and plaintext connections behind one listener with the TLS() matcher
• Running an RPC service (net/rpc) alongside HTTP handlers without a reverse proxy
• Building service meshes or sidecars that need to demux inbound traffic by protocol before handing it to per-protocol servers

Under The Hood

Architecture The library centers on a single mutable cMux struct (cmux.go) that wraps a root net.Listener and holds an ordered slice of matchersListener entries, each pairing a list of MatchWriter matchers with its own buffered muxListener channel; Serve() runs a single accept loop on the root listener and spawns a goroutine per incoming connection that walks the registered matcher groups in registration order, calling each against a MuxConn wrapping a bufferedReader (buffer.go) that tees sniffed bytes into an internal buffer so they can be replayed once a match is found via doneSniffing(). Matched connections are handed off through a buffered channel to the corresponding muxListener.Accept(), letting each protocol’s own server consume from what looks like an ordinary net.Listener with no awareness of the multiplexing underneath. Coordination relies on a sync.Mutex around close-time channel teardown and a sync.WaitGroup to drain in-flight connections on shutdown; the design is intentionally flat, with no dependency injection and few exported interfaces beyond Matcher/MatchWriter/ErrorHandler, so a change to the core cMux struct or the sniffing scheme would ripple through nearly every exported function.

Tech Stack Written in Go (module github.com/soheilhy/cmux, targeting Go 1.23+), with a minimal dependency footprint — only golang.org/x/net for the HTTP/2 framing and HPACK header decoding used by the HTTP/2 and gRPC header matchers, plus its transitive golang.org/x/text. There is no external build tooling beyond standard go build/go test; CI runs gofmt -s, go vet, go build, and go test -race across two Go versions for the library, and separately for the example/ directory, which carries its own go.mod. It operates purely at the net.Listener/net.Conn level of the standard library — a low-level networking primitive rather than an application framework.

Code Quality Testing lives in cmux_test.go (covering HTTP1/HTTP2/TLS matching, error handling, timeouts, and concurrent Serve/Close behavior) and patricia_test.go (prefix-tree matcher tests), both run with the race detector in CI; bench_test.go adds benchmarks for matcher performance. Error handling is explicit and idiomatic — custom typed errors (ErrNotMatched, errListenerClosed, ErrServerClosed) implement net.Error’s Temporary()/Timeout() methods rather than relying on string comparisons, and HandleError() lets callers override default retry-vs-fatal error classification. Naming follows standard Go convention, exported types carry doc comments, and gofmt -s/go vet are enforced in CI, though there is no dedicated linter beyond that.

What Makes It Unique cmux’s core mechanism — a buffered reader that transparently tees bytes consumed during protocol sniffing back into a replay buffer, so a listener can inspect a connection without consuming it for the real handler — is a genuinely useful, non-obvious solution to running multiple protocol servers on one port without a reverse proxy, complemented by a patricia-tree-based prefix matcher for allocation-light multi-string matching. Connection sniffing and protocol multiplexing is nonetheless a known pattern used at larger scale by reverse proxies and service-mesh listener filters; cmux’s contribution is packaging it as a small, dependency-light, single-port Go library rather than inventing a new underlying technique.