pond

Pond is a minimalistic, high-performance Go library for managing concurrent tasks through the worker pool pattern. It lets you run a large number of tasks concurrently while capping the number of goroutines active at any given time, which avoids resource exhaustion and rate-limit violations when, for example, firing off thousands of HTTP requests or database queries.

Version 2 rewrote the library around Go generics, adding type-safe pools that return results or errors, bounded and unbounded queues, awaitable task groups, subpools that borrow a fraction of a parent pool’s capacity, dynamic pool resizing, and automatic panic recovery. Workers are created lazily as tasks arrive and torn down when idle, so a pool scales to zero rather than holding goroutines open with nothing to do.

With zero external dependencies and a small, fluent API surface, pond is aimed at Go services that need a drop-in concurrency limiter rather than a full task-queue system — the kind of building block used inside HTTP clients, batch processors, and background workers where goroutine sprawl needs a hard ceiling.

What You Get

• pond.NewPool(n) for fire-and-forget concurrency-limited task pools, plus NewResultPool[T] for pools whose tasks return a typed value
• Type-safe task submission variants — Submit, SubmitErr — for tasks that return nothing, an error, a value, or a value and an error
• Task groups (NewGroup, NewGroupContext) for submitting a batch of related tasks and waiting for them all to finish or for the first error
• Bounded or unbounded queues via WithQueueSize, with blocking or non-blocking (WithNonBlocking) submission when the queue is full
• Subpools that share a parent pool’s workers but cap their own fraction of concurrency, useful for isolating one workload from another
• Dynamic resizing of a running pool’s worker count, and automatic panic recovery that turns a task panic into a returned error instead of crashing the process
• Built-in metrics — running workers, waiting/submitted/successful/failed/dropped task counts — for monitoring pool health at runtime

Common Use Cases

• Capping concurrent outbound HTTP requests to stay under a third-party API’s rate limit
• Limiting concurrent database connections or queries issued by a batch job
• Processing a large backlog of independent jobs (image resizing, file parsing, webhook delivery) with a fixed worker ceiling
• Fan-out/fan-in task groups where a caller needs to wait for a batch of related goroutines and short-circuit on the first error
• Building a request-scoped subpool so one tenant or code path can’t starve the rest of an application’s shared worker capacity

Under The Hood

Architecture The core lives in a single flat package built around a pool/task/future triad: pool.go implements a BasePool interface backed by atomic counters for worker/task bookkeeping and an internal/linkedbuffer.LinkedBuffer[T] for the task queue, while internal/future provides an awaitable completion primitive built on context.WithCancelCause so Task.Wait()/Err() can block until a task resolves. Submitted tasks are written into the linked buffer and drained by lazily-spawned worker goroutines that scale to zero when idle; group.go layers a composite future over multiple task futures so a Group can wait for all tasks or short-circuit on the first error, and subpools wrap a parent pool with their own concurrency ceiling rather than duplicating the dispatch loop. Because nearly every public type funnels through the same future/queue primitives, a change to the cancellation-cause mechanism in internal/future would ripple through every pool, group, and subpool call site.

Tech Stack Pond has zero third-party dependencies — everything is standard library (context, sync, sync/atomic, errors, math) — and leans on Go generics (NewResultPool[T], LinkedBuffer[T]) for type-safe result and error handling. The module targets Go 1.20+, builds and tests via a Makefile (make test-ci, make coverage), and ships several standalone example programs under examples/ (including a Prometheus metrics integration) each with their own go.mod.

Code Quality The suite runs to roughly a hundred test functions across the root package and the internal/linkedbuffer and internal/future packages, exercising concurrent submission, cancellation, panics, and queue edge cases directly against a small hand-rolled internal/assert helper rather than a third-party assertion library. CI runs the full suite across five Go versions (1.21–1.25) on Ubuntu, macOS, and Windows, with a separate coverage job uploading to Codecov and CodeQL scanning on every push — a comprehensive matrix even though there’s no committed linter configuration. Error handling favors sentinel errors (ErrQueueFull, ErrPoolStopped, ErrMaxConcurrencyReached) over swallowed failures.

What Makes It Unique Worker pools are a well-established pattern, so pond’s novelty is in the specifics rather than the concept: pools scale workers to zero when idle instead of holding a fixed set open, subpools borrow a fraction of a parent pool’s capacity without a second dispatch loop, and the v2 rewrite adds generic result/error task types plus built-in panic recovery that turns a task panic into a returned error. The project’s own documentation backs a specific, testable performance claim — that it can outperform unbounded goroutines in some workloads — rather than resting on generic feature marketing.