leaky-bucket

leaky-bucket is a Rust crate that implements a token-based rate limiter following the leaky bucket algorithm, built for use inside a Tokio async runtime. When the configured capacity is exhausted, tasks trying to acquire tokens are suspended until enough tokens have drained back into the bucket, giving callers a simple acquire/try_acquire API for throttling work such as outbound HTTP requests.

Unlike most token-bucket implementations, it does not require a coordinating background task to keep refilling tokens on a timer. Instead, one of the waiting tasks temporarily takes on the role of “core” and is responsible for sleeping the refill interval and waking the others, and this role automatically hands off if the current core task is dropped. Combined with fair and unfair scheduling modes, this makes it a lightweight primitive for rate-limiting concurrent async workloads without spawning extra tasks.

What You Get

• A RateLimiter with a fluent Builder for configuring initial tokens, max capacity, refill amount, refill interval, and fair/unfair scheduling
• acquire, acquire_one, try_acquire, and acquire_owned methods covering both blocking-until-available and non-blocking call patterns
• A self-coordinating design where one waiting task becomes the temporary “core” responsible for refills, so no dedicated background task or thread is needed
• Automatic release of reserved capacity when an in-flight acquire future is dropped/cancelled, so cancelled tasks don’t permanently consume wait slots
• Optional tracing feature flag for instrumenting rate limiter internals
• Runnable examples covering basic usage, threaded usage, overflow behavior, and unfair scheduling trade-offs

Common Use Cases

• Throttling outbound HTTP requests to a third-party API so a service stays under a provider’s rate limit
• Limiting the throughput of a shared resource (database connections, queue consumers) across many concurrent Tokio tasks
• Smoothing bursty producer workloads so downstream consumers aren’t overwhelmed
• Enforcing per-client or per-tenant request quotas inside an async Rust server

Under The Hood

Architecture The crate centers on RateLimiter (src/lib.rs), which pairs an AtomicUsize fast-path token counter (state) with a parking_lot::Mutex<Critical> guarding an intrusive doubly-linked list of waiting Task nodes (src/linked_list.rs). Acquiring tokens first tries a lock-free fast path that decrements the atomic counter directly; if insufficient tokens are available, the caller’s future is linked into the waiter list and one waiting task is promoted to “core,” taking responsibility for sleeping the refill interval and driving wakeups for itself and other waiters — eliminating the need for a separate background coordinator task. Core-switching hands the role to another waiter automatically if the current core future is dropped, and Drop impls on the futures ensure any reserved capacity or list linkage is cleaned up if a task is cancelled mid-acquire. This self-electing coordinator pattern, built on raw pointers and unsafe (explicit Send/Sync impls on RateLimiter and the acquire futures), is the core architectural idea that shapes the rest of the crate.

Tech Stack The crate targets Rust edition 2018 with an MSRV of 1.71 and depends on tokio (time feature) for its interval-sleeping primitive, parking_lot for the critical-section mutex, and pin-project-lite to safely project pinned future state; an optional tracing dependency (feature-gated) adds instrumentation. Dev-dependencies (anyhow, futures, pin-project, a local helpers crate, and tokio with test-util/multi-thread runtime features) support its test suite and examples. There is no async-runtime abstraction — it is Tokio-specific by design.

Code Quality The test suite (tests/) is organized by concern — fast-path behavior, overflow handling, drop/cancellation semantics, idle behavior, threaded/concurrent access, core-movement correctness, and a regression test tied to a specific GitHub issue — and CI (.github/workflows/ci.yml) runs cargo build, cargo test --all-targets --all-features, doctests, and a separate clippy job across both the MSRV and stable Rust toolchains, plus a weekly scheduled run. Given the crate’s reliance on unsafe pointer manipulation for its intrusive linked list, the breadth of concurrency-focused tests (drop safety, threaded contention, core movement) is a meaningful quality signal, though there is no miri or loom-based verification job visible for the unsafe code paths specifically.

What Makes It Unique Most token-bucket rate limiters need an external timer or spawned background task to keep refilling capacity, which adds task-spawning overhead and a lifecycle to manage. leaky-bucket’s core-switching design lets whichever waiting task happens to need coordination temporarily absorb that responsibility, then hand it off — a self-organizing approach that avoids the extra task entirely while still supporting both a lock-free fast path for the common case and configurable fair/unfair scheduling for contended cases.