Durable Streams Client

@durable-streams/client is the reference TypeScript client for the Durable Streams protocol, a minimal HTTP-based approach to durable, offset-addressable data streams built out of Electric’s production experience syncing Postgres in real time. Rather than treating a stream connection as fragile state that dies when a tab suspends or a network flaps, the client models a stream as a durable, replayable sequence that any client can resume from an arbitrary offset.

The package exposes three complementary APIs: a fetch-like stream() function for read-only consumption (with promise helpers, ReadableStream output, and backpressure-aware subscribers), a DurableStream class for full read/write handles including create/append/close/delete, and an IdempotentProducer for high-throughput, exactly-once writes using Kafka-style producer IDs, epochs, and per-batch sequence numbers with automatic batching and pipelining.

It is aimed at teams building AI token-streaming UIs, agentic tool-output pipelines, database-change fan-out, and collaborative multi-user/multi-agent apps where a WebSocket or SSE connection alone isn’t durable enough — the client handles reconnection, catch-up reads, and live tailing over plain HTTP so state survives refreshes, tab switches, and device changes.

What You Get

• A fetch-like stream() function for simple, read-only stream consumption with promise, ReadableStream, and subscriber-based access patterns
• A DurableStream class handle for full lifecycle control: create, connect, head, append, appendStream, close, and delete
• IdempotentProducer for Kafka-style exactly-once writes with automatic batching, pipelining, and zombie-producer fencing via epochs
• Offset-based resumability so any client can pick up a stream from where it left off after a refresh, reconnect, or device change
• Automatic live-mode selection between SSE and long-poll, with catch-up-only mode for one-shot reads
• Configurable retry/backoff and pluggable error handling, including support for token-refresh-on-401 flows via async header functions

Common Use Cases

• Streaming LLM token responses to a chat UI with resume-safe reconnection across network drops
• Fanning out agentic tool-output and progress events to web, mobile, and native clients with full replay history
• Syncing database change events (e.g. Postgres CDC) to many concurrent viewers via CDN-friendly HTTP fan-out
• Building collaborative multi-user, multi-agent sessions where several clients tail and append to the same durable stream
• Implementing exactly-once event ingestion pipelines using IdempotentProducer instead of hand-rolled deduplication

Under The Hood

Architecture The package separates protocol-transport concerns from consumer-facing API surface: fetch.ts and sse.ts implement the low-level HTTP transport (backoff/retry and SSE frame parsing), response.ts builds the StreamResponse abstraction that turns raw chunks into promise, ReadableStream, and subscriber consumption modes, stream.ts implements the stateful DurableStream handle (create/connect/append/close/delete), stream-api.ts wraps that into the simpler fetch-like stream() function, and idempotent-producer.ts layers Kafka-style exactly-once semantics (producer id, epoch, per-batch sequence) on top of a DurableStream handle using a fastq-backed queue for batching and pipelining. index.ts curates a narrow public export surface over this internal layering, and types.ts/constants.ts centralize the protocol’s header and query-param contracts so transport, handle, and producer code share one source of truth. Extensive internal decoupling means the core abstraction most callers depend on (StreamResponse) can evolve without touching the transport layer.

Tech Stack The client is written in TypeScript, targets Node.js >= 18 and browser fetch/SSE environments, and ships dual ESM/CJS builds via tsdown. Its only runtime dependencies are @microsoft/fetch-event-source for resilient SSE consumption and fastq for the batching queue behind IdempotentProducer — a deliberately minimal dependency footprint for a library meant to run in browsers, mobile runtimes, and edge workers alike. Devependencies include fast-check for property-based testing and the monorepo’s own reference server package for integration tests, all orchestrated through a pnpm workspace with changesets for release management.

Code Quality The package has a comprehensive test suite (300+ test cases across dedicated files for the stream API, idempotent producer, SSE parsing, backoff behavior, error handling, and property-based tests via fast-check), backed by a dedicated GitHub Actions workflow (client-tests.yml) and a monorepo-wide conformance-test package that exercises client behavior against the protocol spec directly. TypeScript is used in strict mode (extending a shared root tsconfig.json), errors are modeled as a typed class hierarchy (FetchError, DurableStreamError, StaleEpochError, SequenceGapError, etc.) rather than opaque throws, and naming/documentation follows a consistent JSDoc-annotated style throughout the source.

What Makes It Unique Most streaming client libraries treat resumability as an afterthought bolted onto WebSockets or SSE; this client instead treats an HTTP durable stream itself as the primitive, with offset-based resumability, refresh-safety, and shareable stream URLs built into the core read/write API rather than layered on top. Its IdempotentProducer brings a genuinely Kafka-style exactly-once write model (client-declared epochs with server-side zombie fencing, per-batch sequence numbers) to a plain-HTTP client library, which is uncommon outside dedicated message-broker SDKs, and its automatic SSE/long-poll mode selection with base64-encoded binary support extends that same reliability story beyond JSON-only streams.