async-openai

async-openai is an unofficial, fully asynchronous Rust client for the OpenAI API, generated from and kept in sync with OpenAI’s OpenAPI specification. It covers the breadth of the platform, including the Responses and Chat APIs, audio, images and video, embeddings, fine-tuning, batches, vector stores, assistants, realtime, and more, each behind granular feature flags so you compile only what you use.

Beyond raw endpoint coverage it focuses on developer ergonomics: an ergonomic builder pattern for every request, Server-Sent Events streaming, automatic retries with exponential backoff on rate limits, tower-based middleware, WASM support, and the ability to bring your own types or point the client at Azure OpenAI and other OpenAI-compatible providers.

What You Get

• Broad, typed coverage of OpenAI APIs (Responses, Chat, audio, images, video, embeddings, fine-tuning, batches, assistants, realtime, and more)
• An ergonomic builder pattern for constructing every request object
• Server-Sent Events streaming for incremental responses
• Automatic retries with exponential backoff when rate limited
• Granular feature flags, WASM support, tower middleware, and Azure/OpenAI-compatible provider configuration

Common Use Cases

• Calling OpenAI chat, responses, or embeddings endpoints from an async Rust service
• Streaming model output token-by-token into an application via SSE
• Targeting Azure OpenAI or other OpenAI-compatible providers from Rust with custom types and headers

Under The Hood

Architecture - The workspace splits into the async-openai crate and an async-openai-macros proc-macro crate. In async-openai/src, a Client (client.rs) parameterized by a Config (config.rs, with OpenAI and Azure variants) is the entry point; accessor methods return per-API handler structs (chat.rs, responses/, embedding.rs, audio/, image.rs, fine_tuning.rs, batches.rs, assistants/, realtime.rs and more) that build requests and dispatch them through an executor (executor.rs) over reqwest. Request/response types live under types/ and are largely modeled on the bundled openapi.yaml, with derive_builder generating fluent builders; SSE streaming and tower-based retry/timeout middleware wrap the HTTP layer.

Tech Stack - Rust with serde/serde_json for (de)serialization, reqwest for HTTP, futures and eventsource-stream for async streaming, tower for middleware (limit/retry/timeout), thiserror for errors, secrecy for API-key handling, and tracing for observability. Nearly every dependency is optional and gated behind granular Cargo feature flags, and the crate supports WASM targets.

Code Quality - The codebase is cleanly modularized by API surface, uses strong typing throughout, and centralizes error handling in error.rs with thiserror. Consistent maintenance and a healthy release cadence back a large, actively used SDK; an examples/ directory and AGENTS.md document usage and contribution practices.

API Design - The public API is ergonomic and discoverable: create a client, call client.chat()/client.embeddings()/etc., and build typed requests with builder methods. Feature flags keep the compiled surface minimal, config supports Azure and custom providers, and streaming plus retries are handled for you, so common tasks require little boilerplate while the surface area remains large enough to warrant reading the docs.