AI Gateway Provider

AI Gateway Provider plugs Cloudflare’s AI Gateway into the Vercel AI SDK, letting you route generateText/streamText calls through Cloudflare’s gateway for any number of underlying model providers (OpenAI, Anthropic, Google, Groq, Mistral, and more) behind one unified interface. Rather than wiring each provider’s SDK independently, you wrap them with createAiGateway() and pass an array of providers to fall back through automatically if one fails.

It supports both bring-your-own-key (BYOK) usage, where you supply each provider’s own API key, and Cloudflare’s unified billing mode, where the Gateway itself authenticates to providers. It is runtime-agnostic — Node.js, Cloudflare Workers, and Edge runtimes are all supported — and ships dedicated provider adapters (ai-gateway-provider/providers/openai, /anthropic, etc.) for a dozen-plus AI vendors.

What You Get

• createAiGateway() factory that wraps an ordered list of Vercel AI SDK providers with automatic fallback on failure
• Dedicated provider adapters for OpenAI, Anthropic, Google, Google Vertex, Groq, Mistral, Cohere, DeepSeek, Fireworks, Perplexity, OpenRouter, Azure, ElevenLabs, and Deepgram
• Both BYOK (bring-your-own-key per provider) and Cloudflare unified-billing authentication modes
• Runtime-agnostic support across Node.js, Cloudflare Workers, and Edge runtimes
• Streaming (streamText) and tool/structured-output support passed through transparently to the underlying provider

Common Use Cases

• Adding automatic multi-provider fallback (e.g. OpenAI → Anthropic → Groq) so an AI feature stays available if one vendor has an outage
• Centralizing AI Gateway observability, caching, and rate-limiting for a Vercel AI SDK app without changing application call sites
• Running the same AI application across Cloudflare Workers and Node.js without per-runtime provider wiring
• Consolidating billing/authentication for multiple LLM vendors behind Cloudflare’s AI Gateway instead of managing each vendor key separately

Under The Hood

Architecture - Lives at packages/ai-gateway-provider in the cloudflare/ai monorepo (alongside workers-ai-provider, ai-search-provider, and gateway-core); src/index.ts exposes createAiGateway(), which composes a list of already-instantiated Vercel AI SDK providers (from src/providers/*.ts, one file per vendor) and returns a model factory that retries the next provider in the array on failure, while src/auth.ts handles both BYOK and Cloudflare-unified-billing authentication headers. Tech Stack - TypeScript, built with tsdown for dual ESM/CJS output, tested with Vitest (unit tests per feature plus a test/e2e suite), managed as an Nx/pnpm workspace across the sibling packages. Code Quality - Test coverage is organized by concern — fallback.test.ts, auth.test.ts, stream-text.test.ts, tools-structured.test.ts, abort.test.ts, resume.test.ts — indicating deliberate coverage of streaming, cancellation, and structured-output edge cases beyond the happy path. API Design - createAiGateway({accountId, gateway, apiKey})(provider1, provider2, ...) mirrors the Vercel AI SDK’s own provider-factory pattern closely, so adopting it requires no new mental model beyond wrapping existing provider instances in an ordered fallback list.