Portkey AI

Portkey AI (npm package portkey-ai) is the official Node.js client for Portkey, a control panel and AI gateway that sits in front of OpenAI and 250+ other LLM providers. The SDK is built as a drop-in replacement for the OpenAI Node SDK: you swap import OpenAI from 'openai' for import Portkey from 'portkey-ai', keep the exact same chat.completions.create()-style calls, and instantly gain gateway features like automated fallbacks, retries, load balancing, semantic caching, and virtual keys for storing provider credentials.

Beyond chat and completions, the client wraps a large surface of Portkey’s control-plane APIs directly in TypeScript: assistants, batches, files, fine-tuning, embeddings, moderations, prompts, virtual keys, configs, guardrails, webhooks, and realtime/WebSocket sessions. Every request can carry Portkey-specific headers (trace IDs, custom metadata, provider overrides, cache controls) generated via createHeaders(), and provider-specific auth (AWS, Azure, Vertex, Bedrock, Workers AI, Hugging Face) is handled through typed constructor options rather than manual header wrangling.

What You Get

• A Portkey client class that mirrors the OpenAI Node SDK’s method signatures, so existing OpenAI-based code migrates with a near drop-in import swap
• Typed wrappers for chat completions, completions, embeddings, images, audio, moderations, assistants, threads, batches, files, fine-tuning, and realtime WebSocket sessions
• createHeaders() for attaching Portkey routing/config metadata (virtual keys, trace IDs, cache settings, custom metadata, provider overrides) to any request
• Built-in streaming support via a Stream helper that parses server-sent event responses incrementally
• Typed provider-credential options for AWS Bedrock, Azure OpenAI, Google Vertex AI, Cloudflare Workers AI, Hugging Face, and Fireworks, so multi-provider auth doesn’t require hand-rolled headers

Common Use Cases

• Migrating an existing OpenAI Node integration to a multi-provider gateway without rewriting call sites
• Adding automatic fallback and retry behavior across multiple LLM providers or API keys for production reliability
• Centralizing LLM request logging, tracing, and cost/latency analytics across a Node.js application
• Storing provider API keys as Portkey virtual keys instead of embedding raw provider credentials in application code
• Building realtime voice/chat features on top of Portkey’s WebSocket-based realtime API

Under The Hood

Architecture The SDK is a thin, layered wrapper: client.ts defines the public Portkey class extending ApiClient (in baseClient.ts), which owns request construction, header assembly, retries, and streaming. Actual endpoint groups (chat, completions, assistants, threads, files, embeddings, realtime, and Portkey control-plane resources like configs, virtual keys, and guardrails) live as separate typed modules under src/apis/, each built on shared getMethod/postMethod/putMethod/deleteMethod helpers rather than duplicating fetch logic per resource. createHeaders() centralizes Portkey-specific header generation so every API module can attach routing metadata consistently. A dedicated beta/realtime module isolates WebSocket session handling from the synchronous REST path, and streaming.ts provides a Stream class that parses chunked responses without pulling the whole HTTP layer into that concern. The design cleanly separates “how to make an HTTP/WS request to Portkey” (baseClient, streaming) from “what each Portkey resource looks like” (apis/*), so adding a new endpoint means adding a typed module rather than touching the core client.

Tech Stack Written in TypeScript, compiled to CommonJS (dist/) via a custom build script. Runtime dependencies are deliberately minimal: agentkeepalive for HTTP keep-alive connection pooling, dotenv for environment loading, the openai package for OpenAI-compatible type/method parity, and ws for realtime WebSocket sessions. Tests run under Jest with ts-jest/babel-jest, linting via ESLint with TypeScript and unused-imports plugins, formatting via Prettier. No bundler is used for distribution — it ships plain compiled JS with .d.ts type declarations, matching how the official OpenAI Node SDK is distributed.

Code Quality Tests exist under tests/ for chat, completions, assistants, images, audio, and moderations, each split per-provider (e.g. openai.test.ts, anthropic.test.ts, anyscale.test.ts), indicating provider-compatibility is treated as its own testable surface rather than assumed. There is no CI workflow that runs the test suite on push/PR (the only GitHub Actions workflows found are for npm publishing on release and a commit-message checker), so test execution currently depends on running npm test locally or as part of a maintainer’s manual pre-release process. Error handling is explicit and typed — error.ts defines specific error classes (APIError, APIConnectionError, APIConnectionTimeoutError, APIUserAbortError) rather than throwing generic errors or swallowing failures. The large, fully-typed ApiClientInterface constructor options give strong compile-time guarantees around provider-specific configuration.

API Design The standout design choice is deliberate API mimicry: rather than inventing a new client shape, Portkey’s SDK mirrors the OpenAI Node SDK’s method names and call signatures almost exactly, so portkey.chat.completions.create(...) reads identically to the OpenAI equivalent. This makes adoption close to a one-line import swap for teams already on OpenAI’s SDK, trading originality in API shape for near-zero migration friction — a pragmatic choice for a gateway product whose value is provider-agnostic routing, not a novel calling convention.