An open source AI-native proxy that secures, observes, and governs agent-to-LLM, agent-to-tool, and agent-to-agent communication through MCP, A2A, and unified LLM routing.
agentgateway is a next-generation open source proxy purpose-built for the agentic AI era. Written in Rust for performance and reliability, it sits between your AI agents and the services they depend on — LLM providers, MCP tool servers, and other agents — enforcing security policies, capturing telemetry, and managing traffic without requiring changes to your agent framework or model provider.
Unlike traditional API gateways retrofitted for AI, agentgateway speaks AI-native protocols natively. It implements the Model Context Protocol (MCP) in full, including stdio, HTTP, SSE, and Streamable HTTP transports, allowing any MCP-compatible client to discover and call tools through a single federation point. Its A2A gateway layer enables structured agent-to-agent communication with capability discovery and modality negotiation. And its LLM gateway exposes a unified OpenAI-compatible API that routes transparently to OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, and custom model endpoints.
agentgateway brings enterprise-grade infrastructure concerns to AI workloads: multi-layered guardrails (regex, OpenAI moderation, AWS Bedrock Guardrails, Google Model Armor), fine-grained RBAC backed by a CEL policy engine, JWT and OAuth authentication, budget and spend controls with per-token cost tracking, and full OpenTelemetry metrics, logs, and tracing. It runs as a standalone binary, integrates with Kubernetes via the Gateway API and a built-in controller, and ships with a built-in React UI for exploring gateway configuration and live traffic.
As a Linux Foundation project with an active release cadence and growing contributor community, agentgateway is positioned as the connective infrastructure layer for production agentic systems — providing the same observability and control plane capabilities that service meshes brought to microservices, now applied to the fast-moving world of AI agents.
Architecture agentgateway is structured as a layered, modular proxy organized around distinct protocol planes. A listener and bind management layer accepts incoming connections and dispatches to protocol-specific handlers — an HTTP/HTTPS proxy for LLM and MCP traffic, a TCP proxy layer for lower-level routing, and an HBONE tunnel for Kubernetes waypoint integration. The core routing engine uses an in-memory store fed by two interchangeable control planes: a static YAML/file-based configuration watcher and a dynamic xDS control plane compatible with Envoy’s API, enabling GitOps and service-mesh-style management. State is managed through a reactive store that notifies listeners of bind and policy changes, keeping the hot path free of configuration reloads. The CEL policy engine is deeply integrated, evaluating expressions against request context at both the gateway layer and per-tool, enabling fine-grained access control without bespoke code paths.
Tech Stack The proxy core is written in Rust using Tokio for async I/O, Hyper and Axum for HTTP handling, and Rustls with AWS-LC for cryptography — deliberately avoiding OpenSSL in the core path. Protocol buffers are handled via Prost and Tonic for the xDS/gRPC control plane. The LLM adapter layer uses the async-openai crate as a foundation, with custom serialization and streaming guardrail wrappers for each provider (OpenAI, Anthropic, Gemini, Bedrock, Azure, Vertex). Token counting uses tiktoken-rs. The Kubernetes controller is written in Go, using the controller-runtime and Envoy go-control-plane libraries to bridge the Gateway API CRDs to the agentgateway xDS control plane. The web UI is a React 19 application built with Vite, using TanStack Router and Query, Monaco Editor for YAML/CEL configuration editing, and Recharts for traffic visualization.
Code Quality The codebase demonstrates extensive automated testing with over 100 test modules spread across the proxy, LLM, MCP, and policy layers. Integration tests use Wiremock for HTTP mocking and a real Keycloak instance in CI for authentication validation. The CI pipeline runs on three platforms (Linux, macOS, Windows) using cargo fmt and cargo clippy with -D warnings as hard gates. The Insta crate is used for snapshot testing of request/response transformations. Error handling is typed throughout using thiserror with explicit propagation — silent error swallowing is rare. The codebase enforces Rust 2024 edition and targets MSRV 1.90, with rustfmt and a deny.toml for dependency auditing providing consistent style and supply-chain hygiene.
What Makes It Unique agentgateway is the only open source proxy that implements MCP, A2A, and unified LLM routing in a single binary rather than treating them as separate concerns. Its CEL-based policy engine allows attribute-based access control expressions that reference token counts, model names, user identities, and tool parameters simultaneously — a level of granularity not available in general-purpose API gateways. The Kubernetes Inference Gateway extension adds real-time inference routing using live signals from GPU utilization, KV cache state, LoRA adapter availability, and queue depth, enabling hardware-aware model routing that no existing gateway addresses. Being a Linux Foundation project from the outset rather than a vendor donation also shapes its governance model, making it a credible neutral hub for a multi-vendor AI ecosystem.
agentgateway is released under the Apache License 2.0, one of the most permissive open source licenses available. You can use it commercially, modify it, distribute it, and embed it in proprietary products without any copyleft obligations. There is no dual-license trap — the full feature set, including guardrails, RBAC, the Kubernetes controller, and the UI, is available under the same Apache 2.0 terms with no “enterprise edition” gating. As a Linux Foundation project, the governance and IP are managed neutrally, which matters for organizations that need confidence the project will remain community-owned.
Running agentgateway yourself requires meaningful infrastructure investment depending on your deployment target. The standalone binary is operationally straightforward — a single Rust executable with no external runtime dependencies — but you are responsible for configuration management, TLS certificate rotation, high availability setup (running multiple instances behind a load balancer), and log/metric pipeline integration. The Kubernetes deployment adds the controller as an additional component to manage, and you will need to maintain compatibility between the controller and the proxy as you upgrade. Configuration is expressed in YAML with xDS as the dynamic alternative; there is no hosted configuration service or UI persistence layer, so configuration lives in files or a Git repository.
As an early-stage but fast-moving project (v1.3.0 released with active weekly cadence), there is no commercial support tier, no SLA, and no managed cloud offering at time of writing. The community Discord and GitHub issues are the primary support channels. Organizations considering agentgateway for production should budget time for active engagement with the project, self-managing upgrades across a rapidly evolving API surface, and building their own operational runbooks. The trade-off is significant control and the absence of vendor lock-in — you own the full data plane and policy layer.
No Code Platforms · AI Development · Developer Tools
Visual LLM workflow platform with RAG pipelines, agent capabilities, and model management for building production AI applications.
Developer Tools · Game Development · Design Tools
Free, MIT-licensed 2D and 3D game engine with one-click multi-platform export and no royalties.
Developer Tools · Databases · Search
The open-source Postgres development platform that replaces Firebase with authentication, real-time APIs, edge functions, storage, and vector embeddings — all built on PostgreSQL.