api-client-python

The Runloop Python library provides typed, convenient access to the Runloop REST API for creating and managing cloud devboxes (sandboxed compute environments purpose-built for AI coding agents), executing commands inside them, managing blueprints and disk snapshots, and running benchmark scenarios to evaluate agent behavior. It ships both a generated REST client (Runloop / AsyncRunloop) with full request/response typing and a hand-authored object-oriented layer (RunloopSDK / AsyncRunloopSDK) that wraps common workflows like devbox lifecycle management, command execution, and snapshot/resume in a more Pythonic interface.

The library is generated with Stainless from Runloop’s OpenAPI spec, giving it Pydantic-typed request params and response models across every resource — devboxes, blueprints, snapshots, benchmarks, scenarios, secrets, network policies, MCP configs, and more. Both synchronous and asynchronous clients are powered by httpx (with optional aiohttp backend support for improved async concurrency), and the async SDK is the recommended default for production use given the I/O-bound nature of devbox operations.

Beyond basic HTTP access, the SDK layer adds ergonomic conveniences for agent-oriented workflows: context-manager-based devbox creation with automatic cleanup, streaming stdout/stderr callbacks for both blocking and background command execution, and first-class support for scenario/benchmark runs used to evaluate coding-agent performance against reproducible tasks.

What You Get

• Sync and async REST clients - Runloop and AsyncRunloop classes with full Pydantic type coverage for every request param and response field, generated directly from Runloop’s OpenAPI spec.
• Object-oriented SDK layer - RunloopSDK / AsyncRunloopSDK wrap the generated client in a Pythonic interface for devbox creation, command execution, and snapshot/resume flows, including context-manager cleanup.
• Devbox lifecycle management - create, suspend, resume, and shut down cloud sandboxes, with disk snapshot and restore support for baselining and forking environment state.
• Command execution with streaming - cmd.exec for blocking calls and cmd.exec_async for long-running background processes, both with optional real-time stdout/stderr/output callbacks.
• Agent evaluation primitives - scenarios, scenario runs, benchmarks, and benchmark runs/jobs for defining and executing reproducible tasks to score AI agent behavior.
• Platform integration resources - blueprints, secrets, network policies, MCP configs, gateway configs, and axons (event/SQL access) round out the full Runloop platform surface.

Common Use Cases

• Spinning up isolated sandboxes for AI coding agents - create a devbox from a blueprint, run agent-issued shell commands, capture stdout/stderr, and tear down cleanly when the task finishes.
• Snapshotting and forking environment state - suspend a devbox, snapshot its disk, and resume multiple isolated copies from the same baseline for parallel agent runs or reproducible testing.
• Running agent benchmarks and scenarios - define scenario scorers and benchmark jobs to evaluate how well an AI agent completes tasks inside a controlled devbox environment.
• Injecting credentials safely into agent workloads - use Runloop’s secrets and network-policy/gateway-config resources to expose runtime secrets to a devbox while shielding upstream credentials from the agent itself.
• Integrating MCP tool access - configure MCP Hub connections (e.g. GitHub tools) so agents running inside a devbox can call external tools through a managed gateway.

Under The Hood

Architecture The library separates a fully generated REST layer from a hand-authored ergonomic layer. _client.py and _base_client.py define Runloop/AsyncRunloop, wiring together per-resource client classes under resources/ (devboxes, blueprints, benchmarks, scenarios, axons, secrets, network_policies, mcp_configs, and more) that are exposed as properties on the top-level client and each map directly to one API namespace. Sitting above that, sdk/ provides object-oriented wrappers (devbox.py, async_devbox.py, blueprint.py, benchmark_run.py, scenario_builder.py, etc.) that compose multiple REST calls into higher-level operations such as “create a devbox and block until running” or “snapshot, resume, and verify isolation”, exposed through RunloopSDK/AsyncRunloopSDK in sdk/sync.py and sdk/async_.py. Swapping out the underlying transport or retry policy touches only _base_client.py; changing how a single resource behaves touches one file in resources/ without rippling into the SDK layer, since the SDK depends on the resource clients through a narrow interface.

Tech Stack Built for Python 3.9+ with httpx[http2] as the default transport (optional aiohttp backend via DefaultAioHttpClient for higher async concurrency), pydantic v2 for request/response typing, anyio for async runtime abstraction, and distro/uuid-utils/typing-extensions as supporting utilities. Packaging uses hatchling with uv as the dependency/lock manager (uv.lock), and the project ships both a pyproject.toml for standard installs and a fully typed py.typed marker. Docs are built with Sphinx/furo; examples are generated into EXAMPLES.md from metadata embedded in examples/*.py via a dedicated script.

Code Quality The repo has an extensive test suite (137+ test files) split into tests/api_resources/ (one file per generated resource, exercising request/response shapes against a mock server), tests/sdk/ (behavioral tests for the object-oriented wrapper, including sync and async devbox variants), and tests/smoketests/ for end-to-end example verification. Type checking is enforced with pyright and mypy, linting/formatting with ruff, and the exception hierarchy (RunloopError → APIError → APIStatusError → specific subclasses like AuthenticationError, RateLimitError, NotFoundError) gives callers precise, typed error handling rather than generic exceptions. CI is configured via GitHub Actions, and most of the codebase carries a “generated by Stainless” header, meaning quality is enforced upstream at the OpenAPI-spec level and regenerated consistently rather than hand-maintained per file.

What Makes It Unique Unlike a typical single-purpose API wrapper, this SDK targets a specific niche: provisioning ephemeral, snapshot-able cloud sandboxes for AI coding agents, with first-class primitives for benchmark scenarios and scorers to evaluate agent task completion. The layering of a fully generated, always-in-sync REST client underneath a separately maintained object-oriented SDK is a deliberate design choice that lets Runloop regenerate the low-level client automatically from spec changes while iterating independently on developer ergonomics (context managers, streaming callbacks, snapshot/resume helpers) in the higher-level layer.