Beta9

Beta9 is the open-source engine powering Beam, a serverless runtime built for AI and ML workloads. It provides a Pythonic decorator-based API to deploy endpoints, task queues, functions, sandboxes, and persistent pods on GPU-enabled containers without managing Kubernetes, Docker registries, or autoscaling logic yourself. The platform uses a custom container runtime based on runc/runsc with CRIU checkpoint-restore support and a proprietary CLIP image format that enables container launch times measured in milliseconds rather than minutes.

The Go backend handles all orchestration: a gateway service exposes gRPC and HTTP APIs derived from protobuf definitions, a scheduler manages container placement across worker pools, and provider integrations cover AWS EC2, Lambda Labs, Crusoe, OCI, Hydra, and generic Kubernetes clusters. Redis serves as the primary coordination store for worker state, task queues, container lifecycle, and the backlog scheduler that bins work into available worker capacity.

The Python SDK (pip install beam-client) mirrors the server abstractions precisely. Decorating a function with @endpoint, @task_queue, or @function is all that is required to deploy it as a versioned, autoscaling service. Volume mounts, cloud bucket integration, hot-reloading during development, webhook callbacks, and scheduled jobs are all first-class SDK primitives. For code execution sandboxes, the Sandbox class provides a remote isolated environment with file system access, port exposure, process management, and memory snapshot/restore.

Beta9 can be self-hosted on any Kubernetes cluster using the provided Helm chart, with k3d-based local development via a single make setup command. The project is released under AGPL-3.0, making it free to self-host but requiring that modifications be shared under the same license if distributed as a network service.

What You Get

• Sub-second container startup - A custom CLIP image format and runc/runsc runtime launch containers in milliseconds, eliminating cold start penalties for AI inference workloads
• Serverless GPU endpoints - Deploy autoscaling HTTP endpoints with QueueDepthAutoscaler backed by H100, A10G, 4090, T4, and other GPU types using a single Python decorator
• Task queue with retry policies - Replace Celery with @task_queue decorators that support typed input schemas, configurable max retries, versioned deployments, and webhook callbacks on completion
• Isolated sandbox environments - Spin up ephemeral containers via the Sandbox class with full file system access, port exposure, process management, stdin/stdout streaming, and memory snapshot/restore for LLM-generated code execution
• Persistent pods and ASGI support - Deploy long-running services, web servers, or ASGI applications (FastAPI, Django) as versioned pods with HTTP port forwarding and Tailscale mesh networking
• Distributed volume storage - Mount persistent volumes or S3-compatible cloud buckets directly into containers for model weights, datasets, and output artifacts shared across container instances
• Hot-reload development mode - Iterate on deployed functions without redeploying by syncing local code changes live using the watchdog-based file syncer
• Multi-cloud worker pools - Provision GPU workers across AWS EC2, Lambda Labs, Crusoe, OCI, Hydra, or on-premises Kubernetes clusters from a single unified scheduler

Common Use Cases

• LLM inference API deployment - An AI startup wraps a fine-tuned Llama model in @endpoint(gpu='H100') to serve a production inference API that autoscales from zero to 50 containers based on request queue depth
• Distributed ML training jobs - A research team fans out hyperparameter sweep runs across 200 parallel containers using function.map(), collecting results from each worker through the Output abstraction
• AI code execution sandbox - A coding assistant product uses Sandbox to safely execute LLM-generated Python scripts in isolated containers with network permission controls and real-time stdout streaming to the user
• Async media processing pipeline - A video platform replaces Celery with @task_queue to process user-uploaded files asynchronously with automatic retries, input validation via typed schemas, and webhook notifications on completion
• Model weight caching layer - An ML team mounts a shared Volume to all inference containers so model weights are loaded once from S3 and reused across container restarts without re-downloading on each cold start
• Scheduled batch inference - A data team uses @schedule(every='1h') to run nightly embedding generation jobs against a PostgreSQL dataset, pushing results to S3 through a mounted CloudBucket

Under The Hood

Architecture Beta9 follows a layered service architecture where the gateway is the single entry point for all client traffic, routing gRPC and REST requests to abstraction-specific handlers registered at startup. Each workload type (endpoint, task queue, function, sandbox, pod) lives in its own package under pkg/abstractions/ and implements a shared interface that integrates with a central scheduler and Redis-backed repository layer. The scheduler maintains a request backlog and dispatches work to worker pools, which provision container capacity from pluggable cloud providers. Autoscaling is handled per-abstraction through a generic Autoscaler[I, S] type that samples queue depth or CPU load on a one-second tick and signals instances to scale up or down. This design allows each abstraction to define its own scaling heuristics while sharing the same container lifecycle machinery, creating a reasonably clean separation between the orchestration layer and the workload-specific logic.

Tech Stack The backend is written in Go 1.23 using gRPC with grpc-gateway for dual gRPC/REST serving, zerolog for structured logging, and VictoriaMetrics for runtime metrics. Redis is the coordination backbone, used for worker registration, container state, task queues, event buses, and distributed locking via a custom redislock library. PostgreSQL (accessed through Squirrel query builder) stores persistent backend state including workspaces, deployments, and usage records. Container execution uses runc for CPU workloads and gVisor’s runsc for sandboxed execution, with CRIU checkpoint-restore for GPU memory snapshots via cuda-checkpoint integration. Image distribution relies on a proprietary CLIP format stored in S3 or local object store, enabling lazy-loading mounts. The Python SDK targets Python 3.8+ and uses betterproto-generated gRPC stubs, cloudpickle for function serialization, FastAPI and uvicorn inside the runner, and watchdog for file syncing.

Code Quality The Go codebase has extensive unit tests across all major packages, with schedulers, pool managers, backlog queues, and provider credential caches all tested using miniredis for in-memory Redis and go-sqlmock for database interactions. The Python SDK has comprehensive unit tests for every abstraction using unittest.mock, covering initialization, local invocation, remote dispatch, and error conditions. Error handling is explicit throughout: Go functions return typed errors, Python raises domain-specific exception classes (SandboxConnectionError, SandboxProcessError), and gRPC errors are wrapped with context via a with_grpc_error_handling decorator. The generic Autoscaler[I, S] type and repository interface abstractions make components independently testable. Code style is enforced via Go toolchain conventions and Python type hints with typeguard validation.

What Makes It Unique The most distinctive technical decision is building a custom OCI-compatible container image format (CLIP) that supports lazy loading from S3 through content-addressed block storage, enabling sub-second container startup even for large AI model images. Combined with CRIU checkpoint-restore support for GPU workloads via cuda-checkpoint, this allows containers to be paused and resumed with GPU memory intact, which is unusual outside of specialized ML infrastructure. The generic type-parameterized autoscaler (Autoscaler[I IAutoscaledInstance, S AutoscalerSample]) means any new abstraction gets autoscaling behavior by implementing two functions rather than inheriting a class hierarchy. The multi-provider scheduler with a request backlog and provisioning backoff tracker lets a single Beta9 instance intelligently spread work across heterogeneous GPU fleets, which is the architectural pattern that makes the managed Beam cloud viable at scale.

Self-Hosting

Beta9 is licensed under the GNU Affero General Public License v3.0 (AGPL-3.0). This is a strong copyleft license that permits commercial use, modification, and distribution, but requires that any modifications made to the software be released under the same AGPL-3.0 license if you make the software available over a network to others. For teams self-hosting Beta9 purely for internal use without offering it as a service to third parties, the AGPL imposes no redistribution obligation. Teams building a product that exposes Beta9’s network API to external users should review the license carefully or consider a commercial arrangement with Beam.

Running Beta9 yourself requires a Kubernetes cluster (k3d works locally, standard Kubernetes on cloud), a Redis instance, a PostgreSQL database, and object storage (S3-compatible) for the image registry. The make setup command and provided Helm chart automate the initial installation, but ongoing operations require managing Redis memory, PostgreSQL backups, worker pool scaling limits, and Tailscale mesh networking for secure container-to-gateway communication. GPU worker provisioning depends on configuring provider credentials (AWS, Lambda Labs, Crusoe, OCI) or bringing your own Kubernetes nodes with GPU device plugins. Teams without dedicated infrastructure engineering capacity will find the operational surface non-trivial, particularly for multi-provider GPU fleet management and container image registry maintenance.

Compared to the managed Beam cloud, self-hosting means taking on full responsibility for uptime, GPU hardware procurement and provisioning, network security, and platform upgrades. The managed service provides a production-grade control plane, pre-provisioned GPU fleets including H100s and A10Gs, SLA-backed availability, managed upgrades, and direct support from the Beam team. The open-source codebase maintains feature parity as it is the same engine powering Beam, so self-hosters do not miss functionality — they miss the operational convenience and the GPU marketplace that lets workloads burst onto spot-priced cloud instances automatically.