modal-client

Modal is the official Python SDK for the Modal serverless cloud platform, letting you turn a normal Python function into a remotely-executed, autoscaling cloud workload by adding a decorator — @app.function(gpu="A100") — with no Dockerfile, Kubernetes manifest, or infrastructure code required. Container images are declared inline as Python objects (modal.Image.debian_slim().pip_install(...)), dependencies are captured automatically, and the same function can be invoked synchronously, asynchronously, or as a scheduled/queued job without rewriting it.

Beyond simple function calls, the SDK exposes the full range of primitives the Modal platform offers: Sandbox for full ad-hoc container control (exec, filesystem, networking) suited to running untrusted or agent-generated code, Cls for stateful services with GPU-resident state across calls, Volume and NetworkFileSystem for persistent storage, Dict and Queue for lightweight distributed state, and Secret for credential injection. Every async primitive in the library is auto-mirrored into a blocking equivalent via an internal code-generation step, so the same API works from plain scripts and async applications alike. The package also bundles the modal CLI (modal run, modal deploy, modal shell) for deploying and managing apps outside of Python code.

What You Get

• Decorator-based function deployment (@app.function) with CPU/GPU/memory resource requests
• Programmatic container image definitions (modal.Image) built and cached remotely, no Dockerfile needed
• Sandbox objects for full ad-hoc container control — exec, filesystem access, networking — for running arbitrary or agent-generated code
• Persistent primitives — Volume, NetworkFileSystem, Dict, Queue, Secret — for state and storage across function calls
• A bundled modal CLI for deploying, running, and inspecting apps from the terminal
• Auto-generated sync and async APIs for every operation from one async implementation

Common Use Cases

• Running GPU-heavy AI/ML inference and training jobs without provisioning GPU instances directly
• Bursting batch data-processing or ETL jobs to serverless compute on demand
• Hosting FastAPI/ASGI web endpoints and background jobs that autoscale to zero
• Executing untrusted or LLM-agent-generated code in isolated, ephemeral sandboxes

Under The Hood

Architecture The Modal Python SDK is built around a lazy-hydration object model rooted in modal/_object.py’s _Object base class: every resource (App, Function, Volume, Secret, Dict, Sandbox, etc.) starts unhydrated and is materialized against the Modal control plane only when .hydrate() triggers a Resolver-driven load graph (modal/_resolver.py, modal/_load_context.py), which walks a _deps dependency callback to resolve prerequisite objects before hydrating dependents — a declarative dataflow graph rather than an imperative RPC sequence. Communication with the backend runs over gRPC via a thin wrapper (modal/_grpc_client.py, modal/client.py) around generated protobuf stubs in modal_proto/, with retry/backoff and auth-token management layered on top (modal/_utils/grpc_utils.py, auth_token_manager.py). The public surface (modal/app.py, modal/cls.py, modal/functions.py) is entirely decorator-driven — @app.function, @app.cls, @app.function.web_endpoint — registering _PartialFunction wrappers (modal/_partial_function.py) that get serialized (cloudpickle-based, modal/_serialization.py, modal/_vendor/) and shipped to remote containers whose entrypoint lives in modal/_container_entrypoint.py. Every internal module is implemented once as async, and the synchronicity library mechanically generates the blocking public API as a thin sync/async dual wrapper, so a change to _object.py’s hydration contract ripples through — and can break — every resource type that subclasses it.

Tech Stack The SDK targets Python 3.10–3.14 and layers on aiohttp for HTTP, grpclib (pinned narrowly per Python version) plus generated protobuf/grpc stubs for control-plane RPCs, cbor2 for compact serialization, click for the bundled CLI, rich for terminal output, and watchfiles for modal serve hot-reload. The critical dependency is synchronicity (a Modal Labs library), which auto-generates the synchronous public API from the async internal implementation at import time. Packaging uses setuptools with a dynamic version sourced from modal_version, and the repository is a polyglot monorepo — this PyPI package lives under py/ alongside sibling js/ (npm) and go/ SDKs sharing the same modal_proto/ protobuf definitions, so protocol changes propagate to three language clients from one source of truth. Dev tooling uses uv-managed dependency groups, ruff for lint/format, both mypy and pyright for type checking, pre-commit hooks, and a Bazel BUILD file alongside the public GitHub Actions workflows.

Code Quality The py/test/ directory holds roughly a hundred test files (client_test.py, cls_test.py, sandbox_test.py, container_test.py, app_composition_test.py, e2e_test.py, and more) run with pytest, pytest-asyncio for the async surface, pytest-timeout, and pytest-markdown-docs, which executes documentation examples as tests. Type checking is dual-enforced: mypy covers most of the tree, while pyright is deliberately scoped to files mypy structurally can’t check — modules that ship a .pyi stub alongside a .py implementation, such as cls.py, mount.py, and sandbox.py — a documented split rather than an oversight. Both modal/ and modal_proto/ ship py.typed markers. Error handling is explicit and typed through a dedicated modal/exception.py hierarchy (InvalidError, ExecutionError, etc.) rather than bare exceptions, and deprecations are versioned and dated rather than silently removed. A CodeQL SAST workflow runs in CI alongside the standard test workflow.

API Design The core ergonomic bet is that a plain Python function becomes a remotely-executed, autoscaled, GPU-backed cloud function by adding a single decorator — no separate deployment manifest, Dockerfile, or infra-as-code required for the common case — and modal.Image lets you declare the container image inline in Python rather than in a separate Dockerfile. Every async primitive is exposed twice for free (fn.remote() vs fn.remote.aio()) via the synchronicity wrapper, so the same codebase serves sync scripts and async apps without a parallel API to maintain. Sandboxes (modal/sandbox.py, the largest single module in the package) expose full ad-hoc container control — exec, filesystem, networking — as a first-class SDK object rather than a side feature, which is unusual for a “serverless functions” SDK and is clearly built for running arbitrary or agent-generated code, not just fixed jobs. The bundled CLI (modal run, modal deploy, modal shell) mirrors the Python API closely, and the repository distributes an installable skill for coding agents (modal skills install) that ships version-aligned reference docs, a deliberate DX investment for LLM-assisted usage.