baml

baml-py is the PyO3-based Python runtime that powers generated BAML clients. BAML (Basically A Made-up Language) is a small, TypeScript-like DSL for defining LLM functions with typed inputs and outputs; its compiler generates a baml_client package for your target language, and that generated code calls into this runtime for everything else — prompt rendering, provider dispatch, schema-aligned parsing of model output, streaming, and tracing.

Under the hood, baml-py wraps the shared Rust baml-runtime engine (the same core used by BAML’s TypeScript, Ruby, and Go clients) via pyo3, exposing async and sync clients, a TypeBuilder for defining runtime-dynamic schemas, a ClientRegistry for swapping model providers at runtime, and a Collector for capturing per-call logs, timing, and token usage. Its most distinctive piece is the JSON-ish parser baked into the runtime, which coerces imperfect or partially-streamed LLM output into the exact typed shape your BAML function declared, rather than requiring a strict JSON-mode round trip.

What You Get

• Async and sync generated clients (baml_client) backed by a shared Rust runtime, so both await b.MyFunction(...) and blocking call styles work from the same BAML definition
• BamlStream / BamlSyncStream wrappers that turn the FFI’s raw event stream into a normal Python async/sync iterator, so partial LLM output is a regular Python for-loop rather than a callback
• A TypeBuilder for defining or extending output schemas at runtime (e.g. dynamic enums/classes), without regenerating baml_client
• A ClientRegistry for overriding which LLM provider/model a function uses at call time, useful for testing or per-request routing
• A typed exception hierarchy (BamlValidationError, BamlClientHttpError, BamlClientFinishReasonError, BamlTimeoutError, …) instead of raw provider errors
• Built-in Collector/FunctionLog/Usage types for capturing per-call token usage, timing, and raw HTTP requests without external instrumentation

Common Use Cases

• Running the generated baml_client inside a FastAPI or async Python service to call LLMs with compile-time-checked input/output types
• Streaming partial structured output (e.g. a partially-filled JSON object) to a frontend while the model is still generating
• Swapping between model providers (OpenAI, Anthropic, local models) per environment via ClientRegistry without touching business logic
• Extending a BAML-defined schema with app-specific fields at runtime via TypeBuilder instead of maintaining multiple .baml variants
• Capturing structured logs of every LLM call (prompt, latency, token usage) via Collector for debugging or evals

Under The Hood

Architecture baml-py is a thin PyO3 (Rust) extension module built with maturin: engine/language_client_python/src/lib.rs registers the pyo3 module and delegates to src/runtime.rs, errors.rs, parse_py_type.rs, and serde_py.rs, which wrap the same baml-runtime engine crate shared by BAML’s TypeScript, Ruby, and Go clients. The Python-facing package (python_src/baml_py) is an ergonomics layer over that FFI surface: __init__.py re-exports the compiled pyo3 symbols, ctx_manager.py threads tracing context across async/thread boundaries via contextvars, stream.py bridges the Rust FunctionResultStream into a Python async/sync iterator using a background thread and queue.Queue, and type_builder.py exposes the Rust TypeBuilder/ClassBuilder/EnumBuilder through a fluent Python API. Because the core logic lives in the shared Rust crate, the pyo3 FFI contract in lib.rs/runtime.rs is the abstraction every language binding depends on — changing it cascades to every BAML client, not just Python.

Tech Stack Built on pyo3 0.23 with the abi3-py38 feature for a stable ABI across Python versions, plus pyo3-async-runtimes to bridge Rust’s Tokio runtime into Python’s asyncio. The crate depends on internal workspace crates (baml-runtime, baml-compiler, baml-cli, baml-types, internal-baml-core, and the jsonish schema-aligned parser) rather than duplicating logic, and is packaged via maturin with python-source = "python_src". The Python side targets 3.10+, ships a py.typed marker for full static typing, and declares no runtime Python dependencies of its own — everything beyond stdlib (asyncio, contextvars, threading, queue) comes from the compiled extension.

Code Quality The Rust crate has no colocated unit tests (no #[test] or #[cfg(test)] in the FFI layer); testing instead happens end-to-end in the sibling integ-tests/python-v1 project via pytest-asyncio against a real generated baml_client. Error handling is deliberate: errors.rs/errors.py convert native panics and anyhow errors into a typed Python exception hierarchy rather than leaking raw Rust errors. The Rust side enables deny-level lints for dead_code, unused_imports, unused_variables, and unused_must_use, and the Python side is fully typed with a py.typed marker. CI includes dedicated release workflows for building and publishing the Python wheel plus a repo-wide pre-commit config, though the crate would benefit from tests that don’t require the full codegen pipeline to exercise.

API Design The public surface is intentionally small and hides FFI plumbing behind idiomatic Python: BamlStream/BamlSyncStream present a plain iterator interface over what is internally a background thread and event queue, TypeBuilder offers a chainable API (tb.Person.add_property(...).description(...)) instead of exposing the raw Rust builder types, and CtxManager propagates tracing context automatically so callers never pass it manually. The trade-off is that baml-py read in isolation is a thin, mostly re-exported FFI surface — its real ergonomics only appear once paired with the .baml compiler that generates the baml_client code actually calling into it.