PyO3
Rust bindings for the Python interpreter, for writing native extensions or embedding Python in Rust.
Repository Health
Technical Analysis
PyO3 provides safe, ergonomic Rust bindings to the CPython, PyPy, and GraalPy interpreters, letting you write native Python extension modules in Rust or embed a Python interpreter inside a Rust binary. Procedural macros like #[pyclass], #[pyfunction], and #[pymodule] generate the FFI glue automatically, so most day-to-day code never touches the raw C API exposed by the companion pyo3-ffi crate.
Paired with maturin or setuptools-rust for packaging, PyO3 is the foundation for a large share of the performance-critical Python ecosystem — projects like polars, pydantic-core, cryptography, and ruff embed Rust logic behind a Python-facing API built on PyO3.
What You Get
- #[pyclass], #[pyfunction], and #[pymodule] macros that generate the C API glue for exposing Rust types and functions to Python
- GIL-aware smart pointers (Py<T>, Bound<‘py, T>) that make CPython’s reference-counting and interpreter-lock rules visible in the Rust type system
- A pyo3-build-config crate that auto-detects CPython/PyPy/GraalPy installations and wires up linking, abi3, and free-threaded builds
- First-class conversions for common crates via optional features (chrono, num-bigint, serde, uuid, indexmap, hashbrown) with no hand-written glue required
- An async bridge (via pyo3-async-runtimes) plus an experimental-async feature for #[pyfunction]
Common Use Cases
- Rewriting a CPU-bound Python hot path (parsing, crypto, numeric kernels) as a native Rust extension published to PyPI via maturin
- Embedding a Python interpreter inside a Rust CLI or service to run user-supplied scripts or plugins
- Building a Rust core library once and exposing the same functionality to Python users as an ergonomic wheel, alongside a native Rust API
- Wrapping an existing C/C++ library from Rust and re-exposing it to Python without writing a second, Python-specific binding layer
Under The Hood
Architecture - The workspace splits responsibilities across eight crates: pyo3-ffi holds the raw, unsafe bindings to the CPython C API; pyo3-build-config (driven by build.rs) probes the target Python installation and emits the cfg flags that select CPython/PyPy/GraalPy, abi3, and free-threaded code paths; pyo3-macros and pyo3-macros-backend implement the #[pyclass]/#[pyfunction]/#[pymodule] procedural macros; pyo3-introspection extracts .pyi type stubs from compiled crates; and the top-level pyo3 crate (src/marker.rs for the Python<‘py> GIL token, src/instance.rs for Py<T>/Bound<‘py,T> smart pointers, src/pyclass/ for class wrapping, src/types/ for built-in Python type wrappers, src/conversions/ for FromPyObject/IntoPyObject impls) assembles these into the public, safe API that most users interact with.
Tech Stack - Rust 2021 edition with a minimum supported rustc of 1.83, organized as a Cargo workspace of pinned-version path dependencies (pyo3-ffi, pyo3-macros, pyo3-build-config all locked to =0.29.0). Core runtime dependencies are deliberately minimal (libc, once_cell, portable-atomic as a fallback for platforms without 64-bit atomics); a long list of optional integration crates (chrono, chrono-tz, bigdecimal, bytes, either, hashbrown, indexmap, jiff, num-bigint/num-complex/num-rational, ordered-float, rust_decimal, serde, smallvec, uuid, parking_lot) are gated behind Cargo features so consumers only compile what they use. #![no_std] at the crate root keeps the core binding surface allocator-agnostic.
Code Quality - 111 integration test files under tests/ cover class semantics, conversions, exceptions, buffers, coroutines, and more, plus a dedicated UI-test harness (tests/test_compile_error.rs + tests/ui) that asserts macro misuse produces the intended compile errors. Workspace-wide Clippy and rustc lints are enforced as warnings, including undocumented_unsafe_blocks and unsafe_op_in_unsafe_fn, which is notable given that the crate’s entire purpose is wrapping unsafe FFI calls; #![warn(missing_docs)] additionally forces documentation coverage across the public API. CI runs Codecov coverage and Codspeed performance benchmarks on every change.
API Design - The procedural-macro layer (#[pyclass], #[pyfunction], #[pymodule]) collapses what would otherwise be hundreds of lines of manual CPython C API calls into a few attributes, and the prelude module gathers the commonly needed traits and types into one import. Getting started is a maturin init away, and an extensive mdBook guide (guide/src/, 20+ chapters covering classes, conversions, async, free-threading, and a dedicated migration guide per release) plus a curated ecosystem page keeps the learning curve manageable for a topic — Rust/Python FFI plus the GIL — that is intrinsically hard.
Used by 11 apps in this directory
Arroyo
Data Engineering · Analytics
A distributed stream processing engine written in Rust that lets you write SQL to run stateful, real-time computations over data streams with subsecond results.
Databend
Databases · Data Engineering
Open-source enterprise data warehouse unifying analytics, vector search, full-text search, and AI agent orchestration in a single Rust-built engine on S3.
LanceDB
Databases · AI Development
Open-source, embedded vector database built on the Lance columnar format for fast multimodal search across billions of vectors, backed by Y Combinator (W23).
liteparse
Developer Tools
A fast, lightweight, open-source document parser that extracts spatial text, bounding boxes, and Markdown from PDFs and Office files — entirely on your machine.
Morphik
AI Development · Search · Databases
Morphik is an AI-native ingestion and retrieval engine that lets developers store, search, and reason over visually rich documents — scanned PDFs, manuals, slides, and video — without duct-taping together OCR, an embedding model, and a vector database.
OpenViking
Databases · AI Development
An open-source context database that gives AI agents a unified filesystem for memory, resources, and skills with hierarchical tiered retrieval.
PostgresML
Databases · AI Development
Run ML training and LLM inference natively inside PostgreSQL with GPU acceleration — no data movement required.
PostHog
Analytics · Monitoring · Developer Tools
The all-in-one open source product platform combining analytics, session replay, feature flags, error tracking, AI observability, and a built-in data warehouse in a single self-hostable stack.
Qdrant
Databases · AI Development · Search
Open-source vector database and search engine built in Rust for production-grade AI applications — from semantic search to RAG pipelines and recommendation systems.