duckdb-rs

duckdb-rs is the Rust binding for DuckDB, an embedded analytical (OLAP) SQL database engine. It compiles or links DuckDB’s C++ engine directly into your binary and wraps its raw C API in a safe, ergonomic layer whose design deliberately mirrors rusqlite: open a Connection, prepare a Statement, bind parameters, and map rows into typed Rust structs via ToSql/FromSql.

Beyond row-oriented queries, the crate treats Apache Arrow as a first-class result format — query_arrow returns RecordBatches directly, and the optional polars feature returns DataFrames — so applications already built on the Arrow ecosystem get zero-conversion interop with an in-process analytical database. It also supports writing custom scalar/table functions and full loadable DuckDB extensions in Rust, going beyond what most database driver crates expose.

What You Get

• A safe Connection/Statement/Row API over DuckDB’s C FFI, with typed parameter binding and result mapping via ToSql/FromSql
• Native Apache Arrow interchange (query_arrow) and optional Polars DataFrame interchange, avoiding manual result-set conversion
• An Appender for high-throughput bulk loads, matching DuckDB’s native Appender interface
• Support for authoring custom scalar/table functions and full loadable DuckDB extensions from Rust
• Multiple build backends (bundled, bundled-cmake, system-linked) with pregenerated FFI bindings so most consumers never need bindgen or Clang

Common Use Cases

• Embedding in-process analytical SQL into a Rust service or CLI without running a separate database server
• ETL and data-pipeline tooling that ingests/exports Parquet, CSV, and JSON via DuckDB’s native readers and the Appender
• Arrow-native applications (including those built on Polars) that want a queryable analytical engine with zero-conversion result interchange
• Building custom DuckDB extensions or user-defined functions in Rust instead of DuckDB’s C++/Python extension toolchains

Under The Hood

Architecture The crate splits into three workspace members: duckdb (crates/duckdb) — the safe, ergonomic Rust API; libduckdb-sys (crates/libduckdb-sys) — raw FFI bindings generated against DuckDB’s C API, with a build script that either bundles and compiles DuckDB from source (bundled/bundled-cmake features) or links a system/pre-downloaded library; and duckdb-loadable-macros — proc-macros for building loadable DuckDB extensions. The safe layer wraps raw pointers from libduckdb-sys inside InnerConnection (inner_connection.rs) and RawStatement (raw_statement.rs), exposing Connection, Statement (statement.rs), Row/Rows (row.rs), and Appender for bulk loads. Type conversion is centralized in the types/ module (value.rs, value_ref.rs, to_sql.rs, from_sql.rs, plus optional chrono/decimal/serde_json/url adapters gated by Cargo features), and Arrow interop flows through arrow_batch.rs and arrow_interop/. Extensibility (custom scalar/table functions, virtual tables) lives in vscalar/ and vtab/. The dependency direction is strict: duckdb depends on libduckdb-sys, never the reverse, so upgrading the underlying DuckDB engine is a change isolated to libduckdb-sys and its build script.

Tech Stack Rust 2024 edition on a rolling MSRV that trails the current DuckDB release by at least six months. The core dependency is arrow (with prettyprint/ffi features) for Arrow-native result handling; optional integrations add polars/polars-core for DataFrame interchange, chrono and rust_decimal for typed date/time/decimal handling, serde_json for JSON columns, r2d2 for pooled connections via DuckdbConnectionManager, and uuid/url. FFI plumbing uses cast, hashlink, num, num-integer, and strum for enum handling. Feature-gated build backends cover bundled (the cc crate compiles DuckDB’s amalgamated C sources), bundled-cmake (required for extensions like icu/tpcds/tpch), buildtime_bindgen (regenerate FFI bindings via bindgen instead of using pregenerated ones), and vcpkg/pkg-config for system-linked builds. CI runs a Linux/Windows matrix using sccache and a downloaded prebuilt DuckDB library rather than compiling DuckDB from source on every run.

Code Quality Tests are extensive and embedded directly alongside implementation code across the crate, plus a dedicated nightly test file and example programs that double as executable documentation for the Appender, Arrow virtual tables, Parquet, scalar/table functions, and a loadable extension. Error handling is explicit and typed via a non-exhaustive Error enum covering DuckDB call failures, SQL-to-Rust conversion failures, out-of-range integral values, invalid parameter names, and more — never string-typed or silently swallowed. The crate root enforces #![warn(missing_docs)], requiring doc comments on public items, and a checked-in CI workflow runs the test matrix on every push and pull request across both target platforms.

What Makes It Unique Unlike most Rust SQL crates that talk to a client/server database or a lightweight embedded engine, duckdb-rs embeds a full OLAP-grade analytical database directly in-process while giving Arrow-native access to results — bridging row-oriented APIs and columnar analytics in the same connection. It also supports authoring DuckDB extensions in Rust itself, letting users write custom scalar/table functions and compile them as loadable DuckDB extensions rather than only consuming the database, a capability most database driver crates don’t expose. The dual bundled/system-link build system, with pregenerated FFI bindings shipped by default, is a deliberate tradeoff favoring downstream build-time ergonomics.