asyncpg

asyncpg is a database driver for PostgreSQL written specifically for Python’s asyncio, avoiding the generic DB-API abstraction layer that most Python database libraries use. Instead of wrapping a synchronous C library or speaking a text-based protocol, asyncpg implements PostgreSQL’s binary wire protocol directly in Cython, which lets it decode rows into native Python objects (or fast Record objects) with minimal overhead. The maintainers report roughly 5x throughput over psycopg3 on typical workloads.

Because it exposes PostgreSQL’s protocol features directly rather than hiding them behind a lowest-common-denominator API, asyncpg gives first-class support for prepared statements, server-side cursors with partial iteration, and automatic encoding/decoding of composite types, arrays, ranges, and custom types. It requires Python 3.9+, supports PostgreSQL 9.5 through 18, and needs no external dependencies for the base install (GSSAPI/SSPI auth is an optional extra).

asyncpg is widely used as the underlying PostgreSQL driver for higher-level tools such as SQLAlchemy’s async engine and Databases, making it a common dependency in async web frameworks (FastAPI, Starlette) that need real database concurrency rather than thread-pool-wrapped synchronous drivers.

What You Get

• A connect() coroutine and a connection-pool API (create_pool) for managing PostgreSQL connections under asyncio
• Prepared statement support with automatic caching, plus explicit prepare() for reuse across many queries
• Server-side cursors with partial iteration, so large result sets can be streamed instead of buffered in memory
• Automatic encoding/decoding of PostgreSQL composite types, arrays, ranges, and JSON/JSONB without manual serialization
• Fast Record objects (implemented in C) returned from queries instead of generic dict-like rows
• Transaction and savepoint management via async context managers
• COPY protocol support for bulk data import/export directly from Python iterables or files

Common Use Cases

• Backing async web frameworks like FastAPI or Starlette with real non-blocking PostgreSQL access instead of a thread-pool-wrapped sync driver
• Powering the async engine of higher-level ORMs and query layers such as SQLAlchemy’s asyncio support or the databases library
• Bulk-loading large datasets into PostgreSQL efficiently via native COPY support
• Streaming large query results with server-side cursors in ETL or reporting pipelines without exhausting memory
• High-throughput services (APIs, job queues, analytics ingestion) where PostgreSQL round-trip latency and driver overhead directly affect request throughput

Under The Hood

Architecture asyncpg is organized as a thin, well-factored asyncio driver: connection.py defines the public Connection class and top-level connect() coroutine (asyncpg/connection.py:2083) that negotiates connection parameters (DSN, host/port, SSL, server settings) and hands off to the Cython-implemented protocol package for the actual wire-level exchange; pool.py layers a Pool/PoolConnectionHolder/PoolConnectionProxy design on top of Connection to provide bounded, reusable connection pools with the same query surface (execute, fetch, executemany at pool.py:577-613) proxied transparently to an acquired connection. cursor.py, prepared_stmt.py, and transaction.py each wrap one PostgreSQL protocol concept as its own class, keeping concerns separated: statement caching lives in connection.py’s _StatementCache, while type encoding/decoding is fully isolated in the protocol/codecs/ Cython modules (array.pyx, range.pyx, record.pyx, pgproto.pyx). This separation means the core abstraction that would be most costly to change is the binary protocol layer in protocol/coreproto.pyx — everything above it (Connection, Pool, Cursor) is built assuming synchronous-looking coroutines over that protocol.

Tech Stack The library is Python 3.9+ with its performance-critical path written in Cython (protocol/*.pyx, pgproto/) compiled to a C extension via setuptools and a custom build_ext in setup.py, with Cython pinned to >=3.2.1,<4.0.0 at build time. It has zero runtime dependencies for the base install beyond async_timeout on Python <3.11; GSSAPI/SSPI Kerberos auth is gated behind an optional gssauth extra (gssapi/sspilib). There is no ORM or web-framework dependency — asyncpg is consumed by higher-level layers (SQLAlchemy’s asyncio engine, databases) rather than depending on them. Testing pulls in pytest, flake8 (with flake8-pyi for the .pyi stub files), mypy in strict mode, and uvloop for testing against an alternate event loop implementation.

Code Quality The tests/ directory contains dozens of focused test modules (test_connect.py, test_pool.py, test_cursor.py, test_codecs.py, test_transaction.py, test_cancellation.py, test_adversity.py for failure injection, and more), run via pytest with --strict-markers and coverage tracking configured through tool.coverage in pyproject.toml, including Cython source coverage via the Cython.Coverage plugin. mypy runs in strict = true mode across the pure-Python modules (a subset of internal modules such as connection, pool, and exceptions are explicitly exempted via ignore_errors, suggesting the typed surface is intentionally the public API rather than every internal helper). flake8 enforces style, and CI (.github/workflows/tests.yml) runs the suite against real PostgreSQL instances across supported versions and platforms (Linux/macOS/Windows), which is notably more rigorous than driver libraries that only mock the database layer.

What Makes It Unique Unlike DB-API-style drivers (psycopg and friends) that route through a generic cursor abstraction designed to work the same way across many databases, asyncpg commits fully to PostgreSQL’s own binary wire protocol and exposes PostgreSQL-specific capabilities — typed composite/array/range decoding, native COPY support, server-side cursor iteration — as first-class API rather than afterthoughts. Implementing the protocol parsing and row materialization in Cython rather than pure Python is the specific technical choice behind its reported ~5x throughput advantage over psycopg3, since it avoids both the DB-API abstraction tax and much of the interpreter overhead on the hot path of decoding rows.