PyHive

PyHive is a Python library from Dropbox that implements PEP 249 (DB-API 2.0) cursors and connections for Apache Hive, Presto, and Trino, plus SQLAlchemy dialects that register directly with SQLAlchemy’s engine system. Instead of writing separate integration code for each query engine, applications can use the same cursor.execute()/fetchall() pattern across all three backends, or hand a connection string like presto://host:8080/hive/default straight to create_engine() and get automatic table reflection, inspection, and ORM support.

Under the hood it talks to HiveServer2 over Thrift for Hive, and to the REST APIs for Presto and Trino, translating each engine’s native protocol into the same DB-API surface. It supports both synchronous and asynchronous query execution, session-level configuration overrides, and authentication options including LDAP, Kerberos (via GSSAPI), and HTTPS with configurable certificate verification.

The project has been in production use at Dropbox and widely adopted by tools like Apache Superset for connecting to Hive/Presto/Trino clusters. As of 2026 the upstream README notes the project has been donated to Apache Kyuubi, where active development continues; this PyPI package remains the long-standing, widely-depended-on entry point for existing integrations.

What You Get

• Three DB-API 2.0 modules (hive, presto, trino) with connect(), cursor(), execute(), and fetchone/fetchmany/fetchall sharing one base cursor implementation
• SQLAlchemy dialects (hive, hive.http, hive.https, presto, trino.pyhive) registered via setuptools entry_points so create_engine() works out of the box
• Synchronous and asynchronous query execution against Hive, with polling helpers to check operation state and fetch logs mid-query
• Authentication support for LDAP, Kerberos/GSSAPI, and HTTPS with configurable certificate verification modes (none/optional/required)
• Parameter escaping helpers (ParamEscaper subclasses per backend) for safely interpolating values into generated SQL
• A typed DB-API exception hierarchy (Error, Warning, InterfaceError, DatabaseError, etc.) so calling code can catch failures the standard way

Common Use Cases

• Connecting BI and visualization tools (e.g. Apache Superset) to a Hive, Presto, or Trino cluster via SQLAlchemy
• Running ad-hoc or scheduled analytical queries against a Hadoop/Hive data warehouse from Python ETL scripts
• Querying Presto/Trino clusters over HTTP(S) from data pipelines that need session-level query tuning (e.g. query_max_run_time)
• Building pandas.read_sql-based reporting pipelines against Hive or Presto without hand-rolling a Thrift/REST client

Under The Hood

Architecture A single abstract base, common.DBAPICursor, owns the DB-API state machine shared by every backend: row buffering via a deque, poll-and-fetch looping (_fetch_while), and the standard fetchone/fetchmany/fetchall/iterator protocol. hive.py, presto.py, and trino.py each subclass it and supply only what differs per engine: hive.py drives HiveServer2 over generated Thrift stubs (TCLIService) with support for synchronous and asynchronous operation polling; presto.py and trino.py are thin, largely stateless REST clients built on requests. A parallel set of SQLAlchemy dialect classes (sqlalchemy_hive.py, sqlalchemy_presto.py, sqlalchemy_trino.py) sits above the DB-API layer and is wired in purely through setup.py entry_points, so create_engine() resolves a dialect without any pyhive-specific glue code in the caller. The result is a clean three-way split (shared cursor logic, per-engine transport, per-engine SQLAlchemy dialect) with no cross-backend coupling; changing how Hive polling works, for example, only touches hive.py.

Tech Stack The codebase targets both Python 2.7 and Python 3 simultaneously, using future/past polyfills (from builtins import ..., past.builtins.basestring) rather than a 2to3 or six-based split. Hive connectivity depends on Apache Thrift’s generated TCLIService client plus optional sasl/pure-sasl and thrift_sasl packages for GSSAPI/LDAP authentication; Presto and Trino connectivity depends only on requests. SQLAlchemy support requires sqlalchemy>=1.3.0 and is registered declaratively through setuptools entry points rather than runtime registration calls. Testing runs under pytest with pytest-cov and pytest-flake8, orchestrated historically through Travis CI (.travis.yml) against real Hive/Presto/Trino instances rather than mocks alone.

Code Quality The pyhive/tests/ directory has eight test modules, including shared dbapi_test_case.py and sqlalchemy_test_case.py base classes that parameterize common DB-API and dialect behavior across backends, plus dedicated suites for SASL compatibility, Hive, Presto, and Trino (both DB-API and SQLAlchemy layers). Errors are surfaced through an explicit exc.py exception hierarchy (Error, Warning, InterfaceError, DatabaseError, etc.) rather than being swallowed. There are no static type annotations anywhere in the codebase, consistent with its Python 2/3-compatible age, and CI is still configured for the now-legacy Travis CI rather than a modern pipeline. Recent commit activity is low; the maintainers’ own README now points users to Apache Kyuubi’s fork for active development.

API Design The library’s core value is presenting three otherwise-incompatible query engines through one familiar interface: standard DB-API connect().cursor().execute() semantics for Hive, Presto, and Trino, and standard SQLAlchemy connection-string configuration (hive://, presto://, trino+pyhive://) for anything already built on SQLAlchemy. Getting started requires no more boilerplate than presto.connect('localhost').cursor() or a single create_engine() call, and connection kwargs (configuration=, session_props=, Kerberos options) follow consistent naming across backends. The rough edges are backend-specific quirks documented directly in the README, such as the async/async_ keyword change in Python 3.7 and extras that must be chosen at install time (pyhive[hive] vs pyhive[presto] vs pyhive[trino]).