impyla

impyla is a Python client for HiveServer2-compliant distributed query engines such as Apache Impala and Apache Hive. It implements the DB API 2.0 (PEP 249) interface, so connecting, executing SQL, and iterating cursors feels identical to using sqlite3 or a MySQL driver, while supporting nested data and enterprise authentication including Kerberos, LDAP, SSL, and JWT.

Beyond the raw DB API, impyla ships a SQLAlchemy dialect and a converter that turns query results into pandas DataFrames, making it a natural entry point for connecting the Python data stack to Hadoop-scale SQL engines.

What You Get

• A DB API 2.0 (PEP 249) compliant connection and cursor interface
• HiveServer2 transport working with both Impala and Hive, including nested data
• Enterprise auth support: Kerberos, LDAP, SSL, and JWT
• A SQLAlchemy dialect and a pandas DataFrame converter for the data stack

Common Use Cases

• Running SQL queries against Impala or Hive from Python applications
• Loading Hadoop query results into pandas DataFrames for analysis
• Using SQLAlchemy against Impala through the bundled dialect

Under The Hood

Architecture - The impala/dbapi.py module exposes the PEP 249 connect() entry point, delegating to hiveserver2.py, which drives the Thrift-generated HiveServer2 API in _thrift_gen through _thrift_api.py; sqlalchemy.py adds a dialect and util.py provides the pandas conversion, with interface.py and error.py defining the shared contracts and exception hierarchy. Tech Stack - Pure Python on top of thrift and thrift_sasl for transport and SASL, bitarray for result decoding, and optional kerberos and SQLAlchemy dependencies; packaged with setuptools and pyproject metadata, tested via tox across Python versions. Code Quality - The repository maintains a substantial test suite (DB API 2.0 compliance tests, data-type tests, HTTP and fault-injection tests under impala/tests) and is actively maintained by Cloudera with 80+ contributors. API Design - Because it adheres to the familiar DB API 2.0 contract, developers already know the surface — connect, cursor, execute, fetchall — and the SQLAlchemy and pandas helpers reduce boilerplate for analytics workflows, though connection setup for secured clusters requires understanding the auth options.