pyarrow-hotfix

pyarrow-hotfix is a single-purpose Python package written by Apache Arrow committer Antoine Pitrou to address CVE-2023-47248, a critical deserialization vulnerability in PyArrow that allows arbitrary code execution when reading untrusted Parquet or Feather files containing a PyExtensionType column. Rather than requiring every downstream project to force an upgrade to PyArrow 14.0.1 or later, the package can be installed alongside any PyArrow version from 0.14.0 onward and patches the vulnerable deserialization path at import time.

The fix works by monkey-patching PyArrow’s extension-type registry so that any attempt to deserialize the dangerous arrow.py_extension_type raises a RuntimeError with a clear explanation instead of silently unpickling attacker-controlled bytes. Because many organizations pin PyArrow versions for compatibility with other data tooling (pandas, Dask, Spark connectors), pyarrow-hotfix gives them a way to close the security hole immediately while planning a longer-term upgrade path.

The package has no runtime dependencies of its own, supports Python 3.5+, and is tested against PyArrow releases spanning 0.14.0 through 19.x across Linux, Windows, and macOS in CI. It is intentionally minimal — a single module that does one job and gets out of the way.

What You Get

• Import-and-forget activation - a single import pyarrow_hotfix call installs the patch automatically via a module-level install() invocation.
• Broad version coverage - works across PyArrow 0.14.0 through the latest releases, with version-specific patch logic for each internal API PyArrow has exposed over time.
• Zero runtime dependencies - the package only touches PyArrow’s own objects at runtime; it ships no third-party dependencies of its own.
• Explicit opt-out - pyarrow_hotfix.uninstall() reverts the patch for cases where trusted extension-type deserialization is genuinely needed.
• Cross-platform CI verification - the test matrix exercises dozens of PyArrow/Python/OS combinations to confirm the patch behaves correctly on each.

Common Use Cases

• Pinned dependency environments - teams running older PyArrow versions for compatibility with pandas, Spark, or Dask can close the CVE without a disruptive upgrade.
• Data platform hardening - services that ingest Parquet/Feather files from external or semi-trusted sources add the hotfix as a defense-in-depth measure.
• Transitional upgrade windows - organizations mid-migration to PyArrow 14+ use the hotfix as a stopgap until the upgrade is fully rolled out.
• Vulnerability scanner remediation - security teams add pyarrow-hotfix to satisfy automated CVE scanners flagging CVE-2023-47248 in dependency audits.

Under The Hood

Architecture The entire package is a single module, src/pyarrow_hotfix/__init__.py, exposing two functions, install() and uninstall(), with install() invoked automatically at import time. There is no class hierarchy, no plugin system, and no configuration layer — the module directly reaches into the imported pyarrow package’s internals (pa.unregister_extension_type, pa.lib._unregister_py_extension_type, pa.lib._extension_types_initializer) and branches on which of these APIs exists to apply the correct patch for the installed PyArrow version. A ForbiddenExtensionType subclass of pa.ExtensionType is defined inline and swapped in as the handler for the arrow.py_extension_type name, so that any deserialization attempt raises RuntimeError instead of executing arbitrary unpickled code. Because the whole surface area is one file with no external dependencies, there is nothing to break if a core abstraction changes beyond PyArrow’s own internal API evolving.

Tech Stack The package is pure Python with zero runtime dependencies, packaged via Hatchling (pyproject.toml, [build-system] requires = ["hatchling"]) and versioned through a single __about__.py file consumed by tool.hatch.version. It targets Python 3.5+ and declares no dependencies of its own — its only real integration point is the pyarrow package it patches at runtime, which is treated as an optional, dynamically-imported dependency rather than a hard requirement. The test environment (tool.hatch.envs.hatch-test) pulls in pyarrow and pytest only for testing, keeping the installed footprint minimal.

Code Quality Testing is thorough relative to the package’s size: tests/test_in_process.py and tests/test_subprocess.py cover both in-process and subprocess-isolated behavior, using real .arrow and .pq fixture files to exercise IPC and Parquet deserialization paths under both the patched and uninstall()-reverted states. The GitHub Actions workflow runs this suite across roughly two dozen PyArrow versions (0.14.0 through 19.0.1) and multiple Python versions on Linux, Windows, and macOS, which is unusually extensive CI coverage for a package this small and reflects the version-branching complexity the patch logic has to handle correctly. There are no type annotations and no linter/formatter configuration checked in, and error handling is minimal by design — the module either silently no-ops (missing PyArrow, unsupported version) or deliberately raises a loud RuntimeError when blocking a deserialization attempt.

API Design The public surface is deliberately tiny: two functions, install() (called automatically on import) and uninstall(). There is no configuration to learn and no boilerplate — import pyarrow_hotfix is the entire integration for the common case, which is a strong ergonomic choice for a security patch meant to be adopted with minimal friction. The tradeoff is that the auto-install-on-import side effect is implicit magic that some consumers may find surprising, and the reversible uninstall() escape hatch is undocumented in type hints or docstrings, relying entirely on the README for discovery.