PyAV

PyAV provides direct, Pythonic access to FFmpeg’s underlying C libraries — avformat, avcodec, avdevice, avutil, avfilter, swscale, and swresample — for demuxing, decoding, encoding, and muxing audio, video, and subtitle streams. Rather than shelling out to the ffmpeg command-line tool and parsing its output, PyAV exposes containers, streams, packets, codecs, and frames as first-class Python objects, giving precise, low-level control over media pipelines while PyAV manages FFmpeg’s C-level memory and threading details.

Frames interoperate directly with NumPy arrays and Pillow images, and support zero-copy tensor exchange via DLPack, making PyAV a common building block for video pipelines that feed machine-learning models, computer-vision tools, or custom transcoding and streaming servers. Binary wheels bundle a matching FFmpeg build for Linux, macOS, and Windows, so most users get hardware-accelerated decoding and full codec support without compiling FFmpeg themselves.

What You Get

• Direct container/stream/packet/frame API for demuxing and muxing any FFmpeg-supported format
• Prebuilt binary wheels bundling FFmpeg 8.x for Linux, macOS, and Windows — no system FFmpeg build required
• Hardware-accelerated decoding via cuda, d3d11va, videotoolbox, vaapi, and qsv backends
• NumPy, Pillow, and DLPack interop for feeding decoded frames straight into ML/CV pipelines
• Typed Python API via hand-written .pyi stubs across every module, checked with mypy

Common Use Cases

• Transcoding and remuxing video/audio between containers and codecs from Python scripts
• Extracting frames or keyframes from video for thumbnailing, ML datasets, or computer vision
• Building custom screen and webcam recorders or streaming pipelines
• Decoding video directly to NumPy arrays or PIL Images for analysis

Under The Hood

Architecture PyAV is organized as one Python package per FFmpeg subsystem — av/container (core, input, output, streams, a custom pyio layer), av/codec (codec, context, hwaccel), av/audio, av/video, av/filter, av/subtitles, and av/sidedata — each pairing a .pxd C-level declaration file with a .py implementation and a .pyi type stub. The entry point, av/init.py, deliberately imports av._core before anything else to initialize the wrapped FFmpeg libraries, then re-exports the public API. av.container.open() dispatches to InputContainer or OutputContainer, both backed by a PyIO layer that lets arbitrary Python file-like objects supply FFmpeg’s I/O, plus an interrupt callback for timing out blocking network reads. Data flows container → demux → packet → decode → frame (or the reverse for encoding/muxing); since every subsystem imports av._core at load time, changes to that core binding layer or its .pxd declarations ripple across the whole package.

Tech Stack Requires Python 3.12+ and is built with setuptools plus Cython 3.3+, using Cython’s newer “pure Python” syntax (cython.cfunc, cython.cimports, typed annotations) rather than legacy .pyx files. Prebuilt wheels bundle FFmpeg 8.x statically for Linux, macOS, and Windows; building from source instead links against a system FFmpeg via pkg-config (pip install av --no-binary av). Dev tooling includes pytest, NumPy, and Pillow for interop tests, plus ruff, isort, and mypy for linting, all declared as pyproject.toml dependency groups. The package also ships a pyav CLI entry point and an av.datasets module of curated sample media used in docs and tests.

Code Quality Thirty-six test files exercise audio/video frames, codecs, containers, bitstream filters, subtitles, hardware acceleration, and DLPack interop via pytest, run through GitHub Actions CI. Type safety is unusually thorough for a C-extension package: nearly every module ships a hand-written .pyi stub checked with mypy, on top of Cython source that is itself typed rather than untyped .pyx. Errors funnel through a centralized err_check/stash_exception mechanism in av.error that converts FFmpeg’s C return codes into a typed Python exception hierarchy, and naming consistently mirrors FFmpeg’s own vocabulary (Codec, Packet, Stream, Frame). Lint is enforced via a dedicated ruff/isort/mypy dependency group run through make lint.

API Design PyAV’s core design bet is exposing FFmpeg’s own mental model — open a container, iterate streams, demux packets, decode frames — directly as Python objects instead of hiding it behind a higher-level abstraction, which lets developers avoid subprocess-and-pipe hacks around the ffmpeg CLI while keeping precise, low-level control. Interop methods like VideoFrame.to_ndarray(), to_image(), and DLPack export give frames a clean path into NumPy, Pillow, and tensor-consuming ML code. The API asks more boilerplate of callers than higher-level wrappers (manual demux/decode loops, explicit hwaccel setup), but that mirrors the complexity of the domain rather than papering over it, and is comprehensive documentation and cookbook examples make the tradeoff approachable.