PyMuPDF Layout

PyMuPDF Layout is a document layout analysis library for PDFs, distributed as a companion package to PyMuPDF. Instead of rendering each page to an image and running a vision model over the pixels, it trains a Graph Neural Network (BoxRFDGNN) directly on the PDF’s internal text, vector, and image primitives extracted through MuPDF. That lets it detect titles, headings, headers, footers, tables, and text styling on CPU-only hardware, without a GPU, at a fraction of the runtime cost of image-based layout models.

The package ships a compiled features extension (built with SWIG against MuPDF’s C++ API) alongside a pure-Python pymupdf.layout module that wraps ONNX Runtime inference, table-grid extraction, and Markdown/HTML generation. It is the layout engine behind PyMuPDF4LLM: calling pymupdf.layout.activate() wires a trained model into PyMuPDF’s document-to-Markdown pipeline, so any code already using PyMuPDF4LLM gets structured extraction with a single import.

What You Get

• A compiled features C extension (SWIG-wrapped against MuPDF) that extracts structural features per page region for the layout model to consume
• A BoxRFDGNN graph neural network, served via ONNX Runtime, that classifies text, tables, images, and header/footer regions without a GPU
• Multiple table-grid extractor implementations (V1 through V3, plus A/B variants) for detecting and reconstructing table structure at different accuracy/speed tradeoffs
• Markdown, HTML-table, and plain-text generators (MarkdownGenerator, MarkdownHTMLTableGenerator, HTMLGenerator) that turn the model’s layout graph into final document output
• A MultiProcessWrapper that runs layout inference across a worker pool for batch processing multi-page PDFs

Common Use Cases

• RAG document ingestion - convert PDFs into clean Markdown chunks that preserve table and heading structure before embedding
• Automated report parsing - extract structured data from invoices, financial statements, and forms without training a custom vision model
• Header/footer stripping - detect and drop repeating page furniture before downstream text processing
• Table extraction pipelines - pull table grids out of PDFs as structured Markdown or HTML tables instead of raw text runs

Under The Hood

Architecture PyMuPDF Layout’s architecture layers a compiled feature-extraction layer under a pure-Python ML layer. At the bottom, source/features.i (SWIG) wraps MuPDF’s fz_features_for_region() C API to compute per-region structural features directly from PDF internals, with no rasterization step. source/layout/pymupdf_util.py builds model input from a pymupdf.Page via create_input_data_from_page(), and onnx/BoxRFDGNN.py assembles those features into a graph (edges built via k-NN through get_edge_by_knn/build_edge_index in pymupdf_util_edge.py) that’s classified by an ONNX-exported graph neural network (onnxruntime.InferenceSession, via common_util.make_session). Table structure is handled by a separate family of extractors (TableGridExtractor, TableGridExtractorV1A/V1B/V2/V2A/V2B/V3) selected by version string, and results are rendered by pluggable generator classes (MarkdownGenerator, MarkdownHTMLTableGenerator, HTMLGenerator). MultiProcessWrapper sits on top as the public entry point returned by DocumentLayoutAnalyzer.get_model(), spawning a worker pool (multiprocessing.pool.Pool) that lazily constructs one BoxRFDGNN per process via _worker_init(). pymupdf.layout.activate() is the integration seam: it monkey-patches pymupdf._get_layout so PyMuPDF4LLM’s document-to-Markdown path picks up the model without any caller-side wiring. Swapping the core GNN would mean touching BoxRFDGNN.py and the feature-vector code in common_util.py/pymupdf_util_*.py together, since feature extraction and model input shape are tightly coupled.

Tech Stack The package is a hybrid C/Python distribution built with pipcl (Artifex’s own PEP 517 backend) rather than setuptools directly: setup.py invokes SWIG to compile two extensions — features (structural feature extraction) and, on MuPDF >=1.28, tgif (a separate C table-grid runtime with grid/image/model/postprocess/runtime source files under source/tgif/) — against the MuPDF C++ headers embedded in an already-installed pymupdf wheel. Runtime dependencies declared through pipcl.Package(requires_dist=...) are a pinned PyMuPDF version, pyyaml, numpy, onnxruntime, and networkx; the ONNX models ship as resources under source/layout/onnx/ and source/layout/resources/onnx/, configured via global_config.yaml. CI (.github/workflows/test_push.yml) runs a private Artifex test harness (aptest) across Ubuntu, Windows, and both Intel/ARM macOS runners on Python 3.12, requiring an SSH-keyed private repo rather than a self-contained pytest suite.

Code Quality Test coverage is thin and partly gated behind private infrastructure: tests/test_general.py and tests/test_tgif.py contain a modest set of pytest functions, but several (test_competitor_examples, the _test_activate family, test_92) either return immediately, shell out to pip install/markdown_py at runtime, or depend on a private CompetitorExamples/aptest checkout that isn’t part of the public repo, so only a limited fraction of logic is actually exercised by a clean pytest run on the public source. Core library code (BoxRFDGNN.py, common_util.py, pymupdf_util*.py) has no type hints and relies on plain Exception raises rather than typed/custom error classes. Naming is consistent and descriptive (get_edge_by_knn, compute_edge_gap_bboxes), and MultiProcessWrapper.py uses from __future__ import annotations with type hints, but that discipline isn’t applied elsewhere. There is a GitHub Actions workflow, but it runs a proprietary test/build harness (aptest) requiring a secret SSH key rather than executing the visible pytest suite directly, so external contributors can’t easily verify results themselves.

API Design Developer-facing surface area is intentionally small: get_model() in DocumentLayoutAnalyzer.py takes sensible defaults for every parameter (model_name='BoxRFDGNN', feature_set_name='imf+rf', n_workers=1) and returns a ready-to-use MultiProcessWrapper, so a first call needs no configuration. The single documented integration path, pymupdf.layout.activate(), is a one-line, idempotent (if callable(pymupdf._get_layout): return) hook into PyMuPDF4LLM, genuinely low-friction for the common case of making PyMuPDF4LLM layout-aware. The tradeoff is that anything beyond that default path is under-documented: the README describes what the package does but not how to call DocumentLayoutAnalyzer or MultiProcessWrapper directly, table-grid-version selection (table_grid_model_ver='V1' | 'V1A' | ... | 'V4') isn’t explained anywhere in prose, and full API docs live off-repo on pymupdf.readthedocs.io rather than in docstrings — MultiProcessWrapper’s methods have decent inline docstrings but the ONNX/model-internal classes mostly don’t.