tinyhtml5

tinyhtml5 is a focused HTML5 parser for Python that implements the WHATWG HTML parsing algorithm, including its well-defined error-recovery rules for the kind of malformed markup found across the real web. Given an HTML string, file path, pathlib.Path, or file descriptor, it returns a standard xml.etree.ElementTree.Element tree, so callers can walk, query, and transform the result with tooling they already know instead of learning a bespoke DOM API.

The project is a deliberately slimmed-down fork of the long-unmaintained html5lib, created by the CourtBouillon team to satisfy WeasyPrint’s parsing needs. Where html5lib exposed multiple output formats, tree walkers, adapters, and filters for a wide range of HTML-manipulation use cases, tinyhtml5 narrows the surface to a single parse() function and a single ElementTree output, trading flexibility for a smaller, more maintainable codebase that’s easier to reason about and keep current with modern Python.

What You Get

• Single entry point - tinyhtml5.parse() accepts an HTML string, file path, pathlib.Path, or open file descriptor and returns an ElementTree.Element.
• WHATWG-compliant error recovery - malformed HTML (unclosed tags, misnested elements, stray text) is handled with the same recovery rules real browsers use.
• Encoding detection - HTMLBinaryInputStream sniffs charset from BOM, <meta> tags, and transport/override hints before decoding byte input.
• Standard-library output - results are plain xml.etree.ElementTree nodes, so no new tree API to learn and no extra runtime dependency for traversal.
• Official test suite - ships the html5lib-tests tokenizer, tree-construction, and encoding fixtures (tests/tree-construction/*.dat) to pin conformance.

Common Use Cases

• Parsing arbitrary web HTML for downstream rendering - feeding a document tree into a layout/rendering pipeline (its original use case inside WeasyPrint) without writing custom recovery logic.
• Building lightweight scrapers/extractors - converting fetched HTML into ElementTree so .iter()/XPath-like traversal in ElementTree can pull out structured data.
• Normalizing user-submitted or third-party markup - re-serializing a tolerant parse of untrusted or inconsistent HTML into a well-formed tree before further processing.
• Replacing an unmaintained html5lib dependency - projects that only need parse() and ElementTree output can drop in tinyhtml5 for a smaller, actively maintained alternative.

Under The Hood

Architecture tinyhtml5 follows the WHATWG pipeline directly: inputstream.py (HTMLInputStream, HTMLUnicodeInputStream, HTMLBinaryInputStream) normalizes str/bytes/file input and resolves encoding through EncodingParser; tokenizer.py’s HTMLTokenizer turns that character stream into typed tokens (start/end tags, text, comments, doctype) via the tokenizer state machine; parser.py’s HTMLParser then dispatches tokens through a Phase subclass per WHATWG insertion mode (InitialPhase, InBodyPhase, InTablePhase, InSelectPhase, AfterBodyPhase, and around twenty others), each phase mutating parser state and delegating node creation to TreeBuilder in treebuilder.py, which maintains the open-elements stack, the active-formatting-elements list (ActiveFormattingElements), and produces xml.etree.ElementTree nodes as output; a ReparseError triggers a full reset() and second main_loop() pass for the rare cases the spec requires re-tokenizing under different assumptions. The one-phase-class-per-insertion-mode structure is a direct, readable mapping of the parser onto the WHATWG state diagram, so a change to how one insertion mode behaves is isolated to its own Phase subclass rather than rippling through a monolithic dispatcher.

Tech Stack Pure Python 3.10+ (tested through 3.14 and PyPy 3.11 in CI), packaged with flit_core as the build backend and a single runtime dependency, webencodings, for charset name normalization. Output relies entirely on the standard library’s xml.etree.ElementTree rather than a custom or third-party tree type. Documentation is built with Sphinx and the ReadTheDocs theme (doc extra); tests run under pytest with ruff for linting (test extra). Releases are automated via GitHub Actions: a tag push builds with flit, publishes to PyPI using trusted OIDC publishing, and drafts a GitHub release from the Sphinx changelog.

Code Quality The test suite is extensive and largely inherited from the authoritative html5lib-tests conformance corpus: tests/tree-construction/*.dat (dozens of spec test files covering adoption-agency edge cases, tables, foreign content, doctypes, and more), plus dedicated test_tokenizer.py, test_encoding.py, and tokenizer fixture directories, all driven through small TokenizerTestParser/tree-construction harnesses. There are no static type annotations on the public API (typing is documented via Sphinx-style docstrings instead), but internal invariants are enforced with assert statements at phase-transition boundaries, and parse errors are collected explicitly into self.errors rather than silently swallowed. CI runs the full suite across three operating systems and multiple Python/PyPy versions on every push and PR, with ruff check enforced as a required style gate.

What Makes It Unique tinyhtml5’s distinguishing choice is subtractive rather than additive: instead of building new parsing features, it deliberately strips html5lib down to exactly one supported output format (ElementTree) and one public function, removing tree walkers, adapters, filters, and alternate tree builders entirely. That narrowing is what makes an otherwise large, spec-mandated state machine (the WHATWG HTML parsing algorithm is inherently big) maintainable by a small team, while still tracking the same conformance test suite the wider HTML-parsing ecosystem relies on for correctness.