lxml_html_clean

lxml_html_clean is a standalone Python library that provides the Cleaner class for sanitizing HTML documents by stripping scripts, embedded objects, inline styles, and other potentially unsafe markup. It was split out of lxml.html.clean after repeated security reports made bundling a blocklist-based cleaner directly inside lxml problematic for security-sensitive users.

The library exposes a configurable Cleaner class along with convenience functions for autolinking plain-text URLs and wrapping long words in HTML content. It builds directly on lxml’s ElementTree API, giving callers fine-grained control over which tags, attributes, and CSS constructs get removed, killed, or allowed through, plus a host-whitelist mechanism for permitting specific embedded content.

What You Get

• Cleaner class - a configurable sanitizer that strips scripts, styles, forms, embedded objects, and other unsafe HTML content from a parsed document
• clean_html function - a one-call shortcut that parses a string or lxml document and returns the cleaned result
• autolink and autolink_html - turn plain-text URLs and email addresses into anchor tags while skipping code blocks and blacklisted hosts
• word_break and word_break_html - insert zero-width space breaks into long unbroken words for layout-safe rendering
• host_whitelist support - selectively allow embedded iframes or objects from trusted domains instead of stripping them outright

Common Use Cases

• Sanitizing user-submitted HTML - strip scripts, event handlers, and forms from blog comments or rich-text form submissions before storage or display
• Cleaning scraped or syndicated HTML - remove tracking scripts, ads, and layout cruft from RSS/Atom feed content before republishing
• Migrating off lxml’s bundled cleaner - projects still calling lxml.html.clean can switch to lxml_html_clean with minimal code changes since the API is unchanged
• Preparing HTML for constrained renderers - drop unsupported tags like iframe, object, and form before sending HTML to email clients or PDF generators

Under The Hood

Architecture The library is a single-module design centered on clean.py, where the Cleaner class does all the work in one __call__ method: it walks the parsed lxml tree once, building kill_tags/remove_tags/allow_tags sets from the instance’s boolean options, then makes a second pass over doc.iter() to bucket elements into _kill and _remove deques before applying drop_tree() (removes subtree) or drop_tag() (unwraps, keeps children) in the right order so the parent-most element is never left dangling. Supporting concerns - CSS Unicode-escape decoding, sneaky-JavaScript detection in style attributes, conditional-comment stripping, and URL-scheme sanitization - are implemented as private helper methods and module-level compiled regexes rather than separate classes, keeping the whole sanitization pipeline traceable from the one __call__ entry point. autolink() and word_break() are independent recursive tree-walkers that reuse the same fromstring/_transform_result round-trip helpers as clean_html().

Tech Stack The project is pure Python (with a # cython: language_level=3str header enabling optional Cython compilation of clean.py) with a single runtime dependency, lxml>=6.1.1, pinned tightly because recent releases fixed underlying rewrite_links() behavior the cleaner depends on for XSS-safe link handling. It uses setuptools with a classic setup.cfg + pyproject.toml split (no build backend customization), ships inline type stubs (clean.pyi, __init__.pyi, py.typed) for static-typing consumers, and is tested across Python 3.9-3.14 plus a dedicated mypy environment via tox and a GitHub Actions matrix.

Code Quality Tests live in tests/test_clean.py as unittest.TestCase classes covering allow/remove/kill-tag combinations, safe-attribute filtering, and CVE regression cases, supplemented by doctest-style .txt fixtures (test_clean.txt, test_autolink.txt, test_clean_embed.txt) exercised via python -m doctest. Error handling favors explicit TypeError/ValueError raises in Cleaner.__init__ for misconfigured options (e.g. passing a string where a collection is expected) over silent coercion. Naming is consistent with lxml’s own conventions (leading underscore for private module-level regexes and helpers), and the codebase is fully typed via companion .pyi stubs checked by mypy in CI rather than inline annotations in the .py source itself.

What Makes It Unique The project’s distinguishing trait isn’t a novel sanitization algorithm - it’s an explicit disclaimer that its blocklist-based approach is not appropriate for security-sensitive environments, paired with an unusually well-documented CVE remediation history (Unicode CSS-escape bypass, <base>-tag URL hijacking, xlink:href script injection) visible directly in CHANGES.rst. That transparency, combined with being a drop-in extraction that preserves the exact lxml.html.clean API surface, makes it valuable mainly as a maintained, security-patched continuation of code many existing codebases already depend on, rather than as a from-scratch redesign.