soupsieve

Soup Sieve is the CSS selector engine that powers Beautiful Soup’s select() and select_one() methods, implementing selectors from CSS Level 1 through the latest CSS Level 4 drafts. It lets you query, match, and filter parsed HTML or XML trees using familiar CSS selector syntax instead of hand-rolled tree traversal, supporting standard combinators, attribute selectors, and structural pseudo-classes like :has(), :is(), :not(), and :matches(), plus Soup Sieve-specific extensions such as :-soup-contains() for text matching.

Because it operates directly on Beautiful Soup’s tag objects, it works with any of Beautiful Soup’s underlying parsers (html.parser, lxml, html5lib) without change, and can also be imported directly as a standalone API for more specialized selection and filtering logic outside of Beautiful Soup’s built-in shortcuts.

What You Get

• CSS Level 1-4 selector support - class, id, attribute, combinator, and most structural/pseudo-class selectors from the CSS specs, including :has(), :is(), :where(), and :not() with multiple arguments.
• Drop-in integration with Beautiful Soup - since bs4 4.7.0, soup.select() and soup.select_one() are powered directly by Soup Sieve, so no extra wiring is required.
• Standalone API - compile(), match(), filter(), select(), select_one(), closest(), and iselect() for use independent of Beautiful Soup’s built-in shortcuts.
• Custom text-matching pseudo-classes - :-soup-contains() and :-soup-contains-own() let you filter elements by their text content, a feature not present in browser CSS.
• Compiled pattern caching - repeated calls with the same selector string reuse a cached compiled pattern via an LRU cache, with purge() to clear it.

Common Use Cases

• Web scraping - selecting specific elements (e.g. div.article > p:not(.ad)) from scraped HTML pages using familiar CSS syntax instead of manual tree walks.
• HTML/XML data extraction pipelines - filtering parsed documents down to the exact nodes needed before further processing or storage.
• Test fixtures and HTML validation - asserting the presence or shape of expected markup in test suites via sv.match().
• Content migration/cleanup scripts - using :has() and :is() to find and strip elements containing (or matching) specific children across large HTML corpora.

Under The Hood

Architecture Soup Sieve follows a clean three-stage pipeline: css_parser.py tokenizes and parses a CSS selector string into an immutable tree of dataclasses defined in css_types.py (SelectorList, Selector, SelectorAttribute, SelectorNth, Namespaces, CustomSelectors, etc.), which css_match.py’s CSSMatch class then walks against a Beautiful Soup tag by extending a private _DocumentNav mixin that abstracts tree navigation (parent/sibling/child access, doc-root detection) away from the specific bs4 parser backend in use. The public surface in init.py is a thin functional facade (compile, match, filter, select, select_one, iselect, closest) that all funnel through compile(), which delegates to a module-level LRU-cached _cached_css_compile() in css_parser.py so repeated calls with an identical pattern string reuse the same compiled SelectorList and avoid re-parsing. This separation keeps parsing, the selector data model, and matching logic in three independently testable modules; a change to the CSS grammar handled in css_parser.py can ripple into new dataclass fields in css_types.py and new matching branches in css_match.py, but the functional API in init.py stays stable regardless.

Tech Stack Soup Sieve is pure Python (requires-python >=3.10, tested via a Hatch matrix through 3.14) with hatchling as its build backend and no declared runtime dependencies in pyproject.toml’s project table — it imports bs4 directly in init.py and css_match.py to type against and inspect Tag objects, relying on Beautiful Soup itself being present in the caller’s environment, since the two projects are designed to be installed together. Development tooling is managed through Hatch environments: pytest/pytest-cov for tests, mypy in strict mode for type checking, and ruff for linting with an extensive rule selection (flake8-bugbear, pep8-naming, Perflint, and more); documentation is built with zensical/markdown/pymdown-extensions and spell-checked via pyspelling. CI runs through GitHub Actions workflows for testing, docs deployment, and PyPI publishing.

Code Quality Tests live under tests/ organized by CSS spec level (test_level1 through test_level4, test_nesting_1, test_extra) plus test_api.py, test_bs4_cases.py, test_performance.py, test_quirks.py, and test_versions.py, run via pytest with coverage collection configured in pyproject.toml; this level-by-level test organization mirrors the CSS specification directly, suggesting each new selector feature ships with dedicated conformance tests. mypy runs in strict mode against the package, and the codebase is fully type-annotated using postponed evaluation. Errors are surfaced as an explicit SelectorSyntaxError rather than swallowed, and ruff lint covers a broad rule set with an explicit, documented ignore list rather than blanket suppression. This is a well-tested, strictly-typed, actively linted codebase with no obvious gaps.

API Design Soup Sieve’s public API is deliberately small and mirrors familiar CSS/DOM conventions: match/select/select_one/filter/iselect/closest read like their CSS/JS counterparts, and compile() returns a reusable object so callers doing repeated matching against many tags can avoid re-parsing entirely. Its most distinctive design decision is extending real CSS syntax with library-specific pseudo-classes (:-soup-contains(), :-soup-contains-own()) using the vendor-prefix convention browsers use for experimental features, letting it add text-matching capabilities CSS itself doesn’t define without inventing a non-standard selector dialect. A custom keyword argument for registering named selector aliases, combined with namespace support for XML, rounds out an API that stays close to standard CSS while covering the gaps a server-side, non-browser selector engine actually needs.