yarl

yarl provides a single URL class that parses, builds, and compares URLs while handling percent-encoding correctly and transparently. Every modification produces a new immutable URL object rather than mutating state in place, which makes it safe to pass URLs through concurrent and async code paths without defensive copying.

Originally built to back aiohttp’s client and server, yarl is now a standalone dependency used across the asyncio ecosystem wherever precise, spec-correct URL handling matters more than ad-hoc string splitting. A compiled Cython extension backs the hot paths, with a pure-Python fallback for PyPy and environments that can’t build C extensions.

What You Get

• An immutable URL class where every mutation (/, %, .with_*()) returns a new object instead of changing the original
• Decoded and raw (raw_*) accessors for every URL component: scheme, user, password, host, port, path, query, fragment
• A MultiDictProxy-based query API so repeated query keys are handled correctly instead of silently collapsing
• Automatic, spec-correct percent-encoding and decoding of non-ASCII and reserved characters
• A compiled C/Cython extension for hot-path performance, with a pure-Python fallback (YARL_NO_EXTENSIONS) for PyPy or restricted environments
• A human_repr() method that renders a URL back to its human-readable, non-encoded form

Common Use Cases

• Building and normalizing outgoing request URLs inside an aiohttp client or any asyncio-based HTTP library
• Safely joining base URLs with relative paths (url / 'foo' / 'bar') without manual string concatenation
• Adding or replacing query parameters (url % {'bar': 'baz'}) while preserving correct encoding
• Comparing, hashing, or using URLs as dict keys in caches and routers, relying on immutability for correctness
• Parsing incoming request URLs on the server side to inspect path, query, and fragment components

Under The Hood

Architecture: The library centers on a single URL class in yarl/_url.py (~1,700 lines) that stores a parsed urllib.parse.SplitResult internally and derives every public property (scheme, host, port, path, query, fragment) from it lazily via propcache’s under_cached_property, so repeated attribute access after the first call is essentially free. Parsing and normalization logic is split into small focused modules — _parse.py for split/unsplit helpers, _path.py for path segment normalization, _query.py for query-string encode/decode, and _quoters.py/_quoting.py/_quoting_c.pyx for percent-encoding — keeping the core URL class readable while pushing performance-sensitive quoting into a compiled Cython module with a pure-Python (_quoting_py.py) fallback selected automatically at import time. Tech Stack: Pure Python 3 with an optional compiled Cython extension (_quoting_c.pyx) built via a custom in-tree PEP 517 backend (packaging/pep517_backend), and runtime dependencies on multidict (query storage as MultiDictProxy), propcache (cached properties), and idna (internationalized domain name encoding); the build can be forced to pure-Python via a pure-python config setting or the YARL_NO_EXTENSIONS env var for PyPy or systems without a C toolchain. Code Quality: The test suite is extensive and granular — 318+ test functions in test_url.py alone, plus dedicated files for caching, pickling, pydantic integration, path normalization, and dedicated benchmark suites (test_url_benchmarks.py, test_quoting_benchmarks.py) that guard performance regressions alongside correctness; the project ships py.typed for full type-checker support and enforces ruff linting across the main tree. API Design: The public surface reads close to natural URL syntax — url / 'foo' / 'bar' for path joins and url % {'bar': 'baz'} for query updates — while every component is exposed twice, once decoded (url.path) and once raw/encoded (url.raw_path), giving callers an explicit, unambiguous choice instead of a single lossy representation.