url-parse

url-parse is a small, focused Rust crate for decomposing a URL string into its individual components: scheme, username/password, subdomain, domain, top-level domain, port, path segments, query string, and anchor. Rather than wrapping a general-purpose URI standard, it takes a pragmatic, regex-driven approach that recognizes schemes the standard url crate does not handle out of the box, such as sftp, ssh, scp, and s3, and infers the correct default port for each.

The API centers on a Parser struct. Calling Parser::new(None) builds a parser preloaded with default port mappings for common protocols (http, https, ftp, ssh, scp, sftp, s3); passing a HashMap instead lets callers register arbitrary custom schemes with their own port and description. parser.parse(url) returns a public Url struct whose fields — scheme, user_pass, subdomain, domain, top_level_domain, port, path, query, anchor — are all directly accessible, with helper methods like host_str(), username(), password(), path_segments(), and serialize() for round-tripping back to a string.

It is aimed at tooling and infrastructure code — deployment scripts, file-transfer clients, or CLI utilities — that needs to parse addresses using protocols the mainstream URL crates don’t prioritize, or that wants to define its own scheme-to-port table without forking a heavier dependency.

What You Get

• Parser struct - a single entry point (Parser::new()) that parses any URL into a structured Url value via .parse(url).
• Built-in scheme/port table - default mappings for http, https, ftp, ssh, scp, sftp, and s3, so callers don’t have to hardcode default ports themselves.
• Custom scheme support - pass a HashMap<&str, (u32, &str)> to Parser::new() to register application-specific schemes with their own default port and description.
• Structured, field-accessible output - the returned Url struct exposes scheme, user_pass, subdomain, domain, top_level_domain, port, path, query, and anchor directly, plus helper accessors like host_str(), username(), password(), and path_segments().
• Serialization back to string - Url::serialize() reconstructs a URL string from a parsed or manually built Url struct, useful for normalizing or round-tripping addresses.

Common Use Cases

• Parsing file-transfer URLs - a deployment or backup tool accepts sftp://, scp://, or s3:// addresses from users and needs scheme, host, port, and path broken out without pulling in a full URI RFC implementation.
• CLI tools with custom protocol schemes - an internal tool defines its own myschema:// addressing convention and needs a parser that accepts a custom port-mapping table instead of only recognizing web schemes.
• Normalizing and re-serializing URLs - code manipulates individual URL fields (e.g. swapping the port or path) and calls serialize() to produce a valid URL string again.
• Extracting host/credentials from connection strings - scripts that connect to remote services via URL-style connection strings use host_str(), username(), and password() instead of hand-rolled string splitting.

Under The Hood

Architecture The crate is organized as a thin façade over per-component regex extractors: src/core/mod.rs defines the public Parser struct whose parse() method sequentially delegates to focused submodules — scheme.rs, login.rs, domain.rs, port.rs, path.rs, query.rs, anchor.rs — each responsible for extracting exactly one field from the raw URL string via regex, and assembles the results into the public Url struct defined in src/url.rs. defaults.rs supplies the built-in scheme-to-port HashMap, and scheme_separator.rs models the :// vs : distinction as a small enum. There is no shared mutable state or trait-based extension point; adding a new extractable field means adding a new submodule and wiring it into Parser::parse(), which keeps the design simple but means the parsing pipeline itself isn’t pluggable from outside the crate.

Tech Stack The crate targets Rust 2021 edition and has a single runtime dependency, regex (pinned to 1.10.2, built with default-features = false and the std feature only, keeping the dependency tree minimal). There is no async runtime, no I/O, and no platform-specific code — it’s a pure string-processing library. CI (via GitHub Actions, seen in .github/workflows/ci.yaml, cd.yaml, audit.yaml) runs tests, a security audit, and publishes releases to crates.io, with Codecov wired in for coverage reporting.

Code Quality Testing is extensive and colocated with implementation: nearly every submodule (url.rs, core/mod.rs, core/scheme_separator.rs, etc.) carries its own #[cfg(test)] mod tests block exercising typical inputs, edge cases (missing password, missing port, IP-address hosts), and custom-scheme scenarios, and the top-level doc comments in lib.rs and url.rs double as executable doctests. Error handling is explicit and typed via a small ParseError in src/error.rs rather than panics, though several accessor methods (host_str(), serialize()) still call .unwrap() internally on Option fields, which will panic if those invariants are violated by hand-constructed Url values. Naming is consistent and the module boundaries map cleanly to responsibilities.

What Makes It Unique The crate’s differentiator is narrow and practical rather than architectural: it ships default port knowledge for sftp, ssh, scp, and s3 — schemes that mainstream URL-parsing crates typically don’t special-case — and lets callers extend that table with arbitrary custom schemes at construction time. It does not aim to be a spec-complete URI/URL implementation; it trades strict RFC 3986 compliance for a simpler, more permissive regex-based parser tuned for the file-transfer and infrastructure-tooling use cases its README calls out.