webpage

webpage is a small Rust library for fetching and parsing information about a web page. Point it at a URL and it downloads the page, follows redirects, and returns structured details: HTTP transfer info, the parsed title, description, and language, discovered links and RSS feeds, plus OpenGraph and Schema.org metadata. You can also parse local files or in-memory HTML strings without any network access, making it useful for scraping, link previews, and metadata extraction.

What You Get

• One-call fetching that follows redirects and returns both HTTP transfer info and parsed HTML data
• Extraction of title, description, language, links, and RSS feed URLs from the document
• Structured OpenGraph and Schema.org (JSON-LD and microdata) metadata, with optional serde serialization

Common Use Cases

• Generating rich link previews from a pasted URL
• Extracting OpenGraph or Schema.org metadata for indexing or social sharing
• Parsing local HTML files or strings to pull out titles, descriptions, and feeds

Under The Hood

Architecture - The crate splits fetching from parsing: http.rs performs the request and captures transfer metadata (IP, headers, final URL, content type, body), while html.rs, parser.rs, opengraph.rs, and schema_org.rs build the parsed HTML structure. Entry points Webpage::from_url, HTML::from_file, and HTML::from_string let callers choose networked or offline operation. Tech Stack - Targeting the Rust 2021 edition (rust-version 1.63), it parses with html5ever and markup5ever_rcdom, handles URLs with url, and uses curl for fetching under the default feature; serde/serde_json provide optional serialization behind the serde feature. Code Quality - The code is organized into focused modules and backed by an integration test suite under tests/, with a maintained CHANGELOG tracking releases through 2.0.x. API Design - The public API is deliberately small — a handful of constructors returning well-typed structs — so a caller can go from a URL to structured metadata in a single expression, with feature flags to opt into serde and to swap the HTTP backend.