All 28 Dependencies
Every package reader depends on, ranked by repo health score.
Reader is a TypeScript library and CLI that gives AI agents reliable, production-grade access to the web. Built on top of Ulixee Hero — a headless browser purpose-built for stealth — Reader handles TLS fingerprinting, Cloudflare challenge detection, navigator spoofing, and WebRTC masking so scrapers don't get blocked before they even reach the content.
The library exposes three composable primitives: `scrape()` converts one or more URLs into clean markdown or HTML with automatic main-content extraction, noise removal (nav, cookies banners, footers), and batch concurrency control; `crawl()` performs BFS link discovery from a starting URL with configurable depth and page limits, optionally scraping every discovered page; and `browser()` launches a stealthed Chrome instance accessible over the Chrome DevTools Protocol (CDP), letting Playwright or Puppeteer connect to a pre-configured stealth context with zero code changes.
Under the hood, Reader manages a tiered browser pool keyed by proxy URL, with automatic proxy health tracking (circuit breaker, cooldown, revival), per-domain configuration profiles, robots.txt compliance, rate limiting, and a long-running daemon mode that keeps the pool warm between CLI commands. The markdown conversion uses a custom Rust-based library (supermarkdown) via NAPI for high-performance, panic-safe HTML-to-markdown transformation.
The project ships with an npm package (`@vakra-dev/reader`), a companion cloud offering (`@vakra-dev/reader-js`) for those who prefer managed infrastructure, and a comprehensive CLI with `scrape`, `crawl`, `browser`, `start`, `stop`, and `status` subcommands. It targets Node.js 18+ and is licensed under Apache 2.0.
