memchr

memchr provides heavily optimized routines for searching 1, 2, or 3 bytes within a byte slice, plus a memmem submodule for forward and reverse substring search, all operating on &[u8] without regard to text encoding. On x86_64, aarch64, and wasm32 targets it automatically selects SIMD-accelerated implementations at runtime, falling back to portable code elsewhere.

With over 1.1 billion total downloads, memchr is one of the most widely depended-upon low-level crates in the Rust ecosystem, forming the search backbone underneath tools like ripgrep and many parsing and text-processing libraries. It supports no_std environments and compiles down to minimal, allocation-free primitives.

What You Get

• Top-level memchr/memchr2/memchr3 functions for searching 1-3 bytes in forward or reverse direction
• A memmem submodule with forward and reverse substring search over &[u8]
• Automatic runtime CPU-feature detection to select AVX2/SSE2/NEON-accelerated code paths
• no_std support (with only the alloc feature) for use in constrained environments
• wasm32 SIMD-accelerated routines alongside the x86_64 and aarch64 implementations

Common Use Cases

• Powering fast line- and pattern-search in tools like ripgrep and other grep-style utilities
• Implementing custom parsers or lexers that need to quickly scan for delimiter bytes
• Substring search over large byte buffers in log-processing or data-pipeline tools
• Building no_std embedded tooling that still needs fast byte-search primitives

Under The Hood

Architecture The crate’s roughly 16k lines of source are organized around a top-level API for single/multi-byte search (memchr/memchr2/memchr3) and a memmem module implementing substring search, each backed by architecture-specific SIMD implementations (x86_64 SSE2/AVX2, aarch64 NEON, wasm32 SIMD) selected via runtime CPU-feature detection, with portable fallback code for unsupported targets. Tech Stack Pure Rust (edition 2021, MSRV 1.61), with a std feature (enabled by default) gating the runtime-detection machinery and an alloc-only path for no_std use; a separate benchmarks/ workspace and fuzz/ directory are excluded from the published crate via Cargo.toml’s exclude list to keep the published artifact minimal. Code Quality The project maintains a dedicated fuzz/ corpus for continuous fuzz testing of its search routines and a benchmarks/ suite for tracking the performance regressions that matter most for a crate whose entire value proposition is raw search speed; an explicit AI_POLICY.md in the repo signals a deliberate, actively-governed contribution process. API Design The API surface is intentionally narrow and encoding-agnostic — everything operates on &[u8] rather than str — so it composes cleanly whether callers are working with UTF-8 text or arbitrary binary data, and the std/no_std split is handled via a single Cargo feature rather than separate crates.