walkdir

walkdir is a Rust crate that provides an ergonomic, iterator-based API for recursively traversing a directory tree. It handles the platform-specific details of directory traversal on Unix and Windows, supports optional symbolic-link following, lets callers cap the number of simultaneously open file descriptors, and offers a filter_entry adapter for pruning whole subtrees (such as hidden directories) without paying the cost of descending into them.

Written and maintained by BurntSushi (author of ripgrep), walkdir is one of the most widely depended-upon crates in the Rust ecosystem for filesystem traversal, used as a building block by search tools, linters, build systems, and any application that needs to enumerate files under a root path.

What You Get

• A WalkDir builder with a fluent API for configuring max depth, symlink following, sorting, and traversal order
• A lazy Iterator of DirEntry results, so large trees can be processed incrementally without loading the whole tree into memory
• A filter_entry() adapter that can skip descending into whole subtrees (e.g. hidden directories) without extra syscalls
• Cross-platform behavior normalized between Unix and Windows, including consistent handling of file-descriptor limits
• Well-defined error types so IO errors during traversal (permission denied, broken symlinks) surface as recoverable Results per entry

Common Use Cases

• Building command-line search or grep-like tools that need to enumerate files under a directory efficiently
• Implementing linters, formatters, or build tools that must discover all source files in a project tree
• Writing backup, sync, or file-indexing utilities that walk large directory structures and need to skip specific subtrees
• Any Rust application needing predictable, cross-platform recursive directory traversal instead of hand-rolled platform-specific code

Under The Hood

Architecture: The core of walkdir is src/lib.rs (roughly 1,200 lines), which implements WalkDir as a builder that produces an IntoIter/Iter pair driving a depth-first traversal using an explicit stack of open directory handles rather than recursion, which is what lets it bound the number of simultaneously open file descriptors (max_open) and avoid stack-overflow risk on very deep trees. src/dent.rs defines DirEntry, wrapping platform metadata (inode/file-type information) behind a uniform interface, and src/error.rs defines a dedicated Error type that preserves the path and IO cause so a single bad entry doesn’t abort the whole walk — it’s yielded as an Err in the iterator instead. Tech Stack: Pure Rust (2018 edition) with minimal dependencies — same-file for reliable file-identity comparisines across symlinks, and winapi-util gated behind cfg(windows) for Windows-specific file-handle APIs; the crate ships as a Cargo workspace including a small walkdir-list example binary. Code Quality: Tests live in src/tests/ alongside the implementation (declared via mod tests in lib.rs) and the crate has been stable at its core API for years with no unsafe code additions needed for the traversal logic itself; the README documents a manual performance-comparison methodology against find and glibc’s nftw, reflecting the author’s focus on measured, not assumed, performance parity. API Design: The builder-plus-iterator pattern (WalkDir::new("foo").follow_links(true).into_iter()) is idiomatic Rust and composes naturally with standard iterator adapters like filter_map and the crate’s own filter_entry, requiring almost no boilerplate to get a working recursive walk — the primary API surface is small enough to be fully learnable from the README’s four short examples.