@nodelib/fs.walk

@nodelib/fs.walk is a Node.js library for efficiently walking a directory tree and collecting information about every file and directory it contains. Built on top of the sibling @nodelib/fs.scandir package, it exposes three complementary APIs — callback-based walk(), synchronous walkSync(), and stream-based walkStream() — so callers can choose the concurrency and backpressure model that fits their use case, from quick scripts to high-throughput build tooling.

Beyond basic traversal, it lets consumers customize behavior with entry, deep (subtree), and error filter functions, control symbolic-link following, cap directory-read concurrency via a fastq-backed queue, and cancel an in-flight asynchronous walk with a native AbortSignal. Its predictable, well-tested traversal semantics make it a foundational dependency for tools like fast-glob and other file-system utilities.

What You Get

• Three traversal APIs - walk (callback), walkSync (synchronous array), and walkStream (Node.js Readable stream) covering async callback-style, blocking, and streaming consumption patterns.
• Composable filtering - entryFilter, deepFilter, and errorFilter functions let you exclude files/directories from results, skip whole subtrees, or selectively ignore filesystem errors (e.g. ENOENT) without wrapping the API yourself.
• Symlink and abort controls - followSymbolicLinks/throwErrorOnBrokenSymbolicLink options plus native AbortSignal support for cancelling long-running async walks.
• Pluggable FS adapter - the fs option lets you swap in custom lstat/stat/readdir implementations, useful for testing or virtual filesystems.

Common Use Cases

• Build tool file discovery - bundlers and linters recursively enumerate source files while excluding node_modules via a deepFilter.
• Glob engine backend - libraries like fast-glob use fs.walk as their underlying recursive directory reader.
• Static analysis / codemods - CLI tools that need to visit every file under a project root, with stats enabled to inspect file metadata.
• Safe scanning of untrusted trees - using errorFilter and throwErrorOnBrokenSymbolicLink: false to walk directories that may contain broken symlinks or permission errors without crashing.

Under The Hood

Architecture The package is layered into clearly separated pieces: settings.ts normalizes raw Options into an immutable Settings instance (including a nested fsScandirSettings for the sibling @nodelib/fs.scandir package), walk.ts is the public entry point that wires a FileSystemAdapter plus Settings into a reader and a provider, providers/{async,sync,stream}.ts are thin orchestration classes that wire reader callbacks to a consumer-facing shape (a Node callback, a returned array, or a Readable stream), and readers/{async,sync}.ts hold the actual recursive-traversal logic, with readers/common.ts centralizing shared filter and path-prefix helpers. The async reader manages concurrency with a fastq queue and a hand-rolled callback-registration class (AsyncReaderCallbacks) rather than EventEmitter, keeping the public surface fully typed. This separation means the traversal algorithm (in the readers) is decoupled from how results are delivered (in the providers), and the whole package’s typing is anchored to @nodelib/fs.scandir’s Entry/Settings shape via adapters/fs.ts, so a breaking change there ripples directly into this package.

Tech Stack Written in strict TypeScript targeting Node.js ^22.13.0 || >=24, published as ESM ("type": "module"). Runtime dependencies are minimal: the sibling @nodelib/fs.scandir package for the actual directory-read syscalls and fastq for the async concurrency queue. The package builds via TypeScript project references (tsc -b), is linted with ESLint using the shared eslint-config-mrmlnc config, and is tested with Node’s built-in node --test runner against compiled out/**/*.spec.js rather than a third-party test framework. The whole nodelib monorepo is orchestrated with Lerna, and performance is tracked separately via hereby/bencho benchmark suites.

Code Quality Test coverage is extensive relative to the implementation — over 1,300 lines of *.spec.ts across the package, including a 404-line suite for the async reader alone covering queue behavior, error propagation, and abort handling, using sinon to stub filesystem calls. Error handling is explicit and typed throughout: ErrnoException values flow through dedicated onError callbacks and an isFatalError/errorFilter mechanism gives callers fine-grained control instead of errors being silently swallowed. Classes consistently use ECMAScript private fields (#field) for encapsulation, naming is consistent (PascalCase classes, camelCase methods), and the codebase has no loose any typing in its core files. CI runs a standard build/test workflow on every push plus a dedicated CodeQL security-scanning workflow.

API Design The package offers three complementary entry points — callback, sync, and stream — under one cohesive API, letting consumers pick the concurrency/backpressure model appropriate to their use case without switching libraries, which is unusual compared to most single-mode directory-walking libraries. Options are validated once into an immutable Settings object that can be constructed once and reused across many calls to skip repeated validation, and every API happily accepts either a plain options object or a pre-built Settings instance. Filter functions are plain predicates rather than a bespoke query DSL, keeping the surface approachable, and the async/stream paths support cancellation via a standard AbortSignal rather than a custom token type. The README documents every option with runnable examples.