node-klaw

klaw recursively walks a directory tree and emits every file, directory, and symlink it finds as items on a Node.js Readable stream. Instead of collecting the whole tree into an array before you can use it, klaw lets you consume entries as they’re discovered — via data events, the pull-based readable/read() pattern, or for await...of — so large trees can be processed incrementally and piped through transform streams.

The library began life inside fs-extra and was extracted into its own package once it became clear the streaming-walk behavior was useful well beyond that project. It stays deliberately small: a single Walker class extending stream.Readable, with configurable traversal order (queueMethod, pathSorter), depth limiting, symlink handling, and a pluggable fs implementation for testing or instrumentation with tools like mock-fs.

Because each walked item is just { path, stats }, klaw composes naturally with the Node.js stream ecosystem — through2 transforms are the documented pattern for filtering directories, aggregating file stats, or triggering side effects like deletion, all without buffering the full directory listing in memory.

What You Get

• A walk(directory, [options]) function returning a Node.js Readable stream in object mode
• Support for all three stream consumption styles: data events, pull-based readable/read(), and for await...of
• Configurable traversal via queueMethod (shift/pop), pathSorter, and depthLimit
• A filter option to exclude paths (e.g. hidden directories) before they’re stat’d or recursed into
• preserveSymlinks option to control whether symlinks are followed or returned as-is
• Pluggable fs implementation so the walk can run against graceful-fs, mock-fs, or any compatible module
• Per-item error events carrying both the error and the offending item, rather than a single fatal exception

Common Use Cases

• Incrementally processing every file under a directory without loading the full listing into memory first
• Piping walked entries through through2 transforms to filter out directories, aggregate file sizes by extension, or delete matching files
• Building custom file-system tooling (linters, bundlers, static site generators) that needs a streaming directory traversal primitive
• Testing file-system-dependent code by swapping in mock-fs via the fs option instead of hitting the real disk
• Recursively finding files matching a predicate via the filter option before deeper directories are read

Under The Hood

Architecture The entire implementation lives in one file, src/index.js, as a single Walker class extending Node’s stream.Readable in object mode. Traversal state is a plain array (this.paths) seeded with the root directory; each call to the stream’s _read() hook pops or shifts one path off that array (per queueMethod), stats or lstats it depending on preserveSymlinks, and — if the result is a directory within any configured depthLimit — calls readdir to enqueue its children (after applying filter and pathSorter) before pushing the current item downstream. Because traversal is driven entirely by the stream’s own backpressure-aware _read() calls rather than a separate recursive function, the walk naturally pauses when consumers stop reading, and there’s no separate queue-draining loop to maintain. The design intentionally has almost no moving parts: no dependency injection container, no plugin system, just one class whose behavior is fully parameterized through its options object.

Tech Stack klaw is pure Node.js core: stream.Readable, fs (or a caller-supplied drop-in replacement, e.g. graceful-fs or mock-fs), path, assert, and url’s fileURLToPath for accepting file:// URLs alongside plain path strings. There is no build step and no runtime dependencies at all — package.json declares zero dependencies, only devDependencies for linting (standard) and testing (tape, tap-spec). The published package ships only the src/ directory. CI (.github/workflows/ci.yml) runs the test suite across Node 14 through the latest release on both Ubuntu and Windows, confirming the pure-core-API approach stays portable.

Code Quality Tests live under tests/ and use tape with a small custom harness (tests/_test.js) that creates and tears down a temp directory per test case, covering path sorting, symlink handling, depth limiting, filtering, and read-directory error propagation. Linting is enforced via standard (zero-config StandardJS style) and wired into npm test alongside the unit run, so style and tests fail the same command. Error handling favors explicit error events carrying both the underlying error and the specific item that failed, rather than swallowing errors or crashing the process — a deliberate choice documented in the README’s “Error Handling” section. The codebase has no TypeScript of its own, but the README points to a community-maintained @types/klaw package for consumers who want types.

What Makes It Unique klaw doesn’t try to be a general file-utility toolkit; it isolates exactly one concern — turning a directory tree into a stream of stat’d entries — and leaves filtering, aggregation, and side effects to be composed externally via standard stream transforms like through2. That narrowness, combined with zero runtime dependencies and a single-file implementation, is what let it split cleanly out of fs-extra: any project that already understands Node streams gets a walker that fits directly into existing pipelines rather than introducing its own callback or promise-based API and its own conventions.