magic-string-rs

magic_string is a Rust reimplementation of Rich Harris’s popular MagicString library, the string-manipulation engine that underlies bundlers and transpilers like Rollup, Svelte, and esbuild plugins. It lets callers append, prepend, overwrite, move, and remove ranges of a source string while automatically tracking the edits needed to emit an accurate V3 sourcemap, without ever mutating the original text directly.

The crate is organized as a Cargo workspace: a pure-Rust core crate implements the chunk-linked-list editing model and sourcemap encoder, and a node crate wraps it with napi-rs bindings so the same engine ships as the @napi-rs/magic-string npm package. Benchmarks in the repository show it consistently outperforming the original JavaScript MagicString on core operations like overwrite and prepend/append, which is the main reason build-tool authors reach for it when a hot codegen path needs the MagicString API without paying JS string-copy costs.

What You Get

• A MagicString type with append/prepend/append_left/append_right/prepend_left/prepend_right for inserting content relative to any index without disturbing existing edits
• overwrite, remove, and _move for replacing, deleting, and relocating ranges of the original string while preserving edit history
• generate_map and generate_decoded_map for producing V3 sourcemaps (encoded string or raw decoded mapping arrays), with hires and include_content options
• trim, trim_start, trim_end, and trim_lines for regex-pattern-based whitespace/line trimming at the edges of the generated output
• Node.js bindings (@napi-rs/magic-string) built with napi-rs, exposing the identical API surface with prebuilt native binaries for macOS, Windows, and Linux (glibc, musl, and ARM targets)
• A typed Result/Error model (MagicStringErrorType) that distinguishes out-of-range edits, cross-chunk overlaps, and double-split/double-edit errors instead of panicking

Common Use Cases

• Bundler/transpiler codegen - rewriting import specifiers, injecting runtime helpers, or transforming syntax while emitting a sourcemap back to the original file
• Rust-native build tooling - projects like SWC-adjacent or Rust-based JS tooling that need MagicString’s editing semantics without shelling out to Node
• Node performance-sensitive plugins - swapping the pure-JS magic-string package for the native binding in hot codegen loops (per the repo’s own benchmark suite)
• Source-to-source transforms - macro expansion, template compilation, or lint autofixers that need precise, composable string edits with sourcemap fidelity

Under The Hood

Architecture The engine models the source string as a doubly-linked list of Chunk structs (core/src/chunk.rs), each tracking a start/end byte range plus its own intro/outro/content fields; MagicString (core/src/magic_string.rs) holds chunk_by_start/chunk_by_end HashMaps keyed by index for O(1) lookup and a last_searched_chunk cache to avoid re-walking the list on repeated edits at nearby indices. Every edit operation (overwrite, remove, _move, the append_*/prepend_* family) first calls _split_at_index to break chunks at the edit boundary, then mutates only the affected chunk(s) via Rc<RefCell<Chunk>> shared ownership — this is the same chunk-splitting design as the original JS MagicString, ported to Rust’s ownership model instead of JS’s implicit garbage collection. Sourcemap generation (generate_decoded_map/generate_map) walks the chunk list once via Chunk::try_each_next, feeding a Locator (line/column lookup over the original string) into a Mapping builder that produces VLQ-encoded V3 mappings; changing the core chunk-linking invariant would require touching nearly every public method, since all of them assume a fully-connected, non-overlapping chunk chain.

Tech Stack The workspace is plain Cargo ([workspace] members = ["core", "node"]) targeting Rust edition 2021. core depends on serde/serde_json for sourcemap serialization, vlq for mapping encoding, regex for pattern-based trimming, and base64 for inline sourcemap URLs; the optional node-api feature pulls in napi/napi-derive (2.0.0-beta.5) to compile the same crate as a native Node addon. The node/ package wraps that addon with @napi-rs/cli, TypeScript type definitions (index.d.ts), and a binding.js loader that resolves the correct prebuilt .node binary per platform; CI (CI.yaml) cross-compiles for macOS (x64/arm64), Windows (x64/x86), and Linux (glibc/musl, x64/arm64/armv7) via napi-rs’s standard prebuild matrix.

Code Quality The core crate has a dedicated tests/ directory with one file per operation family (overwrite.rs, remove.rs, move.rs, pend.rs, trim.rs, generate_map.rs, is_empty.rs), and public methods carry executable doctests demonstrating expected input/output (visible throughout magic_string.rs). Error handling is explicit and typed via the Result<T, Error> alias rather than panics or unwrap in the public API, with From impls converting io::Error, vlq::Error, regex::Error, and serde_json::Error into a unified MagicStringErrorType. #![deny(clippy::all)] is set at the crate root, enforcing lint-clean code; the Node package additionally runs a mocha/TypeScript test suite (node/tests) against the compiled binding. No formal architecture-decision docs exist beyond inline doc comments, but the comment density is high and consistent across both crates.

What Makes It Unique The project’s differentiator is being a faithful, benchmark-verified Rust port of an established JS API rather than a novel design — its own README benchmark suite (run on Apple M1 hardware) shows it beating the original MagicString on overwrite, prepend/append, and most sourcemap-generation paths, while being roughly on par or slower for a couple of hi-res-sourcemap-heavy cases. Exposing that identical Rust engine to Node through napi-rs, so JS tooling authors can drop in a native replacement without an API rewrite, is the specific technical choice worth noting; the editing model and sourcemap format themselves follow the upstream MagicString/source-map V3 conventions closely rather than introducing new abstractions.