msgpackr-extract

msgpackr-extract is a small native addon whose sole job is to pull strings out of MessagePack-encoded buffers as fast as possible. Creating individual JS strings during decoding is one of the biggest performance bottlenecks in binary parsers, so this module does partial MessagePack parsing at the C++ level, locates string tokens directly in the buffer, and hands back up to 256 strings per call. It exists specifically to back msgpackr, the MessagePack encoder/decoder for Node.js, rather than to be used as a standalone tool.

The package ships prebuilt binaries for the common Node.js platforms (macOS arm64/x64, Linux x64/arm/arm64, Windows x64) as scoped optionalDependencies, resolved at install time via node-gyp-build-optional-packages, with a node-gyp fallback to compile from source when no prebuild matches the host.

What You Get

• A compiled N-API addon exposing extractStrings(buffer, start, end) for fast, batched string extraction from MessagePack buffers
• Prebuilt binaries for darwin-arm64, darwin-x64, linux-x64, linux-arm, linux-arm64, and win32-x64, installed automatically as optional dependencies
• A node-gyp fallback build path (binding.gyp + src/extract.cpp) for platforms without a matching prebuild
• Latin-1 string-block batching that combines consecutive single-byte-encodable strings into one allocation to reduce V8 string-creation overhead
• A CLI helper (download-msgpackr-prebuilds) for explicitly fetching prebuilt binaries in constrained install environments

Common Use Cases

• Transparent dependency of the msgpackr package — installed automatically when an app depends on msgpackr, with no direct API calls from application code
• High-throughput MessagePack decoding in Node.js services that exchange binary payloads (RPC, caching layers, message queues)
• Environments needing a native fallback build when no prebuilt binary matches the host platform/Node ABI
• Performance-sensitive backends where avoiding per-string JS allocation overhead during binary deserialization matters

Under The Hood

Architecture The module is deliberately minimal: index.js just does module.exports = require('node-gyp-build-optional-packages')(__dirname), delegating platform/ABI resolution entirely to that loader, which picks a matching prebuilt .node binary from an optionalDependency package or falls back to a local node-gyp build described by binding.gyp. All real logic lives in one file, src/extract.cpp, in an Extractor class that walks a MessagePack buffer with a 256-entry function-pointer dispatch table (token_handler) keyed by the MessagePack type byte, tracking position/writePosition/stringStart state to emit up to 255 strings per invocation; callers (i.e. msgpackr) re-invoke it with a new offset to continue. There are no JS-side abstraction layers, no configuration surface, and no exported types beyond the single native function — the addon is a narrow, single-purpose extension of msgpackr rather than a general library with its own architecture to speak of.

Tech Stack Built on Node-API (node_api.h) for ABI-stable native bindings across Node versions, with an optional, flag-gated V8 API path (ENABLE_V8_API, controlled via ENABLE_V8_FUNCTIONS) for extra performance primitives. Building and packaging lean on the prebuildify ecosystem — prebuildify-platform-packages, prebuildify-ci, and prebuildify-cross — to cross-compile and publish per-platform binaries as scoped @msgpackr-extract/* optionalDependencies, consumed at install time by node-gyp-build-optional-packages; CircleCI and a GitHub Actions workflow (.github/workflows/prebuild.yml) handle building and attaching those prebuilds to GitHub releases, while binding.gyp drives the local node-gyp fallback compile.

Code Quality There is no dedicated test suite: the package.json test script is simply node ./index.js, a smoke check that the compiled addon loads without throwing, not a suite of assertions against extraction behavior. Error handling in extract.cpp is limited to a couple of explicit napi_throw_type_error calls for truncated/malformed buffers; there’s no linter or formatter configuration, and no TypeScript types are shipped. The C++ itself is compact and has explanatory comments describing the parsing/batching strategy, but overall this is unvalidated, untyped native code that leans entirely on msgpackr’s own test suite (a separate package) to catch regressions.

API Design The public surface is intentionally tiny — one native function, extractStrings(buffer, start, end) — which keeps integration boilerplate at essentially zero for the one consumer it’s built for (msgpackr just requires the module and calls the function). Installation is zero-config for downstream users, since the platform-specific binary is resolved automatically. The tradeoff is that the function’s actual call contract (offset semantics, how partial results signal the need for another call) isn’t documented anywhere beyond the README’s one-paragraph behavioral summary — understanding it requires reading src/extract.cpp or msgpackr’s own decoder, since this package was never meant to have a general audience beyond its single downstream dependency.