iconv-lite

iconv-lite is a pure-JavaScript character encoding conversion library that encodes and decodes buffers to and from strings across hundreds of encodings, from Unicode variants like UTF-8, UTF-16, UTF-32, and UTF-7 to legacy singlebyte and multibyte code pages (Windows-125x, ISO-8859, IBM/DOS, Macintosh, KOI8, Shift_JIS, GBK, Big5, and more). It requires no native code compilation, so it installs quickly and runs unmodified on Windows, in sandboxed environments, and in the browser via bundlers like webpack or Browserify.

Widely depended on by projects such as Express.js’s body-parser, Grunt, Nodemailer, and Yeoman, iconv-lite exposes a simple encode()/decode() API plus a Streaming API built on Node’s stream.Transform for converting large payloads chunk by chunk without buffering the whole input in memory.

What You Get

• A simple encode(str, encoding) / decode(buf, encoding) API that mirrors Node’s built-in Buffer string encodings but covers hundreds more code pages
• A Streaming API (encodeStream() / decodeStream()) for converting large payloads chunk-by-chunk via Node’s Transform streams, with a .collect(cb) convenience helper
• Broad encoding coverage: all Node native encodings, additional Unicode forms (UTF-7, UTF-7-IMAP, UTF-16/-32 with BOM-aware endianness detection), and widespread singlebyte/multibyte legacy code pages (Windows-125x, ISO-8859, IBM/DOS, Macintosh, KOI8, CP932/936/949/950, GB2312/GBK/GB18030, Big5, Shift_JIS, EUC-JP)
• A dual-backend architecture (Node Buffer-based, and a browser/web TextDecoder-based backend) so the same codec logic runs unmodified on the server and in a bundled browser build
• Bundled TypeScript type definitions, including a generated union type of all supported encoding names for compile-time validation
• BOM (Byte Order Mark) handling on both encode and decode paths, configurable via stripBOM and addBOM options

Common Use Cases

• Converting uploaded or legacy files from Windows-125x, ISO-8859, Shift_JIS, GBK, or other regional encodings into UTF-8 strings for processing
• Decoding HTTP request/response bodies whose Content-Type charset isn’t UTF-8 (the pattern used internally by body-parser for non-UTF-8 form/text bodies)
• Streaming large binary files or HTTP payloads through an encoding conversion without loading them fully into memory
• Running character-encoding conversion inside browser bundles (via webpack/Browserify) where native iconv bindings aren’t available
• Re-encoding strings back into a specific legacy charset for output to systems or files that still expect a non-Unicode encoding

Under The Hood

Architecture iconv-lite separates codec logic from platform-specific byte/string primitives through a small dependency-injected “backend” interface (setBackend()): backends/node.js implements allocBytes/bytesToResult/allocRawChars/rawCharsToResult on top of Node’s Buffer, while backends/web.js implements the same interface with Uint8Array/ArrayBuffer and TextDecoder, letting one codec implementation run correctly in both Node and browser bundles. lib/index.js is the core: getCodec() resolves an encoding name through _canonicalizeEncoding() (stripping whitespace, control characters, and non-alphanumerics, explicitly guarding against __proto__/constructor prototype-pollution lookups) and walks the encodings map, which can alias through multiple layers before reaching a codec factory function; resolved codecs are cached in _codecDataCache. encodings/index.js aggregates codec/alias definitions from separate modules (internal, utf32, utf16, utf7, sbcs-codec/sbcs-data(-generated), dbcs-codec/dbcs-data, whatwg-aliases) via an explicit, manually-maintained module list rather than dynamic directory scanning, specifically to stay Browserify-compatible. Streaming support (lib/streams.js) is layered on top via enableStreamingAPI(), dependency-injecting Node’s stream module only when present so browser builds aren’t forced to bundle it.

Tech Stack The published package has zero runtime dependencies — encode/decode logic, all encoding tables, and stream wrapping are implemented entirely in JavaScript. Development tooling includes ESLint via the neostandard shareable config (with a handful of project-specific rule overrides for legacy code compatibility), TypeScript (~5.9) plus @arethetypeswrong/cli to validate the package’s published type exports, c8 for coverage, and mocha as the test runner. A separate generation/ npm workspace regenerates the singlebyte/multibyte encoding-table data from upstream sources (node-iconv, Unicode.org, WHATWG Encoding Standard). The package ships a browser field in package.json that swaps in lib/index-web.js and disables stream for bundlers.

Code Quality The test suite is extensive — dedicated spec files per encoding family (Big5, GBK, Shift_JIS, Cyrillic, Greek, Turkish, DBCS, SBCS, UTF-7, UTF-16, UTF-32, CESU-8, BOM handling, streams) plus a full Web Platform Tests conformance run (test/wpt) and a webpack bundle smoke test. CI (GitHub Actions) runs linting, the full test suite across environments, a dedicated Node-web-backend test grep, performance benchmarks across Ubuntu/Windows/macOS, TypeScript type-checking via tsc/attw, and CodeQL security scanning. Coverage is enforced at 90% statements/branches/lines and 95% functions via c8 configuration excluding only generated data files and vendored WPT fixtures. Encoding-name resolution is explicitly tested against prototype-pollution style lookups (__proto__, constructor), and the WHATWG decode paths document precise error-replacement semantics rather than leaving them implicit.

API Design The public surface is deliberately small and mirrors Node’s native Buffer encoding API: encode(str, encoding, options) and decode(buf, encoding, options) as the primary entry points, with encodingExists() for capability checks and lower-level getEncoder/getDecoder/getCodec for advanced use. Options objects (stripBOM, addBOM, defaultEncoding) are opt-in with sensible defaults, and the Streaming API is intentionally not auto-enabled in browser bundles (avoiding ~100KB of unwanted stream shim code) unless a consumer explicitly calls enableStreamingAPI(require('stream')), which the bundled TypeScript definitions document inline.