dirty-json

dirty-json is a small Node.js library built around a custom lexer and a hand-written shift-reduce parser designed specifically to handle JSON-like text that fails a standard JSON.parse call. It targets the common real-world case where JSON is hand-typed, copy-pasted from logs, or scraped from another source: unquoted object keys, single-quoted strings, unescaped embedded quotes, trailing and repeated commas, and missing values are all recovered into a usable structure rather than throwing.

It exposes a single parse(text, config) function that mirrors the shape of the native API, falls back to JSON.parse automatically when the input turns out to already be valid, and offers an optional duplicateKeys mode that nests repeated object keys instead of silently overwriting them — useful when parsing user-generated or scraped data where strict compliance can’t be guaranteed.

What You Get

• A parse(text, config) function with the same call shape as JSON.parse, making adoption a near one-line change
• Automatic recovery from unquoted keys, single-quoted strings, embedded quotes, and trailing/repeated commas
• Optional automatic fallback to native JSON.parse when the input is actually valid JSON
• An opt-in duplicateKeys mode that preserves repeated object keys instead of overwriting them
• Hand-authored TypeScript definitions (dirty-json.d.ts) shipped alongside the plain JS source

Common Use Cases

• Parsing JSON-like configuration or data that was hand-edited and drifted from strict JSON syntax
• Recovering structured data from log lines or scraped text containing embedded, unescaped quotes
• Ingesting loosely-formatted API responses or legacy data dumps that predate strict validation
• Handling user-submitted or copy-pasted JSON in tools where rejecting the input outright is worse than a best-effort parse

Under The Hood

Architecture The library is a small three-module pipeline: lexer.js tokenizes input via a custom regex-based lexer (built on the lex package) into typed tokens for quotes, floats, ints, punctuation, and a catch-all TOKEN rule for unquoted runs; parser.js implements a hand-written shift-reduce parser as a stack.push() / reduce() loop with dozens of numbered “Rule N” and “Error rule N” branches that repair malformed sequences (missing quotes, dangling commas, embedded quotes) as tokens are shifted; and dirty-json.js is the thin public entry point that calls the parser and falls back to native JSON.parse if it throws. There’s no plugin architecture or injectable strategy — it’s a single linear pipeline (text → tokens → shift-reduce stack → tree walk → plain JS value) — so extending recovery behavior means adding another branch to parser.js’s large reduce() switch statement, which risks affecting matching for other malformed-input categories.

Tech Stack Pure JavaScript (CommonJS, "use strict"), targeting Node >=6, with three runtime dependencies: lex for tokenization, unescape-js for string unescaping, and utf8 for encoding normalization. There’s no build step or bundler — the package ships plain .js files plus a hand-written dirty-json.d.ts for TypeScript consumers. Dev tooling is mocha, istanbul, and coveralls, historically wired through Codeship CI; no GitHub Actions workflow is present in the repo. A jshintConfig block in package.json is the only linting configuration.

Code Quality The test suite is substantial for the package’s size — test/parser_test.js and test/lexer_test.js run well over a hundred assertions comparing dJSON.parse output against native JSON.parse, plus a dedicated corpus under test/nst/ (edge-case JSON files from an external test suite) that’s iterated and asserted against automatically. Error handling is defensive at the top level — the public parse() wrapper catches parser failures and retries with native JSON.parse — but most malformed input is repaired through custom “Error rule N” branches rather than raised as exceptions; the parser only throws a generic Error with row/column info in a couple of genuinely unhandled cases. There’s no TypeScript source, so type safety is documentation-only via the hand-maintained .d.ts file, and naming is dense and abbreviation-heavy (LEX_KV, LEX_CVALUE, LEX_VLIST) with a large block of magic constants.

API Design The public surface is close to zero-config: a single parse(text, config) function that mirrors JSON.parse’s call shape, with two optional named booleans (fallback, duplicateKeys) rather than a sprawling options object, so migrating existing JSON.parse call sites is close to a one-line change. The hand-maintained dirty-json.d.ts gives TypeScript consumers full autocomplete and inline docs for both options, which is more typing investment than most similarly-scoped micro-libraries bother with. Documentation lives entirely in the README as runnable input/output snippet pairs, enough to get from zero to a first successful parse in under a minute, though the exception behavior for the fallback:false case and genuinely unparseable input is under-documented and only discoverable by reading source or tests.