json-machine

JSON Machine is a drop-in replacement for inefficient iteration of large JSON documents in PHP. Instead of loading an entire file into memory with json_decode, it parses JSON incrementally and yields items through PHP generators, so memory usage stays flat no matter how big the input is.

It works over files, streams, in-memory strings, and chunked HTTP responses, supports JSON Pointer to target subtrees, offers recursive iteration, pluggable decoders, and progress tracking - all with no production dependencies beyond the optional ext-json.

What You Get

• A generator-based Items iterator that streams JSON without loading it all into memory
• JSON Pointer support to iterate a specific subtree of a document
• Sources for files, PHP streams, in-memory strings, and chunked HTTP responses
• Recursive iteration for deeply nested structures
• Pluggable decoders, progress tracking, and per-item error handling

Common Use Cases

• Iterating multi-gigabyte JSON files that will not fit in memory
• Streaming and processing large JSON API responses as they download
• Extracting a nested subtree from a big document via JSON Pointer
• ETL and import jobs that transform large JSON datasets record by record

Under The Hood

Architecture - The public Items facade (src/Items.php) wires together a lexer (src/Tokens.php) that emits JSON tokens from a chunk source (FileChunks, StreamChunks, StringChunks) and a Parser (src/Parser.php) that turns tokens into a generator of key/value pairs, filtered by a JSON Pointer; RecursiveItems adds nested iteration. Tech Stack - Pure PHP >=7.2 with no runtime dependencies except optional ext-json for decoding; a Makefile-driven CI pipeline runs PHPUnit with code coverage. Code Quality - The codebase is small, focused, and thoroughly tested with high PHPUnit coverage (advertised via codecov), and separates concerns cleanly across tokenizer, parser, chunk sources, and decoders. API Design - Getting started is a two-line foreach (Items::fromFile($path) as $key => $value), and advanced needs (subtrees, decoders, progress, error catching) are opt-in via an options object, keeping the common case trivial.