srsly

srsly bundles the serialization libraries that Explosion’s own projects — spaCy and Prodigy — repeatedly needed, wrapping ujson, msgpack, cloudpickle, and ruamel.yaml behind one consistent, high-level API rather than requiring each caller to handle every format’s quirks and cross-platform/cross-Python encoding pitfalls separately. It supports JSON, JSONL (newline-delimited JSON, including gzipped variants), MessagePack, Pickle, and YAML through paired _dumps/_loads and file-oriented write_*/read_* functions with a consistent naming scheme.

The package also includes a heavily customized fork of msgpack-numpy with corrected round-trip behavior for np.float64 objects, and MessagePack serialization automatically converts CuPy arrays to NumPy before encoding when both libraries are installed. Because it’s distributed as prebuilt binary wheels, it gives downstream libraries like spaCy a single dependency that covers five serialization formats without each consumer needing to manage separate format-specific packages and their platform-specific build quirks.

What You Get

• JSON API: json_dumps/json_loads, write_json/read_json, plus gzip and JSONL (newline-delimited JSON) variants (write_jsonl/read_jsonl, write_gzip_json/read_gzip_json)
• MessagePack API: msgpack_dumps/msgpack_loads with automatic NumPy array round-trip support (and CuPy-to-NumPy conversion when both are installed)
• Pickle API built on cloudpickle for serializing objects standard pickle can’t handle, such as lambdas and locally defined functions
• YAML API built on ruamel.yaml with unsafe load paths deliberately excluded for safety
• is_json_serializable helper to check whether an object can be JSON-encoded before attempting serialization
• Distributed as prebuilt binary wheels across platforms, avoiding per-machine compilation of the underlying C-accelerated libraries

Common Use Cases

• Persisting NLP pipeline artifacts, training data, and configs in spaCy and Prodigy without hand-writing per-format serialization wrappers
• Writing and reading large NDJSON/JSONL datasets line-by-line, including gzip-compressed variants, for corpus storage
• Serializing NumPy or CuPy array data via MessagePack with correct float64 round-tripping for downstream ML pipelines
• Safely loading YAML configuration files without exposing the unsafe-deserialization code paths of raw PyYAML/ruamel.yaml

Under The Hood

Architecture - the public srsly namespace re-exports functions from format-specific internal modules (_json_api.py, _msgpack_api.py, _pickle_api.py, _yaml_api.py), each a thin, hardened wrapper around its underlying library (ujson, msgpack, cloudpickle, ruamel.yaml respectively); _msgpack_numpy.py is a customized fork of the msgpack-numpy project patched specifically to fix round-trip precision loss on np.float64 values. Tech Stack - pure Python (99%+) with hard dependencies on ujson, msgpack, cloudpickle, and ruamel.yaml, and optional NumPy/CuPy support for the MessagePack array path; distributed as prebuilt wheels via the wheelwright build infrastructure shared with spaCy. Code Quality - a srsly/tests/ directory covering each format API, run via pytest --pyargs srsly; the project is feature-complete and in low-activity maintenance mode (roughly one release every quarter, no urgent bug backlog) reflecting its role as a stable low-level dependency rather than an actively evolving library. API Design - every format follows the same three-tier naming convention — <format>_dumps/<format>_loads for in-memory strings/bytes, write_<format>/read_<format> for files, with - treated as stdin/stdout — so switching between JSON, JSONL, MessagePack, and YAML in calling code is a near mechanical function-name swap.