weasel

Weasel is a lightweight CLI workflow system from Explosion AI for managing end-to-end machine learning pipelines: cloning a project template, loading data, running training/packaging steps in order, and pushing or pulling assets to remote storage. It is the standalone successor to spaCy Projects, decoupled from spaCy itself so it can drive workflows for any use case or domain.

A Weasel project is defined by a project.yml describing commands, assets, and workflows; the weasel CLI (invoked as python -m weasel) then clones templates, runs the defined command sequence, and manages remote storage for pushing/pulling data and trained artifacts between machines or team members.

What You Get

• weasel clone to fetch a pre-defined project template as a starting point
• weasel run to execute declared commands or multi-step workflows defined in project.yml
• weasel push/weasel pull for syncing assets and trained outputs with remote storage backends
• DVC integration (dvc.py) for teams already using Data Version Control alongside Weasel workflows
• A schema-validated project.yml format (schemas.py) so misconfigured workflows fail fast with clear errors

Common Use Cases

• Defining reproducible end-to-end pipelines for training, packaging, and exporting spaCy or other NLP models
• Sharing a standardized project template across a team so everyone runs the same data-to-model workflow
• Syncing large training assets and model artifacts to/from remote storage as part of a workflow
• Migrating existing spaCy Projects workflows to a standalone, spaCy-independent orchestration tool

Under The Hood

Architecture: Weasel’s CLI (weasel/cli/) implements each subcommand as its own module — clone.py, run.py, push.py, pull.py, assets.py, remote_storage.py, dvc.py — dispatched through main.py and weasel/main.py, with schemas.py defining and validating the project.yml structure that drives every command.

Tech Stack: Pure Python packaged as a standard pip-installable CLI tool, with optional DVC integration for asset/version tracking and pluggable remote storage backends for push/pull operations.

Code Quality: Test coverage focuses on validation and schema correctness (test_validation.py, test_schemas.py) with CI configured via GitHub Actions; as a young standalone extraction from spaCy Projects, the test suite is comparatively narrow relative to its parent ecosystem.

API Design: The project.yml + CLI-subcommand model (clone/run/push/pull) mirrors familiar workflow-runner conventions, keeping the learning curve low for anyone who has used similar declarative task runners, and documentation explicitly covers migration from the older spaCy Projects format.