ClearML

ClearML is a Python client library that instruments machine learning training scripts with automatic experiment tracking, data versioning, and pipeline orchestration. Adding two lines of code to any Python script — from clearml import Task and Task.init(...) — captures git state, environment, hyperparameters, console output, resource utilization, and framework-specific metrics (PyTorch, TensorFlow, Keras, scikit-learn, XGBoost, and more) without further manual instrumentation.

Beyond experiment logging, the package exposes primitives for dataset version control (clearml.datasets), hyperparameter optimization and pipeline controllers (clearml.automation), and a model-serving reverse proxy (clearml.router), all built on the same underlying Task abstraction and communicating with a ClearML Server (self-hosted or the hosted app.clear.ml) over its REST API.

What You Get

• Automatic experiment capture - git diff, environment, hyperparameters, console output, and resource metrics logged with two lines of code
• Framework integrations for PyTorch, TensorFlow, Keras, scikit-learn, XGBoost, LightGBM, and more via clearml/binding/frameworks
• Dataset versioning through clearml.datasets.Dataset, differentiable data management on top of S3/GS/Azure/local storage
• Automation primitives - PipelineController, HyperParameterOptimizer, and TaskScheduler for building and scheduling ML workflows
• A FastAPI-based model-serving reverse proxy (clearml.router) with built-in endpoint telemetry

Common Use Cases

• Tracking every hyperparameter, metric, and artifact across hundreds of training runs without manual logging
• Reproducing a colleague’s experiment exactly by pulling the captured git commit, uncommitted diff, and installed package versions
• Versioning and sharing large training datasets stored on S3/GS/Azure without copying files into git
• Running Bayesian hyperparameter searches with Optuna or HpBandSter orchestrated through ClearML’s queueing system

Under The Hood

Architecture The package is layered around a top-level clearml/task.py (~5,700 lines) public Task class that end users import and call Task.init() on; it delegates to clearml/backend_interface/task/task.py for internal task-state management, which talks to clearml/backend_api/session/session.py (a REST session/auth layer) and typed service clients under clearml/backend_api/services/ (tasks, projects, events, queues) to communicate with the ClearML Server’s REST API. Framework integration lives in clearml/binding/ — one module per supported library (matplotlib_bind.py, click_bind.py, hydra_bind.py, joblib_bind.py, fire_bind.py, absl_bind.py, environ_bind.py) that monkeypatches or wraps the third-party library to auto-capture its state. clearml/storage/ abstracts object-storage backends (S3/GS/Azure/local) behind a single manager interface, clearml/automation/ layers pipeline/HPO/autoscaler orchestration on top of the Task primitive, and clearml/router/ adds a FastAPI-based reverse proxy for model serving. Config resolution flows through clearml/backend_config/ (layered config files plus environment-variable overrides). Because binding/, automation/, and datasets/dataset.py all construct or extend Task instances directly, changes to the core Task abstraction would ripple through nearly every subpackage.

Tech Stack A pure-Python package (100% Python by byte count) supporting a broad interpreter range, with per-Python-version pinned dependencies for numpy and Pillow visible in requirements.txt. Core runtime dependencies are requests (HTTP), PyYAML/pyhocon (config), jsonschema (API schema validation), furl (URL parsing), psutil (resource monitoring), numpy/Pillow (artifact and image handling), and pyjwt (auth tokens); pathlib2 and six are still present for older-Python compatibility. Packaging uses plain setuptools with the version single-sourced from clearml/version.py, built as a universal wheel. There is no async runtime or general web framework beyond the vendored FastAPI-based proxy used solely for the model-serving router. GitHub Actions CI is limited to CodeQL static analysis and an issue-triage bot — no visible automated test or build workflow.

Code Quality No test directory or test files exist anywhere in the repository — a repo-wide search for test-related paths returns nothing, so the OSS package ships without a visible automated test suite. Error handling mixes custom exception types (clearml/errors.py) with defensive try/except blocks around optional third-party imports, so framework bindings no-op cleanly when a library isn’t installed. Documentation leans on Sphinx-style docstrings (:param:/:type:) rather than exhaustive inline type annotations, though typing imports (Optional, Union, Sequence) are used throughout the public API surface. Naming follows conventional PEP8 style, and CodeQL is configured for static security analysis, but the absence of any pytest/tox configuration at the root is a notable gap for a package with this much surface area.

What Makes It Unique ClearML’s differentiator is “auto-magic” instrumentation: adding two lines of code captures git diff and uncommitted changes, the full installed-package environment, CLI hyperparameters (via argparse/Click/Fire/Hydra bindings), console output, matplotlib/TensorBoard artifacts, and resource utilization — turning an unmodified training script into a versioned, remotely-executable, queueable Task without explicit logging calls in the common case. clearml/automation/ then layers hyperparameter optimization (via Optuna/HpBandSter) and pipeline controllers directly on top of the same Task graph, so pipelines, HPO sweeps, and individual experiments all share one underlying data model rather than being bolted-on separate systems.