ClearML

ClearML is an open-source MLOps and LLMOps platform that automates the full AI development lifecycle — from experiment tracking and data versioning to pipeline orchestration and model serving. By adding just two lines of code to any Python training script, data scientists and ML engineers gain automatic logging of hyperparameters, source code, environment dependencies, model weights, metrics, and rich media outputs like images, audio, and video samples.

Built around a three-tier architecture of Python SDK, self-hostable ClearML Server, and distributed ClearML Agent workers, the platform supports hybrid deployments across cloud, Kubernetes, and bare-metal infrastructure. Deep integrations with PyTorch, TensorFlow, Keras, XGBoost, LightGBM, FastAI, MegEngine, CatBoost, and scikit-learn mean most existing training scripts instrument themselves automatically through Python import hooks — no refactoring required.

Beyond experiment tracking, ClearML provides version-controlled dataset management over S3, Google Cloud Storage, Azure Blob, and NAS; a pipeline orchestration engine with DAG execution and cross-step artifact passing; GPU-optimized model serving via NVIDIA Triton with sub-5-minute deployment; and container-level fractional GPU memory partitioning for maximizing hardware utilization across concurrent experiments.

The platform has been under active development since 2019 and is used by teams ranging from individual researchers to enterprise MLOps organizations building reproducible, scalable AI workflows.

What You Get

• Automatic Experiment Tracking - Logs source code (including uncommitted diffs), hyperparameters from argparse, Click, Hydra, and TF defines, execution environment packages, model snapshots, TensorBoard scalars/images/audio, Matplotlib plots, and stdout/stderr — all captured by adding Task.init() to existing scripts.
• Version-Controlled Dataset Management - CLI and Python SDK for creating, uploading, and downloading versioned datasets stored on S3, Google Cloud Storage, Azure Blob, or NAS, with differentiable lineage tracking and reproducible data pipeline support via clearml-data.
• Pipeline Orchestration Engine - DAG-based pipeline controller that clones, parameterizes, queues, and monitors task graphs remotely, passing artifacts between steps and supporting nested pipelines-of-pipelines with retry and caching logic.
• Remote Execution and Agent Workers - clearml-agent daemons clone task environments (packages, Docker containers, git state) and re-execute experiments remotely, enabling full reproducibility and distributed training across cloud, Kubernetes, or bare-metal workers from a central queue.
• GPU-Optimized Model Serving - Deploy inference endpoints backed by NVIDIA Triton in under 5 minutes, with built-in model monitoring, traffic routing, and auto-scaling for production workloads.
• Fractional GPU Memory Partitioning - Container-level driver that limits GPU memory allocation per process, allowing multiple concurrent experiments to safely share a single GPU without interference.
• Hyperparameter Optimization - Black-box HPO using Optuna integration and Bayesian algorithms to search parameter spaces across remote workers, with full experiment tracking for every trial.
• Live Orchestration Dashboard - Real-time web UI for monitoring worker queues, autoscalers, pipeline execution status, and compute cluster utilization across all cloud and on-prem resources.

Common Use Cases

• Reproducible deep learning research - A PhD student adds two lines to existing PyTorch Lightning training scripts and gains automatic logging of every hyperparameter variation, dataset version, and checkpoint across 500+ experiments, enabling direct comparison and full environment replay months later.
• Enterprise LLM deployment with RAG - An ML engineering team uses ClearML’s pipeline orchestration to automate data ingestion, fine-tuning, and evaluation stages for a production RAG system, with each stage’s outputs versioned as artifacts and passed downstream automatically.
• Multi-team experiment governance - A fintech ML platform team deploys a self-hosted ClearML Server to enforce experiment reproducibility across 60 data scientists, using project-level organization and tag-based filtering to manage model lineage across regulatory audit requirements.
• Cloud cost optimization with spot instances - A startup configures ClearML’s AWS Auto-Scaler to spin up GPU spot instances on demand, using fractional GPU allocation to run four experiments per instance and the agent’s checkpoint-resume capability to handle spot interruptions transparently.
• CI/CD for ML models - A computer vision team triggers ClearML pipeline runs from GitHub Actions on each dataset update, with automatic experiment tracking, model evaluation against held-out test sets, and conditional deployment to Triton serving if accuracy thresholds are met.

Under The Hood

Architecture ClearML uses a three-tier architecture where the Python SDK acts as a domain-facing client, a separately deployed ClearML Server stores all experiment data and serves the web UI, and ClearML Agent workers execute remote tasks by replaying captured environments. The core domain entities — Task, Model, and Dataset — are structured as facade objects delegating to backend interface implementations, creating clean separation between the public API surface and REST communication mechanics. The ApiServiceProxy performs dynamic API version negotiation at connection time, selecting the highest-compatible service module from versioned subpackages, enabling client-server backward compatibility across long-running deployments without version pinning. Framework bindings for PyTorch, TensorFlow, Keras, XGBoost, and others are implemented as import-time monkey-patchers that inject logging hooks transparently, requiring no changes to existing training code. The pipeline controller implements a DAG execution engine using Python threading and multiprocessing, serializing pipeline state as task artifacts for cross-process reproducibility and failure recovery.

Tech Stack The SDK is pure Python supporting versions 3.6 through 3.14, distributed as a setuptools package with optional extras for storage backends. Core dependencies include attrs for declarative data classes, furl for URL construction, jsonschema for API payload validation, psutil for real-time resource monitoring, requests for HTTP communication, and pyjwt for server authentication. Framework integrations span PyTorch (including Lightning and Ignite), TensorFlow, Keras, AutoKeras, FastAI, XGBoost, LightGBM, MegEngine, CatBoost, and scikit-learn. Storage integrations cover S3-compatible object stores, Google Cloud Storage, Azure Blob Storage, and NAS-mounted filesystems. Linting is enforced with flake8 and flake8-bugbear, and the test harness uses pytest.

Code Quality The codebase carries comprehensive type hints throughout and ships a py.typed marker for PEP 561 compliance, signaling serious commitment to static analysis tooling. Naming conventions follow PEP 8 consistently across a large codebase (over 8MB of Python source). Linting with flake8-bugbear catches common antipatterns automatically. However, the unit test suite is relatively sparse for a project of this maturity — pytest infrastructure exists but actual test coverage within the core SDK package is limited, with most validation happening through integration tests against a live server. Error handling in framework binding modules deliberately uses broad exception suppression to avoid breaking user training scripts when optional integrations fail, which trades reliability for developer experience. Docstrings are abundant in core modules, and the examples directory provides comprehensive working code across all supported frameworks.

Self-Hosting

ClearML is released under the Apache License 2.0, one of the most permissive open-source licenses available. You can use it commercially, modify the source, distribute it, and sublicense it without any copyleft obligations — your own code does not need to be open-sourced. The only requirements are preserving copyright notices and the license text when redistributing. This makes ClearML genuinely suitable for enterprise production deployments without legal complexity.

Self-hosting ClearML requires deploying the ClearML Server, which is a separate component from the Python SDK and is itself open source (available at github.com/clearml/clearml-server). The server runs as a set of Docker containers (MongoDB, Elasticsearch, Redis, and the ClearML API server) and requires meaningful infrastructure investment — production deployments typically need persistent volumes for model artifacts and experiment data, enough memory for Elasticsearch indexing, and a reverse proxy for TLS termination. You are responsible for backups, upgrades, and database maintenance. For agent workers, each machine running clearml-agent needs to be provisioned and configured to pull from your task queue. The operational surface is real: a team deploying ClearML at scale will spend meaningful time on infrastructure management.

The managed ClearML hosted service at app.clear.ml offers a free tier with generous limits and paid enterprise plans that add SSO, RBAC, SLA guarantees, and dedicated support. Some advanced features — including hyper-datasets (annotation-level dataset versioning) and the GenAI App Engine with RBAC-secured endpoints — require the enterprise server tier and are not available in the open-source server. If your team needs fine-grained access controls, enterprise support contracts, or those advanced dataset features, the hosted or on-prem enterprise offering provides them; the community self-hosted path covers the core experiment tracking, pipeline orchestration, and model serving capabilities fully.