ClearML

Managing machine learning experiments has become reproducible and traceable with ClearML

Many machine learning engineers suffer from one recurring problem: running a successful experiment, then failing to reproduce the same result two weeks later because the fine details were not recorded accurately enough. ClearML was created specifically to solve this dilemma; it is an integrated MLOps platform that handles experiment tracking, model management, and automated pipeline orchestration, making the lifecycle of machine learning projects reviewable, reproducible, and scalable.

What is the ClearML tool?

ClearML is an open-source Machine Learning Operations (MLOps) platform, originally developed under the name Trains before the company rebranded it and expanded its scope in 2021. The tool enables technical teams to automatically track every training experiment without needing to make major changes to the source code; adding just two lines of Python is enough to start fully logging parameters, metrics, and outputs.

The platform consists of several integrated core components:

• ClearML Experiment Manager: The core of the platform responsible for experiment tracking and logging hyperparameters, metrics, and charts.
• ClearML Data: A tool for dataset versioning (Data Versioning) and tracking its data lineage.
• ClearML Agent: An agent that automatically executes tasks on remote infrastructure (GPU servers or cloud).
• ClearML Pipelines: A pipeline orchestrator (Pipelines) that connects data processing, training, and evaluation steps.
• ClearML Serving: A layer for deploying models and serving them as a REST API.
• ClearML Hyper-Parameter Optimization (HPO): A tool for automatically tuning hyperparameters using multiple algorithms.

The platform is available in two options: a SaaS hosted on ClearML’s infrastructure, or self-hosted deployment on an organization’s servers using Docker Compose or Kubernetes, making it suitable for environments that impose strict constraints on data privacy.

Key Features

Automatic experiment tracking

As soon as you call Task.init(project_name='my_project', task_name='experiment_1') at the beginning of the script, ClearML starts automatically logging everything: hyperparameter values from argparse or Hydra or Pydantic, loss and accuracy readings in each epoch, charts from matplotlib and plotly, sample images, and environment variables. Most importantly, this works with PyTorch, TensorFlow, Keras, scikit-learn, and XGBoost without any additional modifications.

Comparing visual experiments

The web dashboard allows you to compare two or more experiments side by side at the level of metrics, parameters, and interactive charts. For example, you can compare the training curves of two models with different learning rates on the same time axis, which saves hours of manual analysis.

Dataset Versioning with ClearML Data

The versioning system enables creating a snapshot of a dataset at any moment and linking it to the experiment that used it. Any previous version can be retrieved with a simple command: Dataset.get(dataset_project='my_data', dataset_name='v2.1'), with full tracking of the version history and file add/delete operations.

Hyperparameter Optimization (HPO)

The platform supports multiple algorithms for hyperparameter tuning such as Random Search, Grid Search, Bayesian Optimization (via Optuna), and Hyperband. The user defines the search space, then ClearML launches several parallel copies of the experiment on multiple GPU units to automatically select the best set of parameters.

Running experiments remotely

After installing ClearML Agent on a GPU server, any previous experiment can be sent to run on it at the press of a button from the UI, or via the command task.execute_remotely(queue_name='gpu_queue'). The Agent clones the code from Git, installs the dependencies, and runs the experiment in an isolated environment.

Model Management (Model Registry)

Each trained model is automatically registered in the Model Registry, with the ability to access it later via the API or the interface. Models can be categorized (Published, Archived, Draft), linked to the experiments that produced them, and shared among team members.

How to Use: A Step-by-Step Guide

Step One: Create an Account and Set Up the Connection

1. Go to clear.ml and create a free account.
2. After logging in, go to Settings ← Workspace ← API Credentials and copy the credentials.
3. Install the library: pip install clearml
4. Run: clearml-init and paste the credentials when prompted. The configuration file is created at ~/clearml.conf.

Step Two: Integrate Tracking into the Code

1. Add at the beginning of the script:
   from clearml import Task
task = Task.init(project_name='Image Classification', task_name='ResNet50-lr0.001')
2. If you are using argparse, all parameters will be logged automatically. They can be modified later from the interface before restarting.
3. To log custom metrics: task.get_logger().report_scalar(title='Validation', series='F1-Score', value=0.92, iteration=epoch)

Step Three: Track experiments from the dashboard

1. Open the dashboard at app.clear.ml
2. Select the project to see a list of all experiments with their status (Running, Completed, Failed).
3. Click on any experiment to view its details: metrics, charts, code, installed packages, and runtime.

Step Four: Set up ClearML Agent for Remote Execution

1. On the GPU server: pip install clearml-agent
2. Run: clearml-agent daemon --queue gpu_queue
3. From the dashboard, click any completed experiment ← Clone ← Enqueue ← choose gpu_queue. The Agent will automatically rerun it.

Advantages and Benefits

For machine learning engineers and researchers

ClearML eliminates the need for manual Excel files or Notion notes to track experiments. A researcher running twenty experiments a day can compare them all in minutes instead of hours. Likewise, the problem of “where is the code that produced this model?” is solved fundamentally because every experiment keeps a snapshot of the code and the Git state (commit hash).

For MLOps Teams

The ability to schedule multiple experiments in queues across distributed servers without needing to set up complex infrastructure significantly reduces the operational burden. An organization with 5 GPU servers can manage them all from a single interface.

For teams that work in a distributed manner

An engineer in Riyadh and an engineer in Cairo can work on the same project and see each other’s experiments in real time, with the ability to comment on experiments and share models via the API.

Disadvantages and Challenges

Initial Learning Curve

Despite the simplicity of the initial integration, fully leveraging features such as Pipelines and HPO requires an in-depth reading of the documentation. The API is rich but can feel dense for beginners, and some advanced settings require a solid understanding of Docker’s architecture.

Performance with Very Large Big Data

Uploading a huge artifact (data on the order of hundreds of GB) via the ClearML cloud server can be slow. The recommended solution is to use external storage such as S3 or GCS and record only the pointer in ClearML, but this adds an extra setup layer.

Visual Interface

The control panel is functional, but less polished than some competitors in terms of user experience. Some tasks that can be done with a single click in other tools require several steps here.

ClearML Serving is still maturing

Compared to Triton Inference Server or BentoML, the Serving feature in ClearML remains less mature and is not recommended for high-load production environments without extensive testing.

Hosted Version and Privacy

In the free hosted version, experiment and model data are uploaded to ClearML servers. For organizations with strict compliance requirements (GDPR, HIPAA), self-hosting is required, which entails additional infrastructure and technical expertise.

Comparison with competing tools

ClearML vs MLflow

MLflow is the most widely used for experiment tracking thanks to its simplicity and the support of the Apache community. But ClearML outperforms it in: deeper integration with the codebase without modifying it, built-in Agents for remote execution, and a more mature Data Versioning feature. In contrast, MLflow is easier to integrate with platforms like Databricks and has a broader ecosystem.

ClearML vs. Weights & Biases (W&B)

W&B offers a more polished user experience and more attractive visualizations, and it is the preferred choice in academic circles. However, W&B’s paid plans are much more expensive, and it does not natively include remote task execution at the same level as ClearML. ClearML is the more cost-effective choice for teams that need an integrated MLOps platform.

ClearML vs Neptune.ai

Neptune.ai specializes in experiment tracking and has an excellent interface for managing data teams, but it lacks the task execution and infrastructure management features available in ClearML. Neptune is also relatively more expensive for large teams.

ClearML vs Vertex AI / SageMaker

Full cloud platforms like Google Vertex AI and Amazon SageMaker offer built-in MLOps services, but they lock the user into the specific cloud provider and are much more expensive. ClearML works on any infrastructure (AWS, GCP, Azure, on-premise), which provides exceptional flexibility.

Practical Examples

Scenario 1: Fine-tuning an NLP model for Arabic text classification

A team is fine-tuning an AraBERT model to classify tweets. The engineer defines the search space: a learning rate between 1e-5 and 5e-5, a batch size between 16 and 32, and the number of epochs between 3 and 6. ClearML HPO is launched, running 20 parallel experiments on 4 GPU servers, then returning a performance report sorted by the best F1-score. What used to take a week is now completed in one day.

Scenario 2: Tracking Computer Vision Experiments in the Medical Field

A team working on detecting tumors in X-ray images needs to ensure full reproducibility for every experiment. They use ClearML Data to version datasets with every change to the annotation, and they link every produced model to the exact data version used in its training, which facilitates the model review process for regulatory approval purposes.

Scenario 3: Building an integrated data processing pipeline

A data company builds a 4-step pipeline: data extraction, cleaning, feature extraction, training. Using ClearML Pipelines, each step is defined as an independent step, and when the cleaning step changes, only the affected steps are automatically rerun, reducing total execution time by up to 60%.

Scenario 4: Reviewing models before production

Before deploying a new model, the team manager uses the comparison dashboard in ClearML to compare the candidate model with the current model in production across 12 different metrics, then the deployment decision is made based on documented data rather than guesswork.

Pricing

Free Plan (Free)

The free plan includes unlimited access to tracking features for individual users, with 100 GB of artifact storage, and support for an unlimited number of experiments and projects. It is perfectly sufficient for individual research and personal projects.

Teams Plan (Teams)

Starts at around $15 per user per month and includes expanded storage, user roles and permissions, and support for multiple users within a shared project. This is the recommended plan for small and medium teams.

Enterprise Plan (Enterprise)

Pricing is customized based on size, and includes: on-premise or private cloud deployment, unlimited storage, SLA guarantees, dedicated support, and integration with enterprise identity systems such as LDAP and SSO.

Free Self-Hosting

The platform’s source code is fully available on GitHub under the Apache 2.0 license. Any organization can deploy the entire platform on its servers using Docker Compose without any licensing cost, bearing only the infrastructure costs.

Evaluation and Getting Started Tips

Who is ClearML suitable for?

• Mid-sized and large machine learning teams that run dozens of experiments weekly and need strict organization and documentation.
• Organizations with privacy requirements that prefer self-hosting and not relying on third-party cloud services.
• Academic researchers at universities who need full reproducibility for scientific publishing without high cost.
• MLOps teams that manage multiple GPU servers and need a centralized scheduling system.

Who is it not suitable for?

• Beginners in machine learning who are working on simple educational projects; they will find the platform bigger than they need.
• Teams fully embedded in AWS or GCP may find that SageMaker or Vertex AI are easier to integrate with the rest of their cloud services.
• Those looking for exceptional visualizations as a top priority; W&B excels in this area.

Tips to Get Started Effectively

1. Start by integrating basic tracking into an existing project rather than creating a new project specifically.
2. Invest 30 minutes in understanding the Tags and Projects system from the start to avoid chaos later.
3. Try the Clone + Enqueue feature on an early completed experiment to understand the power of Agents.
4. Use YAML files or Hydra for hyperparameters settings instead of argparse for better management.
5. Check out the clearml-examples repository on GitHub to get ready-made examples for various use cases.

Summary

ClearML is an exceptionally comprehensive MLOps platform for anyone looking for an end-to-end solution that works on any infrastructure without being tied to a specific cloud provider. Its true value doesn’t show in the first trial; it becomes clear after a month of use, when an engineer can, with a single click, know: which code produced which model, with which data, with which parameters, and on which infrastructure.

The practical takeaway: if your team runs more than ten experiments per week and you find it hard to track them, or if you need an MLOps solution that can be deployed on your own servers for free, then ClearML is the best choice in its category. Start with the free plan at clear.ml and integrate it into an existing project; you’ll find yourself needing it in every subsequent project.