Skip to main content

Running Quality Metrics

There are a couple of ways to compute quality metrics via code.


This is equivalent to running the metric run command from the CLI.

Running Builtin Metrics

To run all metrics available via Encord Active, you can to the following:

from pathlib import Path

from encord_active.lib.metrics.execute import run_metrics

project_path = Path("/path/to/your/project/root")
run_metrics(data_dir=project_path, use_cache_only=True)

The run_metrics function also allows you to filter which metrics to run by providing a filter function:

options = dict(
filter_func=lambda m: m().metadata.title == "<query>",

Running Data or Label Metrics only

There is a utility function you can use to run targeted subsets of metrics:

from encord_active.lib.metrics.execute import (
from encord_active.lib.metrics.types import EmbeddingType

run_metrics_by_embedding_type(EmbeddingType.IMAGE, **options)
run_metrics_by_embedding_type(EmbeddingType.OBJECT, **options)
run_metrics_by_embedding_type(EmbeddingType.CLASSIFICATION, **options)

Running Custom Metrics

If you have already written a custom metric - let's call it SuperMetric, then you can execute it on a projects with the following code:

from pathlib import Path

from encord_active.lib.metrics.execute import execute_metrics

from super_metric import SuperMetric

project_path = Path("/path/to/your/project")
execute_metrics([SuperMetric()], data_dir=project_path)

The CLI also has a way to registering metrics to the project to easily execute them afterwards.