Import Predictions

Do you already know what you are doing and only want to look over a Jupyter Notebook example to import your predictions? We provide one here.

To upload predictions in Encord, you need to create a prediction branch. This guide explains everything you need to know for importing predictions.

Predictions Workflow

Import Predictions to Project: Start by importing your labels and predictions into your Project.
Analyze the Predictions: Once your predictions are imported, specify the prediction set you want to analyze.
Select the Predictions: Once analysis completes, select the prediction set you want to view in the Explore tab of your Project.

Supported Prediction Formats

Encord Format (Recommended)

Supports multi-level nested classifications (radio, checklist, or free-form text) under objects or classifications.
Handles all object types and classification.

Only top-level objects and classifications are considered when calculating in model metrics.

COCO Format

Does not support multiple levels of nested classifications (radio, checklist, or free-form text) under tools or classifications.

Confidence Score

You can include confidence scores when uploading predictions. Encord automatically calculates model metrics based on your prediction set and assigned confidence scores.

Prediction Branches

When importing prediction sets into Encord, they are added as branches to individual label rows on your data units (images, videos, audio). Each data unit has the following:

A MAIN branch for ground truth annotations or pre-labels.
Optional Consensus branches and Prediction branches for different prediction sets.

You can export main branch labels from the UI. Consensus labels and model predictions can only be exported using the SDK.

List Branches

Use list_branches to list all the branches (imported or Consensus) in a Project.

List branches

# Import dependencies
from encord import EncordUserClient

# User input
SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"  # Replace with the file path to your SSH private key
PROJECT_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the unique ID for the Project

# Create user client using ssh key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
    # For US platform users use "https://api.us.encord.com"
    domain="https://api.encord.com",
)

project = user_client.get_project(PROJECT_ID)

# Write the branch names to a file. The main branch is always named "main".
with open("branches.txt", "w") as output_file:
    for branch_name in project.list_branches():
        output_file.write(f"{branch_name}\n")

1. Import Predictions

Import your predictions to a Project. Encord currently supports importing predictions from the Encord format and from COCO.

Import Encord-Format Predictions

Use branch_name to create a prediction branch in label_rows_v2 for a data unit. Get the label rows with list_label_rows_v2(branch_name=...), add your predictions, and save the label rows: the predictions are saved to that branch, not to the main branch.

branch_name is case sensitive and can be up to 255 characters long.
branch_name supports letters (a-z, A-Z), digits (0-9), and the following special characters: hyphens (-), underscores (_), periods (.), and forward slashes (/).

Import Any Prediction Type to a Prediction Branch

Every example on the Import Labels page works for predictions, for all label types: bounding boxes, rotatable bounding boxes, polygons, polylines, keypoints, bitmasks, object primitives, and classifications. Make these changes to an example:

Pass branch_name when you get the label rows.
When you call set_for_frames, pass manual_annotation=False to mark the label as a prediction rather than a manual annotation, and pass your model’s score as confidence. If you leave them out, manual_annotation defaults to True and confidence defaults to 1.0.

Get label rows on a prediction branch

label_rows = project.list_label_rows_v2(
    data_title_eq=DATA_UNIT_TITLE,
    branch_name=PREDICTION_BRANCH_NAME,  # Saving these label rows writes to this branch
)

Mark a label as a prediction

box_object_instance.set_for_frames(
    coordinates=coordinates,
    frames=frame_number,
    manual_annotation=False,  # A model prediction, not a manual annotation
    confidence=model_confidence,  # Your model's confidence score for this prediction
)

Example: Bounding Box Predictions

This example imports one bounding box prediction (Cherry) to a video (Cherries_video.mp4) on a prediction branch. The same object instance is tracked across frames 103, 104, and 105, with the model’s confidence score on each frame.

Bounding Box Predictions on a Prediction Branch

# Import dependencies
from encord import EncordUserClient
from encord.objects import Object
from encord.objects.coordinates import BoundingBoxCoordinates

# User input
SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"  # Replace with the file path to your SSH private key
PROJECT_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the unique ID for the Project
PREDICTION_BRANCH_NAME = "my-prediction-branch-I"  # Replace with the name of your prediction branch
ONTOLOGY_OBJECT_TITLE = "Cherry"  # Replace with the title of a bounding box object in your Ontology
DATA_UNIT_TITLE = "Cherries_video.mp4"  # Replace with the title of a video in the Project

# Create user client using ssh key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
    # For US platform users use "https://api.us.encord.com"
    domain="https://api.encord.com",
)

project = user_client.get_project(PROJECT_ID)

# Get the label row for the video on the prediction branch
label_rows = project.list_label_rows_v2(
    data_title_eq=DATA_UNIT_TITLE,
    branch_name=PREDICTION_BRANCH_NAME,
)
if not label_rows:
    raise ValueError(f"No data unit titled '{DATA_UNIT_TITLE}' in the Project")
label_row = label_rows[0]
label_row.initialise_labels()

# Find the bounding box object in the Ontology
box_ontology_object: Object = project.ontology_structure.get_child_by_title(
    title=ONTOLOGY_OBJECT_TITLE, type_=Object
)

# Model output for each frame: the bounding box and the model's confidence score
predictions_per_frame = {
    103: (BoundingBoxCoordinates(height=0.5, width=0.5, top_left_x=0.2, top_left_y=0.2), 0.93),
    104: (BoundingBoxCoordinates(height=0.5, width=0.5, top_left_x=0.3, top_left_y=0.3), 0.88),
    105: (BoundingBoxCoordinates(height=0.5, width=0.5, top_left_x=0.4, top_left_y=0.4), 0.81),
}

# Create one object instance, so the same cherry is tracked across all three frames
box_object_instance = box_ontology_object.create_instance()
for frame_number, (coordinates, model_confidence) in predictions_per_frame.items():
    box_object_instance.set_for_frames(
        coordinates=coordinates,
        frames=frame_number,
        manual_annotation=False,  # A model prediction, not a manual annotation
        confidence=model_confidence,  # The model's confidence score for this prediction
    )

label_row.add_object_instance(box_object_instance)

# Save the predictions to the prediction branch
label_row.save()
print(f"Saved predictions for {label_row.data_title} on branch {label_row.branch_name}")

Import COCO Labels as Predictions

The following code imports COCO labels as predictions to a prediction branch. For more information on importing COCO labels into Encord, refer to our documentation. Replace the values in the # User input section with your own. Each COCO image is matched to a data unit by its video_title or image_title (present in COCO files exported from Encord), or otherwise by its file_name, which must match the data unit title in Encord. confidence_field_name is the name of the field in each COCO annotation that holds your model’s confidence score, for example score.

COCO Import to a Prediction Branch

# Import dependencies
import json
from pathlib import Path

from encord import EncordUserClient
from encord.exceptions import OntologyError
from encord.objects import Object
from encord.utilities.coco.datastructure import FrameIndex

# User input
SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"  # Replace with the file path to your SSH private key
PROJECT_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the unique ID for the Project
PREDICTION_BRANCH_NAME = "my-prediction-branch-I"  # Replace with the name of your prediction branch
COCO_FILE_PATH = "COCOimportfile.json"  # Replace with the full path to your COCO file
CONFIDENCE_FIELD_NAME = "score"  # Replace with the name of the COCO annotation field that holds your model's confidence score

# Create user client using ssh key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
    # For US platform users use "https://api.us.encord.com"
    domain="https://api.encord.com",
)

project = user_client.get_project(PROJECT_ID)

# Load the COCO annotations JSON file
labels_dict = json.loads(Path(COCO_FILE_PATH).read_text())

# Build a mapping from COCO category IDs to the feature hashes of the objects in your Ontology
category_id_to_feature_hash = {}
for coco_category in labels_dict["categories"]:
    try:
        ontology_object = project.ontology_structure.get_child_by_title(title=coco_category["name"], type_=Object)
        category_id_to_feature_hash[coco_category["id"]] = ontology_object.feature_node_hash
    except OntologyError:
        print(f"Could not match {coco_category['name']} to an object in the Ontology. Annotations in this category are skipped.")

# Build a mapping from COCO image IDs to Encord frame indices, using the label rows on the branch
image_id_to_frame_index = {}
data_title_to_label_row = {lr.data_title: lr for lr in project.list_label_rows_v2(branch_name=PREDICTION_BRANCH_NAME)}
for img in labels_dict["images"]:
    if "video_title" in img:
        # COCO file exported from Encord: a video frame. The file name ends with the frame number, for example 105.jpg
        lr = data_title_to_label_row[img["video_title"]]
        frame_num = int(img["file_name"].split("/")[-1].split(".")[0])
    elif "image_title" in img:
        # COCO file exported from Encord: an image
        lr = data_title_to_label_row[img["image_title"]]
        frame_num = 0
    else:
        # Other COCO files: the file name must match the data unit title in Encord
        lr = data_title_to_label_row[img["file_name"]]
        frame_num = 0

    image_id_to_frame_index[img["id"]] = FrameIndex(lr.data_hash, frame=frame_num)

# Import the COCO labels to the branch
project.import_coco_labels(
    labels_dict,
    category_id_to_feature_hash,
    image_id_to_frame_index,
    branch_name=PREDICTION_BRANCH_NAME,
    confidence_field_name=CONFIDENCE_FIELD_NAME,
)

Verify Prediction Import

After importing your predictions, verify that your predictions imported. The following code prints all labels and predictions on all branches.

Verify Prediction Import

# Import dependencies
from typing import cast

from encord import EncordUserClient
from encord.objects.ontology_object_instance import AnswersForFrames
from encord.objects.options import Option

# User input
SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"  # Replace with the file path to your SSH private key
PROJECT_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the unique ID for the Project

# Create user client using ssh key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
    # For US platform users use "https://api.us.encord.com"
    domain="https://api.encord.com",
)

project = user_client.get_project(PROJECT_ID)

# Get the label rows on all branches
# Without include_all_label_branches=True, only the label rows on the main branch are returned
label_rows = project.list_label_rows_v2(include_all_label_branches=True)

# Download the labels for each label row, using a bundle
with project.create_bundle() as bundle:
    for label_row in label_rows:
        label_row.initialise_labels(bundle=bundle)


# Get a readable answer: the option title for a radio button, a list of option titles
# for a checklist, the text for a text attribute, or None when there is no answer
def describe_answer(answer: object) -> object:
    if isinstance(answer, Option):
        return answer.title
    if isinstance(answer, list):
        return [option.title for option in answer]
    return answer


for label_row in label_rows:
    # Print the data unit and the branch the label row belongs to
    print(f"Title: {label_row.data_title}, branch: {label_row.branch_name}")

    # Print essential label information for all objects
    for object_instance in label_row.get_object_instances():
        print(f"objectHash: {object_instance.object_hash}")
        print(f"Object name: {object_instance.object_name}")
        print(f"featureHash: {object_instance.feature_hash}")
        print(f"Object color: {object_instance.ontology_item.color}")
        print(f"Ontology shape: {object_instance.ontology_item.shape}")

        # Print the frame number, the location of the object on the frame, the confidence,
        # and whether the label is a manual annotation (False for a model prediction)
        for annotation in object_instance.get_annotations():
            print(f"Frame {annotation.frame} -> {annotation.coordinates}")
            print(f"Confidence: {annotation.confidence}, manual annotation: {annotation.manual_annotation}")

        # Print the answer to each top-level attribute of the object
        for attribute in object_instance.ontology_item.attributes:
            if attribute.dynamic:
                # A dynamic attribute can change between frames: get_answer() returns each answer with its frame ranges
                for answer_for_frames in cast(AnswersForFrames, object_instance.get_answer(attribute)):
                    frame_ranges = [(frame_range.start, frame_range.end) for frame_range in answer_for_frames.ranges]
                    print(f"Attribute {attribute.title}: {describe_answer(answer_for_frames.answer)} on frames {frame_ranges}")
            else:
                # A static attribute has one answer for the whole object
                print(f"Attribute {attribute.title}: {describe_answer(object_instance.get_answer(attribute))}")

    # Print all essential classification information
    for classification_instance in label_row.get_classification_instances():
        print(f"classificationHash: {classification_instance.classification_hash}")
        print(f"Classification name: {classification_instance.classification_name}")
        print(f"featureHash: {classification_instance.feature_hash}")
        # get_answer() returns an Option for radio buttons, a list of Options for checklists,
        # a string for text classifications, and None when there is no answer
        answer = classification_instance.get_answer()
        if isinstance(answer, Option):
            print(f"Classification answer: {answer.value}")
            print(f"Classification answer hash: {answer.feature_node_hash}")
        elif isinstance(answer, list):
            print(f"Classification answer: {[option.value for option in answer]}")
            print(f"Classification answer hashes: {[option.feature_node_hash for option in answer]}")
        else:
            print(f"Classification answer: {answer}")

        # Print the frame number(s) that a classification appears on
        for classification_annotation in classification_instance.get_annotations():
            print(f"Classification appears on frame: {classification_annotation.frame}")

End-to-End Prediction Import Example

We provide an end-to-end example using a Jupyter Notebook here.

2. Analyze the Predictions

Encord must analyze the predictions before you can view the predictions.

Navigate to the Explore tab of your Project.
Navigate to Predictions > Prediction > Manage.
Click Analyze on the prediction set you want to analyze.
Specify whether you want to import Strict Match or Partial Match predictions.

DO NOT click Import on more than one prediction set at a time.

Strict Match Predictions

All attributes are matched. This mode makes the most comprehensive comparison. A prediction is considered as a true positive only if an exact match of all attributes matches with the ground truth. Strict match

Partial Match Predictions

Only attributes provided in the prediction are compared with the ground truth. Any missing attributes in the prediction, but present in the ground truth, have no impact on the outcome of the prediction. Partial Match

3. Select the Predictions

Once analysis completes:

Navigate to Predictions > Prediction.
Select the prediction set you want to view from the dropdown.

Delete Prediction Sets

Navigate to the Explore tab of your Project.
Navigate to Predictions > Prediction > Manage.
Click the ellipsis icon in the Actions column.
Click Delete.

Get Started

Data

Fundamentals

Annotate & Review

Label and Model Evaluation

Users and Workspaces

Import Predictions

Predictions Workflow

Supported Prediction Formats

Confidence Score

Prediction Branches

List Branches

1. Import Predictions

Import Encord-Format Predictions

Import Any Prediction Type to a Prediction Branch

Example: Bounding Box Predictions

Import COCO Labels as Predictions

Verify Prediction Import

End-to-End Prediction Import Example

2. Analyze the Predictions

Strict Match Predictions

Partial Match Predictions

3. Select the Predictions

Delete Prediction Sets

​Predictions Workflow

​Supported Prediction Formats

​Confidence Score

​Prediction Branches

​List Branches

​1. Import Predictions

​Import Encord-Format Predictions

​Import Any Prediction Type to a Prediction Branch

​Example: Bounding Box Predictions

​Import COCO Labels as Predictions

​Verify Prediction Import

​End-to-End Prediction Import Example

​2. Analyze the Predictions

​Strict Match Predictions

​Partial Match Predictions

​3. Select the Predictions

​Delete Prediction Sets

Predictions Workflow

Supported Prediction Formats

Confidence Score

Prediction Branches

List Branches

1. Import Predictions

Import Encord-Format Predictions

Import Any Prediction Type to a Prediction Branch

Example: Bounding Box Predictions

Import COCO Labels as Predictions

Verify Prediction Import

End-to-End Prediction Import Example

2. Analyze the Predictions

Strict Match Predictions

Partial Match Predictions

3. Select the Predictions

Delete Prediction Sets