Aligned Image with Page Break

Encord Active does not only provide a streamlined method to currate your image data, Active also provides metrics and analytics to optimize your model's performance. Simply upload your model's predictions in to Active Cloud to start.

Your predictions must be imported to Active, before you can use the Predictions feature on the Explorer page and the Model Evaluation page.

STEP 1: Prepare Your Predictions for Import

:warning: Disclaimer: We strongly recommend that you are knowledgeable about the Encord SDK. If you are unfamiliar with the SDK or if you do not understand the following boilerplate code, refer to this topic in the SDK documentation.

Within Encord Active, predictions use the same format as labels. For the most part, creating predictions programmatically is the same as creating labels.

We'll start from the "Label Creation Boilerpate", which shows you how to upload labels into Encord. Then we'll show you how to modify the boilerplate to store predictions.

You can create labels programmatically in Encord following this structure:


# Import dependencies
import os

from encord import EncordUserClient, Project
from encord.objects import LabelRowV2

# Authenticate client and identify project
ssh_private_key_path = os.getenv("ENCORD_CLIENT_SSH_PATH")
project_hash = os.getenv("ENCORD_PROJECT_HASH")

assert ssh_private_key_path is not None
assert project_hash is not None

client = EncordUserClient.create_with_ssh_private_key(ssh_private_key_path)
project: Project = client.get_project(project_hash)

# Add labels
def add_information_to_lr(lr: LabelRowV2):
    ...  # Logic for adding labels/predictions

# Save labels
label_rows: list[LabelRowV2] = project.list_label_rows_v2()
for lr in label_rows:
    lr.initialise_labels()
    add_information_to_lr(lr)
    lr.save()

ℹ️

Note

The documentation to add labels to the label row is available here.

To store the labels as predictions instead, you need to change the following things in the Label Creation Boilerplate:

  • Initialize the label rows without the existing labels [see Storing Predictions Boilerplate line 27-30]
  • Store the predictions as serialized json [see Storing Predictions Boilerplate line 32-36]
  • Make add_information_to_lr use your model to "create labels" (remember that labels and predictions are equivalent in terms of structure) [see Storing Predictions Boilerplate line 19]
# Import dependencies
import os

from encord import EncordUserClient, Project
from encord.objects import LabelRowV2

# Authenticate client and identify project
ssh_private_key_path = os.getenv("ENCORD_CLIENT_SSH_PATH")
project_hash = os.getenv("ENCORD_PROJECT_HASH")

assert ssh_private_key_path is not None
assert project_hash is not None

client = EncordUserClient.create_with_ssh_private_key(ssh_private_key_path)
project: Project = client.get_project(project_hash)

# Make `add_information_to_lr` use your model to "create labels"
def add_information_to_lr(lr: LabelRowV2):
    ...  # Logic for adding labels/predictions


label_rows: list[LabelRowV2] = project.list_label_rows_v2()
serialized_output: list[dict] = []
for lr in label_rows:

# Initialize the label rows without the existing labels
    lr.initialise_labels(  # ignore existing labels
        include_object_feature_hashes=set(),
        include_classification_feature_hashes=set(),
    )
# Store the predictions as serialized json
    serialized_output.append(lr.to_encord_dict())  # Serialize

import json
with open("predictions.json", "w") as f:
    json.dump(serialized_output, f)

Now that you have the predictions.json file, you can move to STEP 2 and import the JSON file into the Active UI.

STEP 2: Import Predictions Set

Once you have the predictions.json file from STEP 1, Prediction Sets can be imported from both the Model Evaluation page and the Upload predictions button ( + ) on the Overview tab of the Predictions page in the Explorer page.

To import Prediction Sets into Active from the Model Evaluation page:
  1. Contact Encord to get started with Encord Active.

  2. Log in to the Encord platform.
    The landing page for the Encord platform appears.

  3. Click Active in the main menu.
    The landing page for Active appears.

  4. Click the Project.
    The landing page for the Project appears with the Explorer tab selected.

  5. Click the Model Evaluation tab.
    The Model Evaluation page appears.

    Import Model Predictions

  6. Click the Import prediction button.
    The Upload predictions dialog appears.

  7. Type a meaningful name for the prediction.

  8. Click the Select Predictions File button.
    A dialog box appears.

  9. Select the JSON file to upload.

  10. Click Open.

  11. Click Start Upload.
    Once the upload completes the Model Evaluation page and the Predictions page on the Explorer page are available for use.

ℹ️

Note

If you have any issues importing your predictions contact your CSM or contact us at [email protected].

Next Steps

Model and Prediction Validation

Clickable Div 1. Import from Annotate 3. Review Prediction Metrics 4. Create Collection 5. Send to Annotate 6. Sync with Annotate 7. Update Collection