Import Predictions

Encord Active does not only provide a streamlined method to currate your image data, Active also provides metrics and analytics to optimize your model's performance. Simply upload your model's predictions in to Active to start.

Your predictions must be imported to Active, before you can use the Predictions feature on the Explorer page and the Model Evaluation page.

STEP 1: Prepare Your Predictions for Import

❗️

CRITICAL INFORMATION

Disclaimer: We strongly recommend that you are knowledgeable about the Encord SDK. If you are unfamiliar with the SDK or if you do not understand the following boilerplate code, refer to this topic in the SDK documentation. The script below is incomplete and requires you to add the label_rows the model predictions you are importing. Always ensure you are using the latest version of the Encord SDK.

Within Encord Active, predictions use the same format as labels. For the most part, creating predictions programmatically is the same as creating labels.

ℹ️

Note

If you want to learn more about working with labels, the content is available here.

⚠️ DO NOT SAVE the Label Row or your labels are overwritten by your predictions.

# Import dependencies
import os
import json

from encord import EncordUserClient, Project
from encord.objects import LabelRowV2

# Authenticate client and identify project
ssh_private_key_path = os.getenv("ENCORD_CLIENT_SSH_PATH")
project_hash = os.getenv("ENCORD_PROJECT_HASH")

assert ssh_private_key_path is not None
assert project_hash is not None

client = EncordUserClient.create_with_ssh_private_key(ssh_private_key_path)

# Gets Project to add labels. This Project already exists in Encord.
project: Project = client.get_project(project_hash)

BATCH_SIZE = 100 # Batch size to split
label_rows = project.list_label_rows_v2()

# splits the label_rows into batches of size BATCH_SIZE
label_row_batches = [label_rows[i:i+BATCH_SIZE] for i in range(0, len(label_rows), BATCH_SIZE)]
serialized_output: list[dict] = []
for labels_batch in label_row_batches:

   bundle_init = project.create_bundle()
   for label_row in labels_batch:
      label_row.initialise_labels(bundle=bundle_init,# ignore existing labels
      include_object_feature_hashes=set(),
      include_classification_feature_hashes=set(),) 


   # Execute bundle, initialising all the labels at once
   bundle_init.execute()

   # Create label_row from model predictions. For example, import predictions.
   for label_row in labels_batch:
      pass
      '''
      Load in model predictions for each label_row here, and replace "pass". 
      ''' 

      # Add the label_row containing predictions as serialized json
      serialized_output.append(label_row.to_encord_dict())  # Serialize


with open("predictions.json", "w") as f:
 json.dump(serialized_output, f)

👍 Tip

Before importing your predictions.json file, we recommend that you compress your JSON file (basically removing the whitespace from the file). Compressing the file significantly reduces the file size by a factor of 10. You can use a tool like https://jqlang.github.io/jq/manual/ with the command jq -c '.' large_input.json > compressed_input.json where large_input.json is the name of your input file and compressed_input.json is the file you can use to import your predictions into Active.

STEP 2: Import Predictions Set

Once you have the predictions.json file from STEP 1, Prediction Sets can be imported from both the Model Evaluation page and the Upload predictions button ( + ) on the Overview tab of the Predictions page in the Explorer page.

❗️

CRITICAL INFORMATION

Encord Active supports importing Prediction Sets files of up to 100MB. If your Prediction Sets file exceeds 100MB, split the file into smaller files and then perform the import.

To import Prediction Sets into Active from the Model Evaluation page:
  1. Contact Encord to get started with Encord Active.

  2. Log in to the Encord platform.
    The landing page for the Encord platform appears.

  3. Click Active in the main menu.
    The landing page for Active appears.

  4. Click the Project.
    The landing page for the Project appears with the Explorer tab selected.

  5. Click the Model Evaluation tab.
    The Model Evaluation page appears.

    Import Model Predictions

  6. Click the Import prediction button.
    The Upload predictions dialog appears.

  7. Type a meaningful name for the prediction.

  8. Click the Select Predictions File button.
    A dialog box appears.

  9. Select the JSON file to upload.

  10. Click Open.

  11. Click Start Upload.
    Once the upload completes the Model Evaluation page and the Predictions page on the Explorer page are available for use.

ℹ️

Note

If you have any issues importing your predictions contact your CSM or contact us at [email protected].

Next Steps

Model and Prediction Validation

Clickable Div 1. Import from Annotate 2. Import Predictions 3. Prediction Metrics 4. Create Collection 5. Send to Annotate 6. Sync with Annotate 7. Update Collection