Data Curation, Management, and Annotation

STEP 1: Set Up your Org

Add Users

An overview of all your Organization’s users and user roles is found on the Users tab of your Organization.

User roles

Organizations have several kinds of users.

Internal: Users that your Organization directly employs. Can be either Member or Admin.
External: Users not directly employed, or contractually employed by your Organization. This includes external annotation teams.
Workforce: Users belonging to a Workforce Organization that are added to another Organization’s Project or Dataset.

Internal users can have the Member OR Admin role in your Organization. The following table outlines permissions of both internal user roles.

Admin	Member
Executive privileges over the Organization such as adding and removing users, and the ability to view all Projects in the Organization.	No administrative privileges over your Organization. Can only view Projects they create, or have been invited to.

In addition to having a user role within the Organization, all users have distinct roles in Projects, Datasets, and Ontologies.

Adding and removing users

Only Organization Admins have the ability to add users to, or remove users from, the Organization.

Users belonging to your Organization are managed on the Users tab of your Organization dashboard. The Users tab displays by default when you navigate to your Organization. All users belonging to your Organization are listed on the Users tab.

To add new users to your Organization:

Click the + Add user button. A dialog appears

Type the email addresses of the users you want to add.
Select the role you want the users to have.
Click Add to add the users to your Organization.

Add User Groups

User groups are collections of members that are grouped together, allowing them to be added to Projects, Datasets, and Ontologies collectively. User groups are managed on the Groups tab of your Organization’s dashboard.

Create user groups

To create a user group:

Navigate to the Groups tab of your Organization.
Click + Create group. A dialog appears.

Give your group a meaningful name and description.
Search for and select users to include in the group.
Click Add to add the selected users to the group. Users can be removed by clicking the delete icon next to the user.
Click Create group to create the user group.

Add Project Tags

Project tags serve as a labeling system that helps to categorize, group, and filter Projects within your Organization. Project tags are created and managed in the Project tags tab of the Organization’s dashboard.

Project tags are applied to Projects on the Project level, not from your Organization dashboard.

The Project tags tab is only visible to Organization Admins, and only Organization Admins can create Project tags.

Create Project tags

Project tags must be created before they can be added to a Project.

Click + New project tag on the Project tags tab.

Give the new Project tag a name.
Press Enter to create the tag.

All Project tag names must be unique.

STEP 2: Data Discoverability Strategy

Index is purpose built to accelerate the speed and ease with which you find the best data from your data lake. Using Index effectively requires some up front planning on your part before even touching the Encord platform. To get the quickest ROI from Index you need a Data Discoverability Strategy. This helps to curate your data in the most efficient manner. Index allows you to visually inspect your data, but if you have billions (yes billions with a B) of data units you cannot visually inspect every single data unit in your data lake. Index provides a number of ways to sort and filter your data. But to turn that lake of data into something more manageable at scale and speed, we want to focus on exactly the things that are critical for you. Building a Data Discoverability Strategy helps you achieve that.

Accelerator	Description
Key Frames	Video Only Imports frames of interest into Index. You can specify a sampling rate (default 1 FPS) and key frames for import into Index or you can import only key frames. How does this help? You ensure critical data imports to Index. This “pre-filters” your data so the data available from your videos is already of a high quality. You control the amount of frames imported into Index. This can significantly speed up how quickly video data imports. Sampling rate + key frames: Control how much of a video imports while ensuring critical frames are included. Key frames only: Only frames you deem critical import into Index. This can significantly reduce video import time into Index. What do I need to do? You DO NOT need to make a Metadata Schema to specify key frames when importing videos. Decide on the key frames you want for each video and specify those frames when importing the video. Specify the key frames when importing videos. Refer to the docs for AWS, GCP, Azure, and OTC to specify the key frames during video imports. When importing local data or if you want to specify key frame after you import your videos, go here.
Custom Metadata	Provides custom filtering criteria for ALL data that has custom metadata. How does this help? You are able to filter your your data on the criteria that is important to you and your use cases. - Want to filter based on your companies UUID for the data. No problem. Want to add transcription data (through custom metadata) and search or filter based on the transcription. Easy Peasy. Need to specify priority on the data and then filter on that priority. Custom metadata helps support that. Want to filter based on time and date stamps. You can. What do I need to do? Decide on the filtering criteria you need. You can always update this later. Create a metadata schema in Encord. Create a Folder in Index. While importing add custom metadata to each data unit you want to be able to filter using your criteria. Refer to the docs for AWS, GCP, Azure, and OTC. When importing local data or if you want to apply custom metadata after importing your data, go here.
Custom Embeddings	Provides visualization mechanism to find patterns and similarity in your data. How does this help? Embedding plots provide at a glance analysis for massive amounts of data. Data similarity search across your data. Natural language search across your data. What do I need to do? Decide on the embeddings you need. You can always update this later. Create a metadata schema in Encord. Create a Folder in Index. While importing add custom embeddings to each data unit you want to using custom embeddings with. Refer to the docs for AWS, GCP, Azure, and OTC. When importing local data or if you want to apply custom metadata after importing your data, go here.

Accelerator

Description

Key Frames

Video Only
Imports frames of interest into Index. You can specify a sampling rate (default 1 FPS) and key frames for import into Index or you can import only key frames.

How does this help?

You ensure critical data imports to Index. This “pre-filters” your data so the data available from your videos is already of a high quality. You control the amount of frames imported into Index. This can significantly speed up how quickly video data imports.

Sampling rate + key frames: Control how much of a video imports while ensuring critical frames are included.
Key frames only: Only frames you deem critical import into Index. This can significantly reduce video import time into Index.

What do I need to do?

You DO NOT need to make a Metadata Schema to specify key frames when importing videos.

Decide on the key frames you want for each video and specify those frames when importing the video.
Specify the key frames when importing videos. Refer to the docs for AWS, GCP, Azure, and OTC to specify the key frames during video imports. When importing local data or if you want to specify key frame after you import your videos, go here.

Custom Metadata

Provides custom filtering criteria for ALL data that has custom metadata.

How does this help?

You are able to filter your your data on the criteria that is important to you and your use cases. - Want to filter based on your companies UUID for the data. No problem.

Want to add transcription data (through custom metadata) and search or filter based on the transcription. Easy Peasy.
Need to specify priority on the data and then filter on that priority. Custom metadata helps support that.
Want to filter based on time and date stamps. You can.

What do I need to do?

Decide on the filtering criteria you need. You can always update this later.
Create a metadata schema in Encord.
Create a Folder in Index.
While importing add custom metadata to each data unit you want to be able to filter using your criteria. Refer to the docs for AWS, GCP, Azure, and OTC. When importing local data or if you want to apply custom metadata after importing your data, go here.

Custom Embeddings

Provides visualization mechanism to find patterns and similarity in your data.

How does this help?

Embedding plots provide at a glance analysis for massive amounts of data.
Data similarity search across your data.
Natural language search across your data.

What do I need to do?

Decide on the embeddings you need. You can always update this later.
Create a metadata schema in Encord.
Create a Folder in Index.
While importing add custom embeddings to each data unit you want to using custom embeddings with. Refer to the docs for AWS, GCP, Azure, and OTC. When importing local data or if you want to apply custom metadata after importing your data, go here.

STEP 3: Create a Cloud Integration

Select your cloud provider.

AWS Integration

GCP Integration

Azure Integration

OTC Integration

Wasabi Integration

Oracle Integration

Direct Access Integration

STEP 4: Create Metadata Schema

Based on your Data Discoverability Strategy, you need to create a metadata schema. The schema provides a method of organization for your custom metadata. Encord supports:

Scalers: Methods for filtering.
Enums: Methods with options for filtering.
Embeddings: Method for embedding plot visualization, similarity search, and natural language search.

Custom metadata

Custom metadata refers to any additional information you attach to files, allowing for better data curation and management based on your specific needs. It can include any details relevant to your workflow, helping you organize, filter, and retrieve data more efficiently. For example, for a video of a construction site, custom metadata could include fields like "site_location": "Algiers", "project_phase": "foundation", or "weather_conditions": "sunny". This enables more precise tracking and management of your data.

Before importing any files with custom metadata to Encord, we recommend that you import a metadata schema. Encord uses metadata schemas to validate custom metadata uploaded to Encord and to instruct Index and Active how to display your metadata.

Metadata schema table

Use add_scalar to add a scalar key to your metadata schema.

Scalar Key	Description	Display Benefits
boolean	Binary data type with values “true” or “false”.	Filtering by binary values
datetime	ISO 8601 formatted date and time.	Filtering by time and date
number	Numeric data type supporting float values.	Filtering by numeric values
uuid	Customer specified unique identifier for a data unit.	Filtering by customer specified unique identifier
varchar	Textual data type. Formally `string`. `string` can be used as an alias for `varchar`, but we STRONGLY RECOMMEND that you use `varchar`.	Filtering by string.
text	Text data with unlimited length (example: transcripts for audio). Formally `long_string`. `long_string` can be used as an alias for `text`, but we STRONGLY RECOMMEND that you use `text`.	Storing and filtering large amounts of text.

Use add_enum and add_enum_options to add an enum and enum options to your meta data schema.

Key	Description	Display Benefits
enum	Enumerated type with predefined set of values.	Facilitates categorical filtering and data validation

Use add_embedding to add an embedding to your metadata schema.

Key	Description	Display Benefits
embedding	512 dimension embeddings for Active, 1 to 4096 for Index.	Filtering by embeddings, similarity search, 2D scatter plot visualization (Coming Soon)

Incorrectly specifying a data type in the schema can cause errors when filtering your data in Index or Active. If you encounter errors while filtering, verify your schema is correct. If your schema has errors, correct the errors, re-import the schema, and then re-sync your Active Project.

Import your metadata schema to Encord

Verify your schema

After importing your schema to Encord we recommend that you verify that the import is successful. Run the following code to verify your metadata schema imported and that the schema is correct.

STEP 5: Create a Folder

You must create a folder in Index to store your files.

Navigate to Files under the Index heading in the Encord platform.
Click the + New folder button to create a new folder. A dialog to create a new folder appears.

Give the folder a meaningful name and description.
Click Create to create the folder. The folder is listed in Files.

STEP 6: Create JSON or CSV for import

To import files from cloud storage into Encord, you must create a JSON or CSV file specifying the files you want to upload.

Find helpful scripts for creating JSON and CSV files for the data upload process here.

All types of data (videos, images, image groups, image sequences, and DICOM) from a private cloud are added to a Dataset in the same way, by using a JSON or CSV file. The file includes links to all of the images, image groups, videos and DICOM files in your cloud storage.

For a list of supported file formats for each data type, go here

Encord supports file names up to 300 characters in length for any file or video for upload.

Create JSON file for Import

For detailed information about the JSON file format used for import go here.

The information provided about each of the following data types is designed to get you up and running as quickly as possible without going too deeply into the why or how. Look at the template for each data type, then the examples, and adjust the examples to suit your needs.

If skip_duplicate_urls is set to true, all object URLs that exactly match existing images/videos in the dataset are skipped.

AWS JSON

Videos

BEST PRACTICE: If you want to use Index or Active with your video data, we STRONGLY RECOMMEND using key frames, custom metadata, and custom embeddings. When specifying key frames set the sampling_rate to 0. This imports only the first frame and any key frames you specify in the video. This can significantly speed up the import of your data into Active and Index and help you to focus on only data you identify as critical.

The following table provides some guidance for the examples provided after the table.

Title	Description
Template	Provides the proper JSON format to import videos into Encord. This template provides examples from the most basic to the most complex.
Data	Imports videos into Encord. Why would I do this? You ONLY want to add labels and classifications to your data. You DO NOT want to use Index or Active.
Key Frames	Imports videos with an Encord title and specifies key frames (frames of interest) for Active and Index. Why would I do this? You ONLY want to see frames that you deem critical in Active and Index. You want to significantly improve the time to import videos into Active and Index. `config` is optional when specifying key frames for Active and Index: Specifying a `sampling_rate` of `0` only imports the first frame and all key frames of your video into Active and Index. `"config": { "sampling_rate": "<samples-per-second>", "keyframe_mode": "frame" or "seconds", },` If `config` is not specified, the `sampling_rate` is 1 frame per second, and the `keyframe_mode` is `frame`.
Custom Metadata	Imports videos with an Encord title, specifies key frames (frames of interest), and custom metadata for Active and Index. Why would I do this? Importing custom metadata allows you to filter your data in Active and Index to make it easier to find the data you want to focus on. This speeds up creating Collections and by extension Datasets. Specifying key frames means you ONLY want to see frames that you deem critical in Active and Index AND you want to significantly improve the time to import videos into Active and Index. `config` is optional when specifying key frames for Active and Index: Specifying a `sampling_rate` of `0` only imports the first frame and all key frames of your video into Active and Index. `"config": { "sampling_rate": "<samples-per-second>", "keyframe_mode": "frame" or "seconds", },` If `config` is not specified, the `sampling_rate` is 1 frame per second, and the `keyframe_mode` is `frame`.
Embeddings	Imports videos with an Encord title, specifies key frames (frames of interest), custom metadata, and custom embeddings for Active and Index. This example includes the following custom metadata types: boolean, varchar, datetime, uuid, number. Why would I do this? Importing custom embeddings allows you to use scatter plots to examine your data AND allows you to use similarity search and natural language searches. Active and Index support different embedding dimensions (Active: 512, Index: 1 to 4096). Importing custom metadata allows you to filter your data in Active and Index to make it easier to find the data you want to focus on. This speeds up creating Collections and by extension Datasets. Specifying key frames means you ONLY want to see frames that you deem critical in Active and Index AND you want to significantly improve the time to import videos into Active and Index. `config` is optional when specifying custom embeddings for Active and Index: Specifying a `sampling_rate` of `0` only imports the first frame and all key frames of your video into Active and Index. `"config": { "sampling_rate": "<samples-per-second>", "keyframe_mode": "frame" or "seconds", },` If `config` is not specified, the `sampling_rate` is 1 frame per second, and the `keyframe_mode` is `frame`. Refer to our documentation for more information about Index with Custom Metadata, Index with Custom Embeddings, Active with Custom Metadata and Active with Custom Embeddings.
Video Metadata	Imports videos with the videoMetadata flag. When the videoMetadata flag is present in the JSON file, we directly use the supplied metadata without performing any additional validation, and do not store the file on our servers. To guarantee accurate labels, it is crucial that the metadata you provide is accurate.

DICOM

For detailed information about the JSON file format used for import go here.

Each dicom_series element can contain one or more DICOM series.
Each series requires a title and at least one object URL, as shown in the example below.
If skip_duplicate_urls is set to true, all object URLs exactly matching existing DICOM files in the dataset will be skipped.

Custom metadata is distinct from patient metadata, which is included in the .dcm file and does not have to be specific during the upload to Encord.

The following is an example JSON for uploading three DICOM series belonging to a study. Each title and object URL correspond to individual DICOM series.

The first series contains only a single object URL, as it is composed of a single file.
The second series contains 3 object URLs, as it is composed of three separate files.
The third series contains 2 object URLs, as it is composed of two separate files.

Template

{
  "dicom_series": [
    {
      "title": "Series-1",
      "objectUrl_0": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/study1-series1-file.dcm"
    },
    {
      "title": "Series-2",
      "objectUrl_0": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/study1-series2-file1.dcm",
      "objectUrl_1": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/study1-series2-file2.dcm",
      "objectUrl_2": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/study1-series2-file3.dcm",
    },
      {
      "title": "Series-3",
      "objectUrl_0": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/study1-series3-file1.dcm",
      "objectUrl_1": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/study1-series3-file2.dcm",
    }
  ],
  "skip_duplicate_urls": true
}

NIfTI

The following is an example JSON file for uploading two NIfTI files to Encord.

Template

{
    "nifti": [
      {
        "title": "<file-1>",
        "objectUrl": "https://my-bucket/.../nifti-file1.nii"
      },
      {
        "title": "<file-2>",
        "objectUrl": "https://my-bucket/.../nifti-file2.nii.gz"
      }
    ],
    "skip_duplicate_urls": true
  }

You can upload multiple file types using a single JSON file. The example below shows 1 image, 2 videos, 2 image sequences, and 1 image group.

Multiple file types


{
  "images": [
    {
      "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/Image1.png"
    }
  ],
  "videos": [
    {
      "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/videos/Cooking.mp4"
    },
    {
      "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/videos/Oranges.mp4"
    }
  ],
  "image_groups": [
    {
      "title": "apple-samsung-light",
      "createVideo": true,
      "objectUrl_0": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/1+(32).jpg",
      "objectUrl_1": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/1+(33).jpg",
      "objectUrl_2": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/1+(34).jpg",
      "objectUrl_3": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/1+(35).jpg"
    },
    {
      "title": "apple-samsung-dark",
      "createVideo": true,
      "objectUrl_0": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/2+(32).jpg",
      "objectUrl_1": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/2+(33).jpg",
      "objectUrl_2": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/2+(34).jpg",
      "objectUrl_3": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/2+(35).jpg"
    }
  ],
  "image_groups": [
    {
      "title": "apple-ios-light",
      "createVideo": false,
      "objectUrl_0": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/3+(32).jpg",
      "objectUrl_1": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/3+(33).jpg"
    }
  ],
  "skip_duplicate_urls": true
}

GCP JSON

Videos

The following table provides some guidance for the examples provided after the table.

Title	Description
JSON for videos	Provides the proper JSON format to import videos into Encord. This template provides examples from the most basic to the most complex.
Data	Imports videos into Encord. Why would I do this? You ONLY want to add labels and classifications to your data. You DO NOT want to use Index or Active.
Key Frames	Imports videos with an Encord title and specifies key frames (frames of interest) for Active and Index. Why would I do this? You ONLY want to see frames that you deem critical in Active and Index. You want to significantly improve the time to import videos into Active and Index. `config` is optional when specifying key frames for Active and Index: Specifying a `sampling_rate` of `0` only imports the first frame and all key frames of your video into Active and Index. `"config": { "sampling_rate": "<samples-per-second>", "keyframe_mode": "frame" or "seconds", },` If `config` is not specified, the `sampling_rate` is 1 frame per second, and the `keyframe_mode` is `frame`.
Custom Metadata	Imports videos with an Encord title, specifies key frames (frames of interest), and custom metadata for Active and Index. Why would I do this? Importing custom metadata allows you to filter your data in Active and Index to make it easier to find the data you want to focus on. This speeds up creating Collections and by extension Datasets. Specifying key frames means you ONLY want to see frames that you deem critical in Active and Index AND you want to significantly improve the time to import videos into Active and Index. `config` is optional when specifying key frames for Active and Index: Specifying a `sampling_rate` of `0` only imports the first frame and all key frames of your video into Active and Index. `"config": { "sampling_rate": "<samples-per-second>", "keyframe_mode": "frame" or "seconds", },` If `config` is not specified, the `sampling_rate` is 1 frame per second, and the `keyframe_mode` is `frame`.
Embeddings	Imports videos with an Encord title, specifies key frames (frames of interest), custom metadata, and custom embeddings for Active and Index. This example includes the following custom metadata types: boolean, varchar, datetime, uuid, number. Why would I do this? Importing custom embeddings allows you to use scatter plots to examine your data AND allows you to use similarity search and natural language searches. Active and Index support different embedding dimensions (Active: 512, Index: 1 to 4096). Importing custom metadata allows you to filter your data in Active and Index to make it easier to find the data you want to focus on. This speeds up creating Collections and by extension Datasets. Specifying key frames means you ONLY want to see frames that you deem critical in Active and Index AND you want to significantly improve the time to import videos into Active and Index. `config` is optional when specifying custom embeddings for Active and Index: Specifying a `sampling_rate` of `0` only imports the first frame and all key frames of your video into Active and Index. `"config": { "sampling_rate": "<samples-per-second>", "keyframe_mode": "frame" or "seconds", },` If `config` is not specified, the `sampling_rate` is 1 frame per second, and the `keyframe_mode` is `frame`. Refer to our documentation for more information about Index with Custom Metadata, Index with Custom Embeddings, Active with Custom Metadata and Active with Custom Embeddings.
Video Metadata	Imports videos with the videoMetadata flag. When the videoMetadata flag is present in the JSON file, we directly use the supplied metadata without performing any additional validation, and do not store the file on our servers. To guarantee accurate labels, it is crucial that the metadata you provide is accurate.

DICOM

For detailed information about the JSON file format used for import go here.

Each dicom_series element can contain one or more DICOM series.
Each series requires a title and at least one object URL, as shown in the example below.
If skip_duplicate_urls is set to true, all object URLs exactly matching existing DICOM files in the dataset will be skipped.

Custom metadata is distinct from patient metadata, which is included in the .dcm file and does not have to be specific during the upload to Encord.

The following is an example JSON for uploading three DICOM series belonging to a study. Each title and object URL correspond to individual DICOM series.

The first series contains only a single object URL, as it is composed of a single file.
The second series contains 3 object URLs, as it is composed of three separate files.
The third series contains 2 object URLs, as it is composed of two separate files.

JSON for DICOM

{
  "dicom_series": [
    {
      "title": "Series-1",
      "objectUrl_0": "https://storage.cloud.google.com/encord-image-bucket/images/study1-series1-file.dcm"
    },
    {
      "title": "Series-2",
      "objectUrl_0": "https://storage.cloud.google.com/encord-image-bucket/images/study1-series2-file1.dcm",
      "objectUrl_1": "https://storage.cloud.google.com/encord-image-bucket/images/study1-series2-file2.dcm",
      "objectUrl_2": "https://storage.cloud.google.com/encord-image-bucket/images/study1-series2-file3.dcm",
    },
      {
      "title": "Series-3",
      "objectUrl_0": "https://storage.cloud.google.com/encord-image-bucket/images/study1-series3-file1.dcm",
      "objectUrl_1": "https://storage.cloud.google.com/encord-image-bucket/images/study1-series3-file2.dcm",
    }
  ],
  "skip_duplicate_urls": true
}

NIfTI

The following is an example JSON file for uploading two NIfTI files to Encord.

{
    "nifti": [
      {
        "title": "<file-1>",
        "objectUrl": "https://my-bucket/.../nifti-file1.nii"
      },
      {
        "title": "<file-2>",
        "objectUrl": "https://my-bucket/.../nifti-file2.nii.gz"
      }
    ],
    "skip_duplicate_urls": true
  }

You can upload multiple file types using a single JSON file. The example below shows 1 image, 2 videos, 2 image sequences, and 1 image group.

Multiple file types


{
  "images": [
    {
      "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/images/Image1.png"
    }
  ],
  "videos": [
    {
      "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/videos/Cooking.mp4"
    },
    {
      "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/videos/Oranges.mp4"
    }
  ],
  "image_groups": [
    {
      "title": "apple-samsung-light",
      "createVideo": true,
      "objectUrl_0": "https://storage.cloud.google.com/encord-image-bucket/images/1+(32).jpg",
      "objectUrl_1": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/1+(33).jpg",
      "objectUrl_2": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/1+(34).jpg",
      "objectUrl_3": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/1+(35).jpg"
    },
    {
      "title": "apple-samsung-dark",
      "createVideo": true,
      "objectUrl_0": "https://storage.cloud.google.com/encord-image-bucket/images/2+(32).jpg",
      "objectUrl_1": "https://storage.cloud.google.com/encord-image-bucket/images/2+(33).jpg",
      "objectUrl_2": "https://storage.cloud.google.com/encord-image-bucket/images/2+(34).jpg",
      "objectUrl_3": "https://storage.cloud.google.com/encord-image-bucket/images/2+(35).jpg"
    }
  ],
  "image_groups": [
    {
      "title": "apple-ios-light",
      "createVideo": false,
      "objectUrl_0": "https://storage.cloud.google.com/encord-image-bucket/images/3+(32).jpg",
      "objectUrl_1": "https://storage.cloud.google.com/encord-image-bucket/images/3+(33).jpg"
    }
  ],
  "skip_duplicate_urls": true
}

Azure JSON

Videos

The following table provides some guidance for the examples provided after the table.

Title	Description
Template	Provides the proper JSON format to import videos into Encord. This template provides examples from the most basic to the most complex.
Data	Imports videos into Encord. Why would I do this? You ONLY want to add labels and classifications to your data. You DO NOT want to use Index or Active.
Key Frames	Imports videos with an Encord title and specifies key frames (frames of interest) for Active and Index. Why would I do this? You ONLY want to see frames that you deem critical in Active and Index. You want to significantly improve the time to import videos into Active and Index. `config` is optional when specifying key frames for Active and Index: Specifying a `sampling_rate` of `0` only imports the first frame and all key frames of your video into Active and Index. `"config": { "sampling_rate": "<samples-per-second>", "keyframe_mode": "frame" or "seconds", },` If `config` is not specified, the `sampling_rate` is 1 frame per second, and the `keyframe_mode` is `frame`.
Custom Metadata	Imports videos with an Encord title, specifies key frames (frames of interest), and custom metadata for Active and Index. Why would I do this? Importing custom metadata allows you to filter your data in Active and Index to make it easier to find the data you want to focus on. This speeds up creating Collections and by extension Datasets. Specifying key frames means you ONLY want to see frames that you deem critical in Active and Index AND you want to significantly improve the time to import videos into Active and Index. `config` is optional when specifying key frames for Active and Index: Specifying a `sampling_rate` of `0` only imports the first frame and all key frames of your video into Active and Index. `"config": { "sampling_rate": "<samples-per-second>", "keyframe_mode": "frame" or "seconds", },` If `config` is not specified, the `sampling_rate` is 1 frame per second, and the `keyframe_mode` is `frame`.
Embeddings	Imports videos with an Encord title, specifies key frames (frames of interest), custom metadata, and custom embeddings for Active and Index. This example includes the following custom metadata types: boolean, varchar, datetime, uuid, number. Why would I do this? Importing custom embeddings allows you to use scatter plots to examine your data AND allows you to use similarity search and natural language searches. Active and Index support different embedding dimensions (Active: 512, Index: 1 to 4096). Importing custom metadata allows you to filter your data in Active and Index to make it easier to find the data you want to focus on. This speeds up creating Collections and by extension Datasets. Specifying key frames means you ONLY want to see frames that you deem critical in Active and Index AND you want to significantly improve the time to import videos into Active and Index. `config` is optional when specifying custom embeddings for Active and Index: Specifying a `sampling_rate` of `0` only imports the first frame and all key frames of your video into Active and Index. `"config": { "sampling_rate": "<samples-per-second>", "keyframe_mode": "frame" or "seconds", },` If `config` is not specified, the `sampling_rate` is 1 frame per second, and the `keyframe_mode` is `frame`. Refer to our documentation for more information about Index with Custom Metadata, Index with Custom Embeddings, Active with Custom Metadata and Active with Custom Embeddings.
Video Metadata	Imports videos with the videoMetadata flag. When the videoMetadata flag is present in the JSON file, we directly use the supplied metadata without performing any additional validation, and do not store the file on our servers. To guarantee accurate labels, it is crucial that the metadata you provide is accurate.

DICOM

For detailed information about the JSON file format used for import go here.

Each dicom_series element can contain one or more DICOM series.
Each series requires a title and at least one object URL, as shown in the example below.
If skip_duplicate_urls is set to true, all object URLs exactly matching existing DICOM files in the dataset will be skipped.

Custom metadata is distinct from patient metadata, which is included in the .dcm file and does not have to be specific during the upload to Encord.

The following is an example JSON for uploading three DICOM series belonging to a study. Each title and object URL correspond to individual DICOM series.

The first series contains only a single object URL, as it is composed of a single file.
The second series contains 3 object URLs, as it is composed of three separate files.
The third series contains 2 object URLs, as it is composed of two separate files.

Template

{
  "dicom_series": [
    {
      "title": "Series-1",
      "objectUrl_0": "https://myaccount.blob.core.windows.net/encordcontainer/study1-series1-file.dcm"
    },
    {
      "title": "Series-2",
      "objectUrl_0": "https://myaccount.blob.core.windows.net/encordcontainer/study1-series2-file1.dcm",
      "objectUrl_1": "https://myaccount.blob.core.windows.net/encordcontainer/study1-series2-file2.dcm",
      "objectUrl_2": "https://myaccount.blob.core.windows.net/encordcontainer/study1-series2-file3.dcm",
    },
      {
      "title": "Series-3",
      "objectUrl_0": "https://myaccount.blob.core.windows.net/encordcontainer/study1-series3-file1.dcm",
      "objectUrl_1": "https://myaccount.blob.core.windows.net/encordcontainer/study1-series3-file2.dcm",
    }
  ],
  "skip_duplicate_urls": true
}

NIfTI

The following is an example JSON file for uploading two NIfTI files to Encord.

Template

{
    "nifti": [
      {
        "title": "<file-1>",
        "objectUrl": "https://my-bucket/.../nifti-file1.nii"
      },
      {
        "title": "<file-2>",
        "objectUrl": "https://my-bucket/.../nifti-file2.nii.gz"
      }
    ],
    "skip_duplicate_urls": true
  }

You can upload multiple file types using a single JSON file. The example below shows 1 image, 2 videos, 2 image sequences, and 1 image group.


{
  "images": [
    {
      "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/Image1.png"
    }
  ],
  "videos": [
    {
      "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/Cooking.mp4"
    },
    {
      "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/Oranges.mp4"
    }
  ],
  "image_groups": [
    {
      "title": "apple-samsung-light",
      "createVideo": true,
      "objectUrl_0": "https://myaccount.blob.core.windows.net/encordcontainer/1-Samsung-S4-Light+Environment/1+(32).jpg",
      "objectUrl_1": "https://myaccount.blob.core.windows.net/encordcontainer/1-Samsung-S4-Light+Environment/1+(33).jpg",
      "objectUrl_2": "https://myaccount.blob.core.windows.net/encordcontainer/1-Samsung-S4-Light+Environment/1+(34).jpg",
      "objectUrl_3": "https://myaccount.blob.core.windows.net/encordcontainer/1-Samsung-S4-Light+Environment/1+(35).jpg"
    },
    {
      "title": "apple-samsung-dark",
      "createVideo": true,
      "objectUrl_0": "https://myaccount.blob.core.windows.net/encordcontainer/2-samsung-S4-Dark+Environment/2+(32).jpg",
      "objectUrl_1": "https://myaccount.blob.core.windows.net/encordcontainer/2-samsung-S4-Dark+Environment/2+(33).jpg",
      "objectUrl_2": "https://myaccount.blob.core.windows.net/encordcontainer/2-samsung-S4-Dark+Environment/2+(34).jpg",
      "objectUrl_3": "https://myaccount.blob.core.windows.net/encordcontainer/2-samsung-S4-Dark+Environment/2+(35).jpg"
    }
  ],
  "image_groups": [
    {
      "title": "apple-ios-light",
      "createVideo": false,
      "objectUrl_0": "https://myaccount.blob.core.windows.net/encordcontainer/3-IOS-4-Light+Environment/3+(32).jpg",
      "objectUrl_1": "https://myaccount.blob.core.windows.net/encordcontainer/3-IOS-4-Light+Environment/3+(33).jpg"
    }
  ],
  "skip_duplicate_urls": true
}

OTC JSON

Videos

The following table provides some guidance for the examples provided after the table.

Title	Description
Template	Provides the proper JSON format to import videos into Encord. This template provides examples from the most basic to the most complex.
Data	Imports videos into Encord. Why would I do this? You ONLY want to add labels and classifications to your data. You DO NOT want to use Index or Active.
Key Frames	Imports videos with an Encord title and specifies key frames (frames of interest) for Active and Index. Why would I do this? You ONLY want to see frames that you deem critical in Active and Index. You want to significantly improve the time to import videos into Active and Index. `config` is optional when specifying key frames for Active and Index: Specifying a `sampling_rate` of `0` only imports the first frame and all key frames of your video into Active and Index. `"config": { "sampling_rate": "<samples-per-second>", "keyframe_mode": "frame" or "seconds", },` If `config` is not specified, the `sampling_rate` is 1 frame per second, and the `keyframe_mode` is `frame`.
Custom Metadata	Imports videos with an Encord title, specifies key frames (frames of interest), and custom metadata for Active and Index. Why would I do this? Importing custom metadata allows you to filter your data in Active and Index to make it easier to find the data you want to focus on. This speeds up creating Collections and by extension Datasets. Specifying key frames means you ONLY want to see frames that you deem critical in Active and Index AND you want to significantly improve the time to import videos into Active and Index. `config` is optional when specifying key frames for Active and Index: Specifying a `sampling_rate` of `0` only imports the first frame and all key frames of your video into Active and Index. `"config": { "sampling_rate": "<samples-per-second>", "keyframe_mode": "frame" or "seconds", },` If `config` is not specified, the `sampling_rate` is 1 frame per second, and the `keyframe_mode` is `frame`.
Embeddings	Imports videos with an Encord title, specifies key frames (frames of interest), custom metadata, and custom embeddings for Active and Index. This example includes the following custom metadata types: boolean, varchar, datetime, uuid, number. Why would I do this? Importing custom embeddings allows you to use scatter plots to examine your data AND allows you to use similarity search and natural language searches. Active and Index support different embedding dimensions (Active: 512, Index: 1 to 4096). Importing custom metadata allows you to filter your data in Active and Index to make it easier to find the data you want to focus on. This speeds up creating Collections and by extension Datasets. Specifying key frames means you ONLY want to see frames that you deem critical in Active and Index AND you want to significantly improve the time to import videos into Active and Index. `config` is optional when specifying custom embeddings for Active and Index: Specifying a `sampling_rate` of `0` only imports the first frame and all key frames of your video into Active and Index. `"config": { "sampling_rate": "<samples-per-second>", "keyframe_mode": "frame" or "seconds", },` If `config` is not specified, the `sampling_rate` is 1 frame per second, and the `keyframe_mode` is `frame`. Refer to our documentation for more information about Index with Custom Metadata, Index with Custom Embeddings, Active with Custom Metadata and Active with Custom Embeddings.
Video Metadata	Imports videos with the videoMetadata flag. When the videoMetadata flag is present in the JSON file, we directly use the supplied metadata without performing any additional validation, and do not store the file on our servers. To guarantee accurate labels, it is crucial that the metadata you provide is accurate.

DICOM

For detailed information about the JSON file format used for import go here.

Each dicom_series element can contain one or more DICOM series.
Each series requires a title and at least one object URL, as shown in the example below.
If skip_duplicate_urls is set to true, all object URLs exactly matching existing DICOM files in the dataset will be skipped.

Custom metadata is distinct from patient metadata, which is included in the .dcm file and does not have to be specific during the upload to Encord.

The following is an example JSON for uploading three DICOM series belonging to a study. Each title and object URL correspond to individual DICOM series.

The first series contains only a single object URL, as it is composed of a single file.
The second series contains 3 object URLs, as it is composed of three separate files.
The third series contains 2 object URLs, as it is composed of two separate files.

JSON for DICOM

{
  "dicom_series": [
    {
      "title": "Series-1",
      "objectUrl_0": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/study1-series1-file.dcm"
    },
    {
      "title": "Series-2",
      "objectUrl_0": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/study1-series2-file1.dcm",
      "objectUrl_1": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/study1-series2-file2.dcm",
      "objectUrl_2": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/study1-series2-file3.dcm",
    },
      {
      "title": "Series-3",
      "objectUrl_0": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/study1-series3-file1.dcm",
      "objectUrl_1": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/study1-series3-file2.dcm",
    }
  ],
  "skip_duplicate_urls": true
}

NIfTI

The following is an example JSON file for uploading two NIfTI files to Encord.

{
    "nifti": [
      {
        "title": "<file-1>",
        "objectUrl": "https://my-bucket/.../nifti-file1.nii"
      },
      {
        "title": "<file-2>",
        "objectUrl": "https://my-bucket/.../nifti-file2.nii.gz"
      }
    ],
    "skip_duplicate_urls": true
  }

You can upload multiple file types using a single JSON file. The example below shows 1 image, 2 videos, 2 image sequences, and 1 image group.

Multiple file types


{
  "images": [
    {
      "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/Image1.png"
    }
  ],
  "videos": [
    {
      "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/videos/Cooking.mp4"
    },
    {
      "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/videos/Oranges.mp4"
    }
  ],
  "image_groups": [
    {
      "title": "apple-samsung-light",
      "createVideo": true,
      "objectUrl_0": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/1+(32).jpg",
      "objectUrl_1": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/1+(33).jpg",
      "objectUrl_2": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/1+(34).jpg",
      "objectUrl_3": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/1+(35).jpg"
    },
    {
      "title": "apple-samsung-dark",
      "createVideo": true,
      "objectUrl_0": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/2+(32).jpg",
      "objectUrl_1": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/2+(33).jpg",
      "objectUrl_2": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/2+(34).jpg",
      "objectUrl_3": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/2+(35).jpg"
    }
  ],
  "image_groups": [
    {
      "title": "apple-ios-light",
      "createVideo": false,
      "objectUrl_0": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/3+(32).jpg",
      "objectUrl_1": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/3+(33).jpg"
    }
  ],
  "skip_duplicate_urls": true
}

Use a Multi-Region Access Point

When using a Multi-Region Access Point for your AWS S3 buckets the JSON file has to be slightly different from the examples provided. Instead of an object’s URL, objects are specified using the ARN of the Multi-Region Access Point followed by the object name. The example below shows how video files from a Multi-Region Access Point would be specified.

{
  "videos": [
    {
      "objectUrl": "Multi-Region-Access-Point-ARN + <object name_1>"
    },
    {
      "objectUrl": "Multi-Region-Access-Point-ARN + <object name_2>",
      "title": "my-custom-video-title.mp4",
      "clientMetadata": {"optional": "metadata"}
    }
  ],
  "skip_duplicate_urls": true
}

Create CSV file for import

In the CSV file format, the column headers specify which type of data is being uploaded. You can add and single file format at a time, or combine multiple data types in a single CSV file.

Details for each data format are given in the sections below.

Encord supports up to 10,000 entries for upload in the CSV file.

Object URLs can’t contain whitespace.
For backwards compatibility reasons, a single column CSV is supported. A file with the single ObjectUrl column is interpreted as a request for video upload. If your objects are of a different type (for example, images), this error displays: “Expected a video, got a file of type XXX”.

Videos

A CSV file containing videos should contain two columns with the following mandatory column headings:
‘ObjectURL’ and ‘Video title’. All headings are case-insensitive.

The ‘ObjectURL’ column containing the objectUrl. This field is mandatory for each file, as it specifies the full URL of the video resource.
The ‘Video title’ column containing the video_title. If left blank, the original file name is used.

In the example below files 1, 2 and 4 will be assigned the names in the title column, while file 3 will keep its original file name.

ObjectUrl	Video title
path/to/storage-location/frame1.mp4	Video 1
path/to/storage-location/frame2.mp4	Video 2
path/to/storage-location/frame3.mp4
path/to/storage-location/frame4.mp4	Video 3

A CSV file containing single images should contain two columns with the following mandatory headings:
‘ObjectURL’ and ‘Image title’. All headings are case-insensitive.

The ‘ObjectURL’ column containing the objectUrl. This field is mandatory for each file, as it specifies the full URL of the image resource.
The ‘Image title’ column containing the image_title. If left blank, the original file name is used.

In the example below files 1, 2 and 4 will be assigned the names in the title column, while file 3 will keep its original file name.

ObjectUrl	Image title
path/to/storage-location/frame1.jpg	Image 1
path/to/storage-location/frame2.jpg	Image 2
path/to/storage-location/frame3.jpg
path/to/storage-location/frame4.jpg	Image 3

Image groups

A CSV file containing image groups should contain three columns with the following mandatory headings:
‘ObjectURL’, ‘Image group title’, and ‘Create video’. All three headings are case-insensitive.

The ‘ObjectURL’ column containing the objectUrl. This field is mandatory for each file, as it specifies the full URL of the resource.
The ‘Image group title’ column containing the image_group_title. This field is mandatory, as it determines which image group a file will be assigned to.

In the example below the first two URLs are grouped together into ‘Group 1’, while the following two files are grouped together into ‘Group 2’.

ObjectUrl	Image group title	Create video
path/to/storage-location/frame1.jpg	Group 1	false
path/to/storage-location/frame2.jpg	Group 1	false
path/to/storage-location/frame3.jpg	Group 2	false
path/to/storage-location/frame4.jpg	Group 2	false

Image groups do not require ‘write’ permissions.

Image sequences

A CSV file containing image sequences should contain three columns with the following mandatory headings: ‘ObjectURL’, ‘Image group title’, and ‘Create video’. All three headings are case-insensitive.

The ‘ObjectURL’ column containing the objectUrl. This field is mandatory for each file, as it specifies the full URL of the resource.
The ‘Image group title’ column containing the image_group_title. This field is mandatory, as it determines which image sequence a file will be assigned to. The dimensions of the image sequence are determined by the first file in the sequence.
The ‘Create video’ column. This can be left blank, as the default value is ‘true’.

In the example below the first two URLs are grouped together into ‘Sequence 1’, while the second two files are grouped together into ‘Sequence 2’.

ObjectUrl	Image group title	Create video
path/to/storage-location/frame1.jpg	Sequence 1	true
path/to/storage-location/frame2.jpg	Sequence 1	true
path/to/storage-location/frame3.jpg	Sequence 2	true
path/to/storage-location/frame4.jpg	Sequence 2	true

Image groups and image sequences are only distinguished by the presence of the ‘Create video’ column.

Image sequences require ‘write’ permissions against your storage bucket to save the compressed video.

A CSV file containing DICOM files should contain two columns with the following mandatory headings: ‘ObjectURL’ and ‘Dicom title’. Both headings are case-insensitive.

The ‘ObjectURL’ column containing the objectUrl. This field is mandatory for each file, as it specifies the full URL of the resource.
The ‘Series title’ column containing the dicom_title. When two files are given the same title they are grouped into the same DICOM series. If left blank, the original file name is used.

In the example below the first two files are grouped into ‘dicom series 1’, the next two files are grouped into ‘dicom series 2’, while the final file will remain separated as ‘dicom series 3’.

ObjectUrl	Series title
path/to/storage-location/frame1.dcm	dicom series 1
path/to/storage-location/frame2.dcm	dicom series 1
path/to/storage-location/frame3.dcm	dicom series 2
path/to/storage-location/frame4.dcm	dicom series 2
path/to/storage-location/frame5.dcm	dicom series 3

Multiple file types

You can upload multiple file types with a single CSV file by using a new header each time there is a change of file type. Three headings will be required if image sequences are included.

Since the ‘Create video’ column defaults to true all files that are not image sequences must contain the value false

The example below shows a CSV file for the following:

Two image sequences composed of 2 files each.
One image group composed of 2 files.
One single image.
One video.

ObjectUrl	Image group title	Create video
path/to/storage-location/frame1.jpg	Sequence 1	true
path/to/storage-location/frame2.jpg	Sequence 1	true
path/to/storage-location/frame3.jpg	Sequence 2	true
path/to/storage-location/frame4.jpg	Sequence 2	true
path/to/storage-location/frame5.jpg	Group 1	false
path/to/storage-location/frame6.jpg	Group 1	false
ObjectUrl	Image title	Create video
path/to/storage-location/frame1.jpg	Image 1	false
ObjectUrl	Image title	Create video
full/storage/path/video.mp4	Video 1	false

STEP 7: Import your data

STEP 8: Create a collection

A Collection is a container for data units (images or videos) that you can use to group your data units together.

Creation of a Collection involves filtering and sorting your data. Once you have selected a smaller group of images, videos or audio files, create a Collection.

Log in to the Encord platform. The landing page for the Encord platform appears.
Go to Index > Files. The All folders page appears with a list of all folders in Encord.
Click in to a Folder. The landing page for the Folder appears and the Explorer button is enabled.
Click the Explorer button. The Index Explorer page appears.

Search, sort, and filter your data until you have the subset of the data you need.

Select one or more of the images/frames in the Explorer workspace. A ribbon appears at the top of the Explorer workspace.

Selecting a video frame selects the entire video. Specific frames from a video cannot be selected.
Click Select all to select all the images in the subset.
Click Add to a Collection.

Click New Collection.
Specify a meaningful title and description for the Collection.

The title specified here is applied as a tag/label to every selected image.
Click Collections to verify the Collection appears in the Collections list.

STEP 9: Create a Dataset from a Collection

Once you have a Collection, you can create a Dataset from your Collection.

STEP 10: Create an Ontology

An Ontology is a structured framework that defines the categories, labels, and relationships used to annotate data consistently and accurately. Ontologies define what you want labelled.

We recommend learning about Ontology structure in Encord before creating Ontologies.

Click the New ontology button in the Ontologies section to create a new Ontology.

Give your Ontology a meaningful title and description. A clear title and description keeps your Ontologies organized. Click Next to continue.

Define your Ontology structure. See our documentation on Ontology structure for more information on the various types of objects, classifications, and attributes.

To add objects:

Click Add object to create a new object.
Give the object a name. For example “Apple”.
Select a shape for the object. For example polygon.
Optionally, enable the Required toggle to mark the object as Required.
Optionally, add attributes to the object.
Repeat these steps for as many objects as necessary.

To add attributes to an object:

You can add attributes to objects that define the object’s characteristics. For example the object “Apple” can have an attribute “Color”.

Click the arrow icon next to an object to add attributes to the object.
Give the attribute a name. For example “Color”.
Click the attribute type to change the attribute type. The default attribute type is a text field.
Click Add option to add an option, if you have chosen a radio button or checklist attribute.
Enter a name for the attribute option. For example, the attribute “Color” can have the options “Red”, “Green”, and “Yellow”.
Click the Back to parent button to return the Ontology creation view.

To add a classification:

Click Add classification to create a new classification.
Give the classification a name. For example, “Time of day”.
Optionally, configure the classification. The default classification type is a text field.
Optionally, enable the Required toggle to mark the object as Required.
Repeat these steps for as many classifications as necessary.

Configure classifications

You can configure classifications to change the classification type, and to add classification options to radio buttons and check lists.

Click the arrow icon next to an object to configure the classification.

Click the classification type to change the classification type. The default classification type is a text field.
Click Add option to add an option if you have selected a radio button or check list classification.
Enter a name for the classification option. For example, the classification “Time of day” can have the options “Night” and “Day”.
Click the Back to parent button to return the Ontology creation view.

STEP 11: Create Task Agent

Task agents enable you to set up custom actions like pre-labeling, leveraging foundation models such as GPT-4, automated quality assurance, or other tailored actions to suit your workflow.

Use the Encord SDK to configure your Task Agent. The Task Agent executes the configured SDK script for all tasks that are routed through the Task Agent stage in your Workflow.

The General Example script shows how to configure a Task Agent with the name Agent 1 and with a pathway called continue to Review.

Agent nodes can be used in conjunction with bundles to efficiently handle bulk actions

Ensure that you:

Replace <private_key_path> with the path to your private key.
Replace <project_hash> with the hash of your Project.
Insert your custom logic where the comment instructs you to do so.

General Example

# Import dependencies
from encord.user_client import EncordUserClient
from encord.workflow import AgentStage

# Authenticate using the path to your private key
user_client = EncordUserClient.create_with_ssh_private_key(
ssh_private_key_path="<private_key_path>"
)

# Specify the Project that contains the Task agent. Replace <project_hash> with the hash of your Project
project = user_client.get_project(<project_hash>)

# Specify the Task Agent
agent_stage = project.workflow.get_stage(name="Agent 1", type_=AgentStage)
for task in agent_stage.get_tasks():

    # Now you have the agent task containing the data hash
    # Insert you custom logic here

# When the custom logic is completed, the task can be moved forward to the selected pathway
task.proceed(pathway_name="continue to Review")

To set up a pre-labeling stage using models:

Train a model on the objects you want pre-labeled.
Set up a Workflow that includes a prelabeling Agent stage.

Create a new Project with the Workflow from Step 2, and attach a Dataset containing the images you want to pre-label.
Add the following script to the pre-labeling Agent stage. The script makes the following assumptions:
- You want to apply polygon labels.
- You only have one model and only one training iteration for that model.

Agent nodes can be used in conjunction with bundles to efficiently handle bulk actions

In the following script:

Replace <private_key_path> with the hash of your private key.
Replace <project_hash> with the hash of your Project.
Replace Agent 1 with the name of your Agent stage.

Pre-Labeling using models

# Import dependencies
from encord.user_client import EncordUserClient
from encord.workflow import AgentStage

# Authenticate using the path to your private key
user_client = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path="<private_key_path>"
)

# Specify the Project that contains the Task agent. Replace <project_hash> with the hash of your Project
project = user_client.get_project(<project_hash>)
model = project.list_models[0]

# Specify the Task Agent
prelabeling_stage = project.workflow.get_stage(name="Agent 1", type_=AgentStage)
for task in prelabeling_stage.get_tasks():
    print(f"{task.data_hash} -> {task.data_title}")

    # Getting a label row for the data unit
    lr = project.list_label_rows_v2(data_hashes=[task.data_hash])[0]
    lr.initialise_labels()

    # Infer the first training iteration of your model
    inference_results = project.model_inference(
        uid=str(
            model.model_iteration_uids[0]   # Take only the first result
        ), 
        data_hashes=[str(task.data_hash)],  # Data hashes
        detection_frame_range=[0, 0],
    )

    ontology_structure = project.ontology_structure
    for inference_result in inference_results:
        for frame, predictions in inference_result.get("predictions").items():
            for pred in predictions.get("objects"):
                ontology_object = ontology_structure.get_child_by_title(pred.get("name"))
                object_instance = ontology_object.create_instance()
                object_instance.set_for_frames(
                    coordinates=PolygonCoordinates.from_dict(pred),
                    frames=int(frame),
                    manual_annotation=False,
                    confidence=pred.get("confidence"),
                    # Add other attributes as needed
                )
                lr.add_object_instance(object_instance)
    lr.save()
    task.proceed(pathway_name="Pre-Labeled")

Triggering the Task Agent

Webhooks are coming soon, allowing you to trigger Task Agents manually.

Task Agents aggregate all tasks that reach the Agent stage in the workflow. Your custom script must be triggered at this stage before the tasks proceed further in the workflow.

STEP 12: Create a Workflow template

Workflows allow you to design and manage the flow of tasks through various stages of a Project. You have control over how tasks progress and how different stages interact.

To set up a Workflow template, navigate to the Annotate section of the Encord platform, and click the + New workflow button

1. Add users to the Workflow:

Add users from your Organization to the Workflow by clicking Invite collaborators.

Collaborators are added based on their role within the project - select the role you would like the collaborator(s) to have.
Start typing the email of a user you would like to add into the area highlighted on the image below, and select the user from the list that appears. Repeat this for every user that will have the same role.
When you are done selecting users for this role, click Add.

Learn more about managing collaborators in the Managing collaborators section.

2. Configure your Workflow:

The canvas is populated with a simple Workflow by default.

Click the Add stage button to show all Workflow components.

Customize your Workflow by pulling components onto the canvas.

Add the stages and other components you require for your Workflow onto the canvas:

All workflows must begin with the Start stage.
All workflows must contain an Annotate stage.
Add routers to your Project to determine different pathways through your Workflow a task can take.
Add as many Review stages as necessary.
All workflows must end at a Complete stage.
Link all components on the canvas by clicking and dragging from one connection point to another.

All Workflows can be saved as a template by clicking the Save as a new template button.

3. Configure the stages of your Workflow:

After you arranged the stages in the composer, it is time time to configure the details of each stage.

Click an Annotate card on the canvas to start editing the annotation stage.
- Give the stage a descriptive name.
- Add annotators. If you’d like to specify annotators for this stage, add them as collaborators. For full details on how collaborators can work on tasks at each stage, see our section on managing collaborators.
- Optionally, add a Webhook to receive notifications when labels are submitted at this stage.

More details about configuring annotation stages can be found in the Annotate section below

Click a Review card on the canvas to start editing the review stage.
- Give the stage a descriptive name.
- Add reviewers. If you want to specify reviewers for this stage, add them as collaborators. For full details on how collaborators can work on tasks at each stage, see our section on managing collaborators.

More details about configuring review stages can be found in the Review section below.

Click a Router card on your canvas. See the router section for more details on how different types of routers can be configured.
Optionally, add a Webhook to the Complete stage to receive a notification when a task has been completed.
Optionally, add User assignment restrictions if users in this node should be prevented from being assigned to tasks they completed in the nodes listed.

More details on routers can be found in the router section below.

STEP 13: Create a Project

Projects in Encord bring together Datasets, Ontologies, and Workflows. Datasets are labeled according to the Ontology, while the Workflow defines how tasks progress through the Project from start to finish.

In the Encord platform, select Projects under Annotate.
Click the + New annotation project button to create a new Project.

Give the Project a meaningful title and description.

If you are part of an Organization, an optional Project tags drop-down is visible. Project tags are useful for categorizing and finding your Projects. Select as many tags as are relevant for your Project.

Click the Attach ontology button.
Select an Ontology from the list using the Select button, or create a new Ontology by clicking the New ontology button.

Click OK to attach the Ontology to the Project.
Click the Attach datasets button.
Select a Dataset from the list using the Attach button, or create a new Dataset by clicking the New Dataset button.

Click OK to attach the Dataset(s) to the Project.

Click the Load from template button to use a Workflow template.

Select the template you want to use and click Load template.
Click Create project to finish creating the Project.

Add users to the Project

After creating a Project you must invite users to act as annotators, reviewers, team managers, and admins. Collaborators can be add individuals, or in user groups.

STEP 14: Label your Data

Now you are ready to label your data. We recommend you and your team watch these introductory videos.

For more details, see our comprehensive tutorial on labeling here.

STEP 15: Export your Labels

To train your models you must export your labels.

Only Project admins can export labels

Descriptions of all information contained in the export file can be found here.

Navigate to Labels and export on the Project Dashboard
Select the Data tab.
Select the data units you want to export labels for.
Click the Export and save button. A pop-up appears.

Give this label version a name.
Select whether you want your labels to be exported in JSON or COCO format.
Toggle this if you want to include signed URLs in your export.
Select which label status(es) to include in the export.
Select what objects to include in the export.
Click Export and save to export your labels.

Was this page helpful?

Overview of Encord Annotate Datasets

On this page

STEP 1: Set Up your Org
Add Users
User roles
Adding and removing users
Add User Groups
Create user groups
Add Project Tags
Create Project tags
STEP 2: Data Discoverability Strategy
STEP 3: Create a Cloud Integration
STEP 4: Create Metadata Schema
Custom metadata
Metadata schema table
Import your metadata schema to Encord
Verify your schema
STEP 5: Create a Folder
STEP 6: Create JSON or CSV for import
STEP 7: Import your data
STEP 8: Create a collection
STEP 9: Create a Dataset from a Collection
STEP 10: Create an Ontology
To add objects:
To add attributes to an object:
To add a classification:
Configure classifications
STEP 11: Create Task Agent
Triggering the Task Agent
STEP 12: Create a Workflow template
STEP 13: Create a Project
Add users to the Project
STEP 14: Label your Data
STEP 15: Export your Labels

Get started

General

Index

Annotate

Active

Other

​STEP 1: Set Up your Org

​Add Users

​User roles

​Adding and removing users

​Add User Groups

​Create user groups

​Add Project Tags

​Create Project tags

​STEP 2: Data Discoverability Strategy

​STEP 3: Create a Cloud Integration

​STEP 4: Create Metadata Schema

​Custom metadata

​Metadata schema table

​Import your metadata schema to Encord

​Verify your schema

​STEP 5: Create a Folder

​STEP 6: Create JSON or CSV for import

Create JSON file for Import

AWS JSON

GCP JSON

Azure JSON

OTC JSON

Use a Multi-Region Access Point

Create CSV file for import

Videos

Image groups

Image sequences

Multiple file types

​STEP 7: Import your data

Import Cloud Data

Import Local Data

​STEP 8: Create a collection

​STEP 9: Create a Dataset from a Collection

​STEP 10: Create an Ontology

​To add objects:

​To add attributes to an object:

​To add a classification:

​Configure classifications

​STEP 11: Create Task Agent

​Triggering the Task Agent

​STEP 12: Create a Workflow template

​STEP 13: Create a Project

​Add users to the Project

​STEP 14: Label your Data

​STEP 15: Export your Labels

STEP 1: Set Up your Org

Add Users

User roles

Adding and removing users

Add User Groups

Create user groups

Add Project Tags

Create Project tags

STEP 2: Data Discoverability Strategy

STEP 3: Create a Cloud Integration

STEP 4: Create Metadata Schema

Custom metadata

Metadata schema table

Import your metadata schema to Encord

Verify your schema

STEP 5: Create a Folder

STEP 6: Create JSON or CSV for import

STEP 7: Import your data

STEP 8: Create a collection

STEP 9: Create a Dataset from a Collection

STEP 10: Create an Ontology

To add objects:

To add attributes to an object:

To add a classification:

Configure classifications

STEP 11: Create Task Agent

Triggering the Task Agent

STEP 12: Create a Workflow template

STEP 13: Create a Project

Add users to the Project

STEP 14: Label your Data

STEP 15: Export your Labels