Skip to main content
Curate and Project Explorer support filtering, creating Collections, and by extension, creating Datasets and Projects based on the custom metadata on your data.

Prerequisites

Before you can filter your data or create a Collection based on your data’s custom metadata, the custom metadata must exist in your Annotate Project.
This content applies to custom metadata (clientMetadata), which is the metadata associated with individual data units. This is distinct from videoMetadata that is used to specify video parameters when using Strict client-only access. It is also distinct from patient metadata in DICOM files.

Where custom metadata is stored

Custom metadata (clientMetadata) is stored on each File in your Folders. Projects do not hold their own copy of the metadata. They show the custom metadata of the Files they use. This means:
  • Every Project that uses a File shows the same custom metadata.
  • If you update the custom metadata of a File, the change appears everywhere that File is used. You never need to update custom metadata in more than one place.
  • Copying a Project does not copy custom metadata. The original Project and the copy use the same Files, so they share the same custom metadata. This is true for both ATTACH and CLONE. See Copy Projects.
  • If you upload the same image twice, you create two separate Files. Each File has its own custom metadata.
Always import and update custom metadata on Files in Folders.

READ THIS FIRST

While not required, we strongly recommend importing a metadata schema before importing custom metadata into Encord. The process we recommend:
  1. Import a metadata schema. If a metadata schema already exists, you can import metadata. You can run a small piece of code to verify that a metadata schema exists
  2. Import your custom metadata.
Importing a schema with user_client.metadata_schema() adds keys to your current schema and does not remove existing keys. The deprecated set_client_metadata_schema_from_dict method overwrites the current schema with the new schema.

Geospatial Custom Metadata

Geospatial custom metadata contains latitude and longitude information. After importing geospatial custom metadata on your data units, you can filter data using a geospatial map in Curate.
  • Geospatial custom metadata is supported only in Curate.
  • Geospatial custom metadata can be applied to all data unit types and on individual frames in videos.
Geospatial Format Latitude values fall between +90 (North) to -90 (South). Longitude values fall between +180 (East) to -180 (West).

Import Custom Metadata

Importing custom metadata (clientMetadata) for any data type follows a similar format. But there are important differences for each data type.

Videos

BEST PRACTICE: If you want to use Curate or Project Explorer with your video data, we STRONGLY RECOMMEND using key frames, custom metadata, and custom embeddings. When you specify key frames, only the first frame and your key frames are imported by default, the same as setting the sampling_rate to 0. This can significantly speed up the import of your data into Curate and Project Explorer and help you to focus on only data you identify as critical.
The following table provides some guidance for the examples provided after the table.
TitleDescription
TemplateProvides the proper JSON format to import videos into Encord.
This template provides examples from the most basic to the most complex.
DataImports videos into Encord.
Why would I do this?
  • You ONLY want to add labels and classifications to your data.
  • You DO NOT want to use Curate or Project Explorer.
Key FramesImports videos with an Encord title and specifies key frames (frames of interest) for Curate and Project Explorer.
Why would I do this?
  • You ONLY want to see frames that you deem critical in Curate and Project Explorer.
  • You want to significantly improve the time to import videos into Curate and Project Explorer.
config is optional when specifying key frames for Curate and Project Explorer:
Specifying a sampling_rate of 0 only imports the first frame and all key frames of your video into Curate and Project Explorer.

If config is not specified, the keyframe_mode is frame. If you specify key frames, only the first frame and your key frames are imported, the same as a sampling_rate of 0. If you do not specify key frames, frames are sampled at 1 frame per second.
Metadata - VideoImports videos with an Encord title, specifies key frames (frames of interest), and custom metadata for Curate and Project Explorer.
Why would I do this?
  • Importing custom metadata allows you to filter your data in Curate and Project Explorer to make it easier to find the data you want to focus on. This speeds up creating Collections and by extension Datasets.
  • Specifying key frames means you ONLY want to see frames that you deem critical in Curate and Project Explorer AND you want to significantly improve the time to import videos into Curate and Project Explorer.
config is optional when specifying key frames for Curate and Project Explorer:
Specifying a sampling_rate of 0 only imports the first frame and all key frames of your video into Curate and Project Explorer.

If config is not specified, the keyframe_mode is frame. If you specify key frames, only the first frame and your key frames are imported, the same as a sampling_rate of 0. If you do not specify key frames, frames are sampled at 1 frame per second.
Metadata - Key FramesImports videos with an Encord title, specifies key frames (frames of interest), and adds custom metadata to individual key frames for Curate and Project Explorer.
Why would I do this?
  • You want to filter individual frames of your videos by custom metadata in Curate and Project Explorer.
When frames is a dictionary, each key is a frame number (or a time in seconds, if keyframe_mode is seconds) and each value holds the custom metadata for that frame.
EmbeddingsImports videos with an Encord title, specifies key frames (frames of interest), custom metadata, and custom embeddings for Curate and Project Explorer. This example includes the following custom metadata types: boolean, varchar, datetime, uuid, number.
Why would I do this?
  • Importing custom embeddings allows you to use scatter plots to examine your data AND allows you to use similarity search and natural language searches. Curate supports embedding dimensions 1 to 4096, while Project Explorer supports embedding dimensions 1 to 2000.
  • Importing custom metadata allows you to filter your data in Curate and Project Explorer to make it easier to find the data you want to focus on. This speeds up creating Collections and by extension Datasets.
  • Specifying key frames means you ONLY want to see frames that you deem critical in Curate and Project Explorer AND you want to significantly improve the time to import videos into Curate and Project Explorer.
config is optional when specifying custom embeddings for Curate and Project Explorer:
Specifying a sampling_rate of 0 only imports the first frame and all key frames of your video into Curate and Project Explorer.

If config is not specified, the keyframe_mode is frame. If you specify key frames, only the first frame and your key frames are imported, the same as a sampling_rate of 0. If you do not specify key frames, frames are sampled at 1 frame per second.
Video MetadataImports videos with the videoMetadata flag. When the videoMetadata flag is present in the JSON file, we directly use the supplied metadata without performing any additional validation, and do not store the file on our servers. To guarantee accurate labels, it is crucial that the metadata you provide is accurate.

Audio Files

The following is an example JSON file for uploading two audio files to Encord.
Waveform generation for long audio files can cause lag. To avoid this, generate the waveform offline and upload it with the audio file. For more information go here.
  • Template: Imports audio files with an Encord title, and with custom metadata. Custom metadata only appears in the Encord UI in Curate and Project Explorer as an option to filter your data.
  • Audio Metadata: Imports one audio file with the audioMetadata flag. When the audioMetadata flag is present in the JSON file, we directly use the supplied metadata without performing any additional validation, and do not store the file on our servers. To guarantee accurate labels, it is crucial that the metadata you provide is accurate.

PDFs

The following is an example JSON file for uploading PDFs to Encord. To speed up the import, you can also include pdfMetadata for each PDF. See PDF Metadata.
  • Template: Imports PDFs with an Encord title, and with custom metadata. Custom metadata only appears in the Encord UI in Curate and Project Explorer as an option to filter your data.
  • Data: Imports two PDFs with no title or custom metadata.
  • Custom Metadata: Imports two pdfs with a title and custom metadata.

Text Files

The following is an example JSON file for uploading text files to Encord. To speed up the import, you can also include textMetadata for each file. See Text Metadata.
  • Template: Imports text files with an Encord title, and with custom metadata. Custom metadata only appears in the Encord UI in Curate and Project Explorer as an option to filter your data.
  • Data: Imports two text files with no title or custom metadata.
  • Custom Metadata: Imports two text files with a title and custom metadata.

Single Images

For detailed information about the JSON file format used for import go here.The JSON structure for single images parallels that of videos.Template: Provides the proper JSON format to import images into Encord.Examples:
  • Data Imports the images only.
  • Custom Metadata: Imports images with an Encord title for the images and with custom metadata for each image. Custom metadata only appears in Curate and Project Explorer as an option to filter your data. This example includes the following custom metadata types: boolean, varchar, datetime, uuid, number.
  • Embeddings: Imports images with an Encord title, custom metadata, and custom embeddings for each image. This example includes the following custom metadata types: boolean, varchar, datetime, uuid, number.
  • Image Metadata: Imports images with image metadata. This improves the import speed for your images.

Image groups

For detailed information about the JSON file format used for import go here.
  • Image groups are collections of images that are processed as one annotation task.
  • Images within image groups remain unaltered, meaning that images of different sizes and resolutions can form an image group without the loss of data.
  • Image groups do NOT require ‘write’ permissions to your cloud storage.
  • Set "createVideo": false for image groups. If you leave out createVideo, Encord creates an image sequence instead.
  • Custom metadata is defined per image group, not per image. See our documentation here to learn how to add clientMetadata to images in an image group.
  • If skip_duplicate_urls is set to True, all URLs exactly matching existing image groups in the Folder are skipped.
The position of each image within the image group needs to be specified in the key (objectUrl_{position_number}).
Template: Provides the proper JSON format to import image groups into Encord.Examples:
  • Data: Imports the image groups only.
  • Custom Metadata: Imports image groups with an Encord title for the image groups and with custom metadata for each image group. Custom metadata only appears in Curate and Project Explorer as an option to filter your data. This example includes the following custom metadata types: boolean, varchar, datetime, uuid, number.

Image sequences

For detailed information about the JSON file format used for import go here.
  • Image sequences are collections of images that are processed as one annotation task and represented as a video.
  • Images within image sequences may be altered as images of varying sizes and resolutions are made to match that of the first image in the sequence.
  • Creating Image sequences from cloud storage requires ‘write’ permissions, as new files have to be created in order to be read as a video.
  • Each object in the image_groups array with the createVideo flag set to true represents a single image sequence.
  • Custom client metadata is defined per image sequence, not per image.
  • If skip_duplicate_urls is set to True, all URLs exactly matching existing image sequences in the Folder are skipped.
The only difference between adding image groups and image sequences using a JSON file is the createVideo flag: true for image sequences (the default if you leave it out) and false for image groups. Both use the key image_groups.
The position of each image within the sequence needs to be specified in the key (objectUrl_{position_number}).
Encord supports up to 32,767 entries (21:50 minutes) for a single image sequence. We recommend up to 10,000 to 15,000 entries for a single image sequence for best performance. If you need a longer sequence, we recommend using video instead of an image sequence.
Template: Provides the proper JSON format to import image groups into Encord.Examples:
  • Data: Imports the images groups only.
  • Custom Metadata: Imports image groups and custom metadata. This example includes the following custom metadata types: boolean, varchar, datetime, uuid, number.

DICOM

For detailed information about the JSON file format used for import go here.
  • Each element in the dicom_series array is one DICOM series.
  • Each series requires a title and at least one object URL, as shown in the example below.
  • Custom metadata (clientMetadata) is defined per series, not per file.
  • If skip_duplicate_urls is set to True, all object URLs exactly matching existing DICOM files in the Folder are skipped.
Custom metadata is distinct from patient metadata, which is included in the .dcm file and does not have to be specific during the upload to Encord.
The following is an example JSON for uploading three DICOM series belonging to a study. Each title and object URL correspond to individual DICOM series.
  • The first series contains only a single object URL, as it is composed of a single file. It also has custom metadata.
  • The second series contains 3 object URLs, as it is composed of three separate files.
  • The third series contains 2 object URLs, as it is composed of two separate files.
For each DICOM upload, an additional DicomSeries file is created. This file represents the series file-set. Only DicomSeries are displayed in the Encord application.
Template

NIfTI

The following is an example JSON file for uploading two NIfTI files to Encord. The first file has custom metadata.
Template
You can upload multiple file types using a single JSON file. The example below shows 1 image, 2 videos, 2 image sequences, and 1 image group.
Multiple file types

Import to Files already in Curate

We recommend importing custom metadata when you import your data, because importing with your data can significantly save you time when importing at scale. However, you can import custom metadata on data that already exists in Encord.
To replace the custom metadata of existing files with a JSON file instead of code, register the files again with "skip_duplicate_urls": true and "upsert_metadata": true. See Update Custom Metadata (JSON).
Importing with Custom Embeddings You can import custom embeddings with custom metadata. When importing custom embeddings with custom metadata keep the following in mind: config is optional when importing your custom embeddings:
If config is not specified, the keyframe_mode is frame. If you specify key frames, only the first frame and your key frames are imported, the same as a sampling_rate of 0. If you do not specify key frames, frames are sampled at 1 frame per second.
Specifying a sampling_rate of 0 only imports the first frame and all keyframes of your video into Curate.
The following examples patch the custom metadata of each data unit. Keys you specify are added or replaced, dictionaries such as $encord are merged into the current value, and all other keys stay as they are. Patching requires encord 0.1.212 or later, and does not work on data units in Cloud-synced Folders. Examples

Folders and Custom Metadata

List custom metadata (Folders)

Import Custom Metadata (Folders)

Before importing custom metadata to Encord, first import a metadata schema. We strongly recommend that you upload your custom metadata to Folders, instead of importing using Datasets. Importing custom metadata to data in folders allows you to filter your data in Curate by custom metadata.
After importing or updating custom metadata, verify that your custom metadata (list the data units with custom metadata) applied correctly. Do not simply add a print command after importing or updating your custom metadata.

Import custom metadata to specific data units in a Folder

This code allows you to import custom metadata on specific data units in Curate. This code OVERWRITES all existing custom metadata on a data unit.

Import custom metadata to all data units in a Folder

This code allows you to update ALL custom metadata on ALL data units in a Folder in Curate. This code OVERWRITES all existing custom metadata on a data unit.

Update custom metadata

The following code updates custom metadata without overwriting all existing custom metadata on a data unit. It replaces the values of keys you specify, adds new keys, and leaves all other keys unchanged. To remove a key, set it to None. The code patches the custom metadata, which requires encord 0.1.212 or later and does not work on data units in Cloud-synced Folders. The Specific data units in Curate code updates the custom metadata of the data units you list. The All data units in a Project code applies the same update to every data unit in the specified Project. Replace {"metadata-1": "value"} with the metadata you want to update.

Bulk import custom metadata to all data units in a Folder

This code allows you to update custom metadata on all data units in a Folder in Curate. This code OVERWRITES all existing custom metadata on a data unit. Using a Bundle sends the updates to Encord in batches of up to 1000 data units, instead of one request per data unit.

Bulk custom metadata import on specific data units

This code updates custom metadata on many specific data units at once. It DOES NOT OVERWRITE existing custom metadata on a data unit: it replaces the values of the keys you specify, adds new keys, and leaves all other keys unchanged. To remove a key, set it to None. The code patches the custom metadata, which requires encord 0.1.212 or later and does not work on data units in cloud-synced Folders.

Add custom metadata to images in an image group

The following script adds clientMetadata to all images / frames in a specified Image Group. Ensure that you:
  • Replace <private_key_path> with the file path to your private access key.
  • Replace <image-group-id> with the File ID (UUID) of the target Image Group.
  • Customize the _get_metadata_for_image function with the clientMetadata you want to add. To add unique metadata for each image, make the function dynamic by passing additional variables.

Patch custom metadata

Patching changes only the custom metadata keys you specify, and leaves every other key on a data unit as it is. Encord applies the patch for you, so you don’t need to read the current custom metadata first, and changes made to other keys at the same time are kept. To replace all of a data unit’s custom metadata, use item.update(client_metadata=...) instead. A patch is a dictionary of the keys to change, applied as a JSON merge patch: Keys that are not in your patch stay as they are. For example, patching {"reviewed": False, "camera": {"fps": 30, "iso": 100}, "draft": True} with {"reviewed": True, "camera": {"fps": 60}, "draft": None} results in {"reviewed": True, "camera": {"fps": 60, "iso": 100}}.
  • Requires encord SDK version 0.1.212 or later.
  • Because None removes a key, a patch can’t set the value of a key to null.
  • Data units in Cloud-synced Folders can’t be patched. Update their custom metadata in your cloud bucket and re-sync the Folder instead.
  • A data unit’s custom metadata can be at most 10 MB after a patch is applied.

Patch custom metadata for one data unit

After the patch, item.client_metadata holds the data unit’s full custom metadata.

Patch custom metadata for many data units

patch_storage_items_client_metadata applies a patch to each data unit, waits for the patches to be applied, and returns those that could not be confirmed as applied.
  • Each patch is applied on its own, and large requests are split up for you.
  • A returned patch has the status NOT_FOUND if the data unit doesn’t exist or you can’t edit it. Otherwise its status is FAILED, and error says why. Sending a patch again is safe, so you can retry the FAILED ones. A status of UNKNOWN means your version of encord does not recognize the outcome, so upgrade encord.
  • The call waits for up to timeout_seconds (10 minutes by default). If the patches take longer, a ClientMetadataPatchTimeoutError is raised. The patches keep being applied: check them with get_client_metadata_patch_job, using the error’s pending_job_uuids. If waiting fails for another reason, a ClientMetadataPatchWaitError is raised, which also has pending_job_uuids.
  • If a data unit can’t be edited, an AuthorisationError is raised and nothing is changed. If some patches were already sent, a ClientMetadataPatchPartiallyStartedError is raised instead, and its not_started_item_uuids lists the data units whose patches were not sent.

Patch custom metadata by file URL

If you know your files by their cloud storage URL rather than by their Encord ID, use patch_storage_items_client_metadata_by_url. It finds the data units with each URL and patches them. It returns the data units found for each URL in item_uuids_by_url, with an empty list for a URL that matches none, and the patches that could not be confirmed as applied in failures.
  • Every data unit with the URL is patched, because the same file can be in Encord more than once. You need to be able to edit all of them.
  • With folders, data units are looked up directly in those Folders, not in their subfolders. Without folders, they are looked up in all the Folders you can access.
  • Data units in Cloud-synced Folders are left out.

Patch custom metadata without blocking your script

patch_storage_items_client_metadata and patch_storage_items_client_metadata_by_url wait until the patches are applied. To start the patches and check on them later, use start_client_metadata_patch_jobs or start_client_metadata_patch_jobs_by_url, which take the same patches and return the background jobs applying them. Pass each job’s UUID to get_client_metadata_patch_job: its status is PENDING until the job finishes, then DONE with the patches that could not be confirmed as applied in failures. If a job’s status is FAILED, send all of its patches again. See the EncordUserClient reference for details.

Datasets and custom metadata

Before importing custom metadata to Encord, first import a metadata schema.We strongly recommend that you upload your custom metadata to Folders, instead of importing using Datasets. Importing custom metadata to data in Folders allows you to filter your data in Curate by custom metadata.

List custom metadata (Datasets)

The following code lists the custom metadata of all data units in the specified Dataset. The code prints the custom metadata along with the data unit’s index within the dataset.

Deprecated - Import custom metadata (Datasets)

Importing custom metadata (clientMetadata) to data units in the Datasets is deprecated. Import custom metadata to data units in Folders instead.
Before importing custom metadata to Encord, first import a metadata schema. We strongly recommend that you import your custom metadata to Folders, instead of importing to Datasets. Importing custom metadata to data in folders allows you to filter your data in Curate by custom metadata.

Import custom metadata to a specific data unit in your Dataset

You can import custom metadata (clientMetadata) to specific data units in the Dataset.
  • Replace <private_key_path> with the path to your private key.
  • Replace <dataset_hash> with the hash of your Dataset.
  • Replace Image1.png and the other file names in the metadata variable with the names of the files in your Dataset to which you want to add metadata.
You can find the titles of the files in your Dataset by listing the custom metadata in the Dataset.
Multiple data units

Import custom metadata (clientMetadata) to all data units in a Dataset

The following code adds the same custom metadata (clientMetadata) to each data unit in the specified dataset. The code prints the custom metadata along with the data units index within the dataset, so that you can verify that the custom metadata was set correctly.

Deprecated - Reserved Keywords

Reserved keywords are strings that are set aside for exclusive use. The following keywords are reserved:
  • keyframes

KEYFRAMES

keyframes is reserved for use with frames of interest in videos. Specifying keyframes on specific frames ensures that those frames import into Curate and Project Explorer. That means frames specified using keyframes are available to filter your frames and for calculating embeddings on your data.
You can include keyframes while importing your videos or after you import your videos. Import keyframes to Specific Data Units (Folder): This code allows you to import keyframes on specific videos in Curate. This code DOES NOT OVERWRITE all existing custom metadata on a data unit. It does overwrite custom metadata with existing values and adds new custom metadata to the data unit.