Custom Metadata in Active

Active and Index support filtering, creating Collections, and by extension, creating Datasets and Projects based on the custom metadata on your data.

Prerequisites

Before you can filter your data or create a Collection based on your data’s custom metadata, the custom metadata must exist in your Annotate Project. s

This content applies to custom metadata (clientMetadata), which is the metadata associated with individual data units. This is distinct from videoMetadata that is used to specify video parameters when using Strict client-only access. It is also distinct from patient metadata in DICOM files.

Custom metadata (clientMetadata) is accessed by specifying the dataset using the <dataset_hash>. All Projects that have the specified Dataset attached contain custom metadata.

READ THIS FIRST

While not required, we strongly recommend importing a metadata schema before importing custom metadata into Encord. The process we recommend:

If a metadata schema already exists, you can import metadata. You can run a small piece of code to verify that a metadata schema exists.

Performing multiple schema imports overwrites the current schema with the new schema.

Metadata schema

If you are not using Index or Active, you do not need to create a Custom Metadata Schema, because you will not be using custom metadata.

Before importing your custom metadata to Encord, we recommend that you import a metadata schema. Encord uses metadata schemas to validate custom metadata uploaded to Encord and to instruct Index and Active how to display your metadata.

To handle your custom metadata schema across multiple teams within the same organization, we recommend using namespacing for metadata keys in the schema. This ensures that different teams can define and manage their own metadata schema without conflicts. For example, team A could use video.description, while team B could use audio.description. Another example could be TeamName.MetadataKey. This approach maintains clarity and avoids key collisions across departments.

Benefits of Using a Metadata Schema

Using a metadata schema provides several benefits:

Validation: Ensures that all custom metadata conforms to predefined data types, reducing errors during data import and processing.
Consistency: Maintains uniformity in data types across different datasets and projects, which simplifies data management and analysis.
Filtering and Sorting: Enhances the ability to filter and sort data efficiently in the Encord platform, enabling more accurate and quick data retrieval.

Metadata Schema Table

Metadata Schema keys support letters (a-z, A-Z), numbers (0-9), and blank spaces ( ), hyphens (-), underscores (_), and periods (.). Metadata schema keys are case sensitive.

Use add_scalar to add a scalar key to your metadata schema.

Scalar Key	Description	Display Benefits
boolean	Binary data type with values “true” or “false”.	Filtering by binary values
datetime	ISO 8601 formatted date and time.	Filtering by time and date
number	Numeric data type supporting float values.	Filtering by numeric values
uuid	Customer specified unique identifier for a data unit.	Filtering by customer specified unique identifier
varchar	Textual data type. Formally `string`. `string` can be used as an alias for `varchar`, but we STRONGLY RECOMMEND that you use `varchar`.	Filtering by string.
text	Text data with unlimited length (example: transcripts for audio). Formally `long_string`. `long_string` can be used as an alias for `text`, but we STRONGLY RECOMMEND that you use `text`.	Storing and filtering large amounts of text.

Use add_enum and add_enum_options to add an enum and enum options to your metadata schema.

Key	Description	Display Benefits
enum	Enumerated type with predefined set of values.	Facilitates categorical filtering and data validation

Use add_embedding to add an embedding to your metadata schema.

Key	Description	Display Benefits
embedding	1 to 4096 for Index. 1 to 2000 for Active.	Filtering by embeddings, similarity search, 2D scatter plot visualization (Coming Soon)

Incorrectly specifying a data type in the schema can cause errors when filtering your data in Index or Active. If you encounter errors while filtering, verify your schema is correct. If your schema has errors, correct the errors, re-import the schema, and then re-sync your Active Project.

While still supported, we STRONGLY RECOMMEND that you use the latest metadata schema instructions.

ENUM Name	String Representation	Description	Display Benefits
NUMBER	number	Numeric data type supporting float values	Filtering by numeric values
STRING	string	Textual data type	Filtering by string
BOOLEAN	boolean	Binary data type with values “true” or “false”	Filtering by binary values
DATETIME	datetime	ISO 8601 formatted date and time	Filtering by time and date
ENUM	enum	Enumerated type with predefined set of values	Facilitates categorical filtering and data validation
EMBEDDING	embedding	1 to 4096 for Index. 1 to 2000 for Active.	Filtering by embeddings, similarity search, 2D scatter plot visualization (Coming Soon)
LONG_STRING	long_string	Text data with unlimited length (example: transcripts for audio)	Storing and filtering large amounts of text

Import your metadata schema to Encord

Import schema template

# Import dependencies
from encord import EncordUserClient

SSH_PATH = "<file-path-to-ssh-private-key>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

user_client.set_client_metadata_schema_from_dict({'metadata_1': 'data type', 'metadata_2': 'data type', 'metadata_3': 'data type'})

Example

# Import dependencies
from encord import EncordUserClient

SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

user_client.set_client_metadata_schema_from_dict({'captured_at': 'datetime', 'city': 'datetime', 'dark': 'boolean'})

Verify your schema

After importing your schema to Encord we recommend that you verify that the import is successful. Run the following code to verify your metadata schema imported and that the schema is correct.

Verify your schema template

# Import dependencies
from encord import EncordUserClient

SSH_PATH = "<file-path-to-ssh-private-key>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

schema = user_client.get_client_metadata_schema()

print(schema)

Verify your schema example

# Import dependencies
from encord import EncordUserClient

SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

schema = user_client.get_client_metadata_schema()

print(schema)

Reserved Keywords

Reserved keywords are strings that are set aside for exclusive use. The following keywords are reserved:

keyframes

KEYFRAMES

keyframes is reserved for use with frames of interest in videos. Specifying keyframes on specific frames ensures that those frames are imported into Index and Active. That means frames specified using keyframes are available to filter your frames and for calculating embeddings on your data.

keyframes Format

 client_metadata = {
     "keyframes": [<frame_number>, <frame_number>, <frame_number>, <frame_number>, <frame_number>]
}

Example
client_metadata = {
    "keyframes": [13, 17, 19, 23, 127, 149, 307, 557]
}

You can include keyframes while importing your videos or after you import your videos.

Import `keyframes` to specific data units in a folder

This code allows you to import keyframes on specific videos in Index. This code DOES NOT OVERWRITE all existing custom metadata on a data unit. It does overwrite custom metadata with existing values and adds new custom metadata to the data unit.

Boilerplate
# Import dependencies
from encord import EncordUserClient

# Authentication
SSH_PATH = "<private_key_path>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

# Define a dictionary with item UUIDs and their keyframes updates
updates = {
    "<data-unit-id>": {"keyframes": [<frame_number>, <frame_number>, <frame_number>, <frame_number>, <frame_number>]},
    "<data-unit-id>": {"keyframes": [<frame_number>, <frame_number>, <frame_number>, <frame_number>, <frame_number>]},
    "<data-unit-id>": {"keyframes": [<frame_number>, <frame_number>, <frame_number>, <frame_number>, <frame_number>]},
    "<data-unit-id>": {"keyframes": [<frame_number>, <frame_number>, <frame_number>, <frame_number>, <frame_number>]}
}

# Update the storage items based on the dictionary
for item_uuid, metadata_update in updates.items():
    item = user_client.get_storage_item(item_uuid=item_uuid)

    # make a copy of the current metadata and update it with the new metadata
    curr_metadata = item.client_metadata.copy()
    curr_metadata.update(metadata_update)

    # update the item with the new metadata
    item.update(client_metadata=curr_metadata)

Example 1
# Import dependencies
from encord import EncordUserClient

# Authentication
SSH_PATH = "/Users/chris-encord/sdk-ssh-private-key.txt"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

# Define a dictionary with item UUIDs and their keyframes updates
updates = {
    "8ad58157-ca74-4ae4-8f37-a0193430bcac": {"keyframes": [13, 17, 19, 23, 127, 149, 307, 557]},
    "f165fe81-1956-4347-81ca-3a3b198f3f23": {"keyframes": [10, 110, 210, 320, 430]},
    "9bd58157-ca74-4ae4-8f37-a0193431bcba": {"keyframes": [1087, 1549, 2029, 3527]},
    "62daaa33-195e-4faf-be5b-8335a239beb6": {"keyframes": [1000, 2001, 3241, 4637, 5431]}
}

# Update the storage items based on the dictionary
for item_uuid, metadata_update in updates.items():
    item = user_client.get_storage_item(item_uuid=item_uuid)

    # make a copy of the current metadata and update it with the new metadata
    curr_metadata = item.client_metadata.copy()
    curr_metadata.update(metadata_update)

    # update the item with the new metadata
    item.update(client_metadata=curr_metadata)

Folders and custom metadata

List custom metadata (Folders)

Boilerplate

from encord import EncordUserClient
from encord.orm.storage import StorageFolder, StorageItem, StorageItemType, FoldersSortBy

# Authentication
SSH_PATH = "<file-path-to-ssh-private-key-file>"
FOLDER_HASH = "<unique-folder-id>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

folder = user_client.get_storage_folder(FOLDER_HASH)
items = folder.list_items()

for item in items:
    print (item.uuid, item.client_metadata)

Example

from encord import EncordUserClient
from encord.orm.storage import StorageFolder, StorageItem, StorageItemType, FoldersSortBy

# Authentication
SSH_PATH = "/Users/chris-encord/sdk-ssh-private-key.txt"
FOLDER_HASH = "2a838557-d6f4-4408-a980-64246dc5c56b"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

folder = user_client.get_storage_folder(FOLDER_HASH)
items = folder.list_items()

for item in items:
    print (item.uuid, item.client_metadata)

Import custom metadata (Folders)

Before importing custom metadata to Encord, first import a metadata schema.

We strongly recommend that you upload your custom metadata to Folders, instead of importing using Datasets. Importing custom metadata to data in folders allows you to filter your data in Index by custom metadata.

After importing or updating custom metadata, verify that your custom metadata (list the data units with custom metadata) applied correctly. Do not simply add a print command after importing or updating your custom metadata.

Import custom metadata to specific data units in a Folder

This code allows you to import custom metadata on specific data units in Index. This code OVERWRITES all existing custom metadata on a data unit.

Boilerplate
# Import dependencies
from encord import EncordUserClient
from encord.orm.storage import StorageFolder, StorageItem, StorageItemType, FoldersSortBy


# Authentication
SSH_PATH = "<file-path-to-ssh-private-key-file>"
FOLDER_HASH = "<unique-folder-id>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

folder = user_client.get_storage_folder(FOLDER_HASH)

# Define a dictionary with item UUIDs and their respective metadata updates
updates = {
    # "<data-unit-id>": {"metadata": "metadata-value"},
    # "<data-unit-id>": {"metadata": False},
    # "<data-unit-id>": {"metadata": "metadata-value"},
    # "<data-unit-id>": {"metadata": True}
}

# Update the storage items based on the dictionary
for item_uuid, metadata in updates.items():
    item = user_client.get_storage_item(item_uuid=item_uuid)
    item.update(client_metadata=metadata)

Example 1
# Import dependencies
from encord import EncordUserClient
from encord.orm.storage import StorageFolder, StorageItem, StorageItemType, FoldersSortBy

# Authentication
SSH_PATH = "/Users/chris-encord/sdk-ssh-private-key.txt"
FOLDER_HASH = "2a838557-d6f4-4408-a980-64246dc5c56b"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

folder = user_client.get_storage_folder(FOLDER_HASH)

# Define a dictionary with item UUIDs and their respective metadata updates
updates = {
    # "<data-unit-id>": {"metadata": "metadata-value"},
    # "<data-unit-id>": {"metadata": False}, # examples
    "8ad58157-ca74-4ae4-8f37-a0193430bcac": {"dark": True},
    "f165fe81-1956-4347-81ca-3a3b198f3f23": {"light": False},
    "9bd58157-ca74-4ae4-8f37-a0193431bcba": {"count": "100"},
    "62daaa33-195e-4faf-be5b-8335a239beb6": {"taken_at": "2024-02-24"}
}

# Update the storage items based on the dictionary
for item_uuid, metadata in updates.items():
    item = user_client.get_storage_item(item_uuid=item_uuid)
    item.update(client_metadata=metadata)

Import custom metadata to all data units in a Folder

This code allows you to update ALL custom metadata on ALL data units in a Folder in Index. This code OVERWRITES all existing custom metadata on a data unit.

Boilerplate
# Import dependencies
from encord import EncordUserClient
from encord.orm.storage import StorageFolder, StorageItem, StorageItemType, FoldersSortBy

# Authentication
SSH_PATH = "<file-path-to-ssh-private-key-file>"
FOLDER_HASH = "<unique-folder-id>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

folder = user_client.get_storage_folder(FOLDER_HASH)
items = folder.list_items()

for item in items:
     item.update(client_metadata={"metadata": "value", "metadata": "value"})

Example
# Import dependencies
from encord import EncordUserClient
from encord.orm.storage import StorageFolder, StorageItem, StorageItemType, FoldersSortBy

# Authentication
SSH_PATH = "/Users/chris-encord/sdk-ssh-private-key.txt"
FOLDER_HASH = "2a838557-d6f4-4408-a980-64246dc5c56b"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

folder = user_client.get_storage_folder(FOLDER_HASH)
items = folder.list_items()

for item in items:
     item.update(client_metadata={"dark": True, "captured_at": "2024-02-24"})

Update custom metadata to specific data units in a folder

This code allows you to update custom metadata on specific data units in Index. This code DOES NOT OVERWRITE all existing custom metadata on a data unit. It does overwrite custom metadata with existing values and adds new custom metadata to the data unit.

Boilerplate
# Import dependencies
from encord import EncordUserClient

# Authentication
SSH_PATH = "<private_key_path>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

# Define a dictionary with item UUIDs and their respective metadata updates
updates = {
    # "<data-unit-id>": {"metadata": "metadata-value"},
    # "<data-unit-id>": {"metadata": False},
    # "<data-unit-id>": {"metadata": "metadata-value"},
    # "<data-unit-id>": {"metadata": True}
}

# Update the storage items based on the dictionary
for item_uuid, metadata_update in updates.items():
    item = user_client.get_storage_item(item_uuid=item_uuid)

    # make a copy of the current metadata and update it with the new metadata
    curr_metadata = item.client_metadata.copy()
    curr_metadata.update(metadata_update)

    # update the item with the new metadata
    item.update(client_metadata=curr_metadata)

Example 1
# Import dependencies
from encord import EncordUserClient

# Authentication
SSH_PATH = "/Users/chris-encord/sdk-ssh-private-key.txt"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)


# Define a dictionary with item UUIDs and their respective metadata updates
updates = {
    # "<data-unit-id>": {"metadata": "metadata-value"},
    # "<data-unit-id>": {"metadata": False}, # examples
    "8ad58157-ca74-4ae4-8f37-a0193430bcac": {"dark": True},
    "f165fe81-1956-4347-81ca-3a3b198f3f23": {"light": False},
    "9bd58157-ca74-4ae4-8f37-a0193431bcba": {"count": "100"},
    "62daaa33-195e-4faf-be5b-8335a239beb6": {"taken_at": "2024-02-24"}
}

# Update the storage items based on the dictionary
for item_uuid, metadata_update in updates.items():
    item = user_client.get_storage_item(item_uuid=item_uuid)

    # make a copy of the current metadata and update it with the new metadata
    curr_metadata = item.client_metadata.copy()
    curr_metadata.update(metadata_update)

    # update the item with the new metadata
    item.update(client_metadata=curr_metadata)

Bulk custom metadata import to all data units in a Folder

This code allows you to update custom metadata on all data units in a Folder in Index. This code OVERWRITES all existing custom metadata on a data unit.

Using bundle allows you to update up to 1000 label rows at a time.

# Import dependencies
from encord import EncordUserClient
from encord.http.bundle import Bundle
from encord.orm.storage import StorageFolder, StorageItem, StorageItemType, FoldersSortBy

# Authentication
SSH_PATH = "<ssh-private-key>"
FOLDER_HASH = "<unique-folder-id>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

folder = user_client.get_storage_folder(FOLDER_HASH)
items = folder.list_items()

# Use the Bundle context manager
with Bundle() as bundle:
    for item in items:
        # Update each item with client metadata
        item.update(client_metadata={"metadata-1": "value", "metadata-2": False}, bundle=bundle)

Bulk custom metadata import on specific data units

This code allows you to update custom metadata on specific data units in a Folder in Index. This code DOES NOT OVERWRITE existing custom metadata on a data unit. It does overwrite custom metadata with existing values and adds new custom metadata to the data unit.

Using bundle allows you to update up to 1000 label rows at a time.

# Import dependencies
from encord import EncordUserClient
from encord.http.bundle import Bundle
from encord.orm.storage import StorageFolder, StorageItem, StorageItemType, FoldersSortBy

# Authentication
SSH_PATH = "<ssh-private-key>"
FOLDER_HASH = "<unique-folder-id>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

folder = user_client.get_storage_folder(FOLDER_HASH)
updates = {
    # "<data-unit-id>": {"metadata-1": "metadata-value"},
    # "<data-unit-id>": {"metadata-2": False},
    # "<data-unit-id>": {"metadata-1": "metadata-value"},
    # "<data-unit-id>": {"metadata-2": True}
}

# Use the Bundle context manager
with Bundle() as bundle:
    for storage_item in folder.list_items():
        # Update each item with client metadata
        update = updates[storage_item.uuid]
        storage_item.update(client_metadata=update, bundle=bundle)

Datasets and custom metadata

Before importing custom metadata to Encord, first import a metadata schema.

We strongly recommend that you upload your custom metadata to Folders, instead of importing using Datasets. Importing custom metadata to data in Folders allows you to filter your data in Index by custom metadata.

List custom metadata (Datasets)

The following code lists the custom metadata of all data units in the specified Dataset. The code prints the custom metadata along with the data unit’s index within the dataset.

# Import dependencies
from encord import EncordUserClient
from encord.client import DatasetAccessSettings

# Authenticate with Encord using the path to your private key
client = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path="<private_key_path>"
)

# Specify a dataset to read or write metadata to
dataset = client.get_dataset("<dataset_hash>")

# Fetch the dataset's metadata
dataset.set_access_settings(DatasetAccessSettings(fetch_client_metadata=True))

# Read the metadata of all data units in the dataset.
for data_unit, data_row in enumerate(dataset.data_rows):
    print(f"{data_row.client_metadata} - Data Unit: {data_unit}")

Import custom metadata (Datasets)

Before importing custom metadata to Encord, first import a metadata schema. We strongly recommend that you import your custom metadata to Folders, instead of importing to Datasets. Importing custom metadata to data in folders allows you to filter your data in Index by custom metadata.

Import custom metadata to a specific data unit in your Annotate Project

You can import custom metadata (clientMetadata) to specific data units in the Dataset.

You can find the <data unit number> by reading all metadata in the Dataset. The sample code provided prints all <data unit numbers.

# Import dependencies
from encord import EncordUserClient
from encord.client import DatasetAccessSettings

# Authenticate with Encord using the path to your private key
client = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path="<private_key_path>"
)

# Specify a dataset to read or write metadata to
dataset = client.get_dataset("<dataset_hash>")

# Fetch the dataset's metadata
dataset.set_access_settings(DatasetAccessSettings(fetch_client_metadata=True))

# Add metadata to a specific data unit by replacing <data unit number> with the number of the data unit
data_row = dataset.data_rows[<data unit number>]

# Replace {"my": "metadata"} with the metadata you want to add
data_row.client_metadata= {"my": "metadata"}
data_row.save()
print(data_row.client_metadata)

Import custom metadata (`clientMetadata`) to all data units in a dataset

The following code adds the same custom metadata (clientMetadata) to each data unit in the specified dataset. The code prints the custom metadata along with the data units index within the dataset, so that you can verify that the custom metadata was set correctly.

# Import dependencies
from encord import EncordUserClient
from encord.client import DatasetAccessSettings

# Authenticate with Encord using the path to your private key
client = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path="<private_key_path>"
)

# Specify a dataset to read or write metadata to
dataset = client.get_dataset("<dataset_hash>")

# Fetch the dataset's metadata
dataset.set_access_settings(DatasetAccessSettings(fetch_client_metadata=True))

# Add metadata to all data units in the dataset.
# Replace {"my": "metadata"} with the metadata you want to add
for data_unit, data_row in enumerate(dataset.data_rows):
    data_row.client_metadata = {"my": "metadata"}
    data_row.save()
    print(f"{data_row.client_metadata} - Data Unit: {data_unit}")