Custom Metadata in Index

Active and Index support filtering, creating Collections, and by extension, creating Datasets and Projects based on the custom metadata on your data.

Prereqs

Before you can filter your data or create a Collection based on your data's custom metadata, the custom metadata must exist in your Annotate Project.

ℹ️
Note
This content applies to custom metadata (clientMetadata), which is the metadata associated with individual data units. This is distinct from videoMetadata that is used to specify video parameters when using Strict client-only access. It is also distinct from patient metadata in DICOM files.

Custom metadata (clientMetadata) is accessed by specifying the dataset using the <dataset_hash>. All Projects that have the specified Dataset attached contain custom metadata.

READ THIS FIRST

While not required, we strongly recommend importing a metadata schema before importing custom metadata into Encord. The process we recommend:

Import a metadata schema.
Import your custom metadata.

If a metadata schema already exists, you can import metadata. You can run a small piece of code to verify that a metadata schema exists.

ℹ️
Note
Performing multiple schema imports overwrites the current schema with the new schema.

Import metadata schema

Before importing your custom metadata to Encord, we recommend that you import a metadata schema. Encord uses metadata schemas to validate custom metadata uploaded to Encord and to instruct Active how to display your metadata.

Metadata schemas support the following data types for your metadata:

NUMBER = "number"
STRING = "string"
BOOLEAN = "boolean"
DATETIME = "datetime"
ENUM = "enum"
EMBEDDING = "embedding" - 512 dimension embeddings for Active, 1 to 4096 for Index
LONG_STRING = "long_string" - Unlimited string length. Example, transcripts for audio.

ℹ️
Note

Encord supports ONE metadata schema for each Organization. Performing multiple imports overwrites the current schema with the new schema.

NUMBER supports float values

Boolean supports "true" and "false"

DATETIME format is ISO 8601 date time (for example: yyyymmdd, yyyy-mm-dd, yyyy-mm-ddThh:mm:ss.msusps)

ℹ️
Note
Incorrectly specifying a data type in the schema can cause errors when filtering your data in Active. If you encounter errors while filtering, verify your schema is correct. If your schema has errors, correct the errors, re-import the schema, and then re-sync your Active Project.

Import your metadata schema to Encord


# Import dependencies
from encord import EncordUserClient

SSH_PATH = "<file-path-to-ssh-private-key>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

user_client.set_client_metadata_schema_from_dict({'metadata_1': 'data type', 'metadata_2': 'data type', 'metadata_3': 'data type'})


# Import dependencies
from encord import EncordUserClient

SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

user_client.set_client_metadata_schema_from_dict({'captured_at': 'datetime', 'city': 'datetime', 'dark': 'boolean'})

Append your metadata schema

The following code appends your current custom metadata schema with new fields.

# Import dependencies
from encord import EncordUserClient

SSH_PATH = "<file-path-to-ssh-private-key>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

# Retrieve the current client metadata schema
current_schema = user_client.get_client_metadata_schema()

# Define the new metadata to append
new_metadata = {'metadata_4': 'data type', 'metadata_5': 'data type'}

# Update the current schema with the new metadata
updated_schema = {**current_schema, **new_metadata}

# Set the updated client metadata schema
user_client.set_client_metadata_schema_from_dict(updated_schema)

# Optionally, print the updated schema to verify the changes
print(updated_schema)

# Import dependencies
from encord import EncordUserClient

SSH_PATH = "/Users/chris-encord/sdk-ssh-private-key.txt"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

# Retrieve the current client metadata schema
current_schema = user_client.get_client_metadata_schema()

# Define the new metadata to append
new_metadata = {'genre': 'string', 'rating': 'number'}

# Update the current schema with the new metadata
updated_schema = {**current_schema, **new_metadata}

# Set the updated client metadata schema
user_client.set_client_metadata_schema_from_dict(updated_schema)

# Optionally, print the updated schema to verify the changes
print(updated_schema)

Verify your schema

After importing your schema to Encord we recommend that you verify that the import is successful. Run the following code to verify your metadata schema imported and that the schema is correct.


# Import dependencies
from encord import EncordUserClient

SSH_PATH = "<file-path-to-ssh-private-key>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

schema = user_client.get_client_metadata_schema()

print(schema)


# Import dependencies
from encord import EncordUserClient

SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH
)

schema = user_client.get_client_metadata_schema()

print(schema)

DEPRECATED FORMAT

Reserved keywords are strings that are set aside for exclusive use. The following keywords are reserved:

keyframes

keyframes is reserved for use with frames of interest in videos. Specifying keyframes on specific frames ensures that those frames import into Index and Active. That means frames specified using keyframes are available to filter your frames and for calculating embeddings on your data.


 client_metadata = {
     "keyframes": [<frame_number>, <frame_number>, <frame_number>, <frame_number>, <frame_number>]
}

client_metadata = {
    "keyframes": [13, 17, 19, 23, 127, 149, 307, 557]
}

You can include key frames while importing your videos or after you import your videos.

Specify key frames to specific data units in a folder

This code allows you to specify key frames on specific videos in Index. This code DOES NOT OVERWRITE all existing custom metadata on a data unit. It does overwrite custom metadata with existing values and adds new custom metadata to the data unit.

ℹ️
Note
Frame 0 can never be excluded from a video. This is because frame 0 is the first frame in the video and all customizations are based on frame 0.

# Import dependencies
from encord import EncordUserClient

# Authentication
SSH_PATH = "<private_key_path>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

# Define a dictionary with item UUIDs and their client_metadata updates
updates = {
    "<data-unit-id-1>": {
        "$encord": {
            "config": {
                "sampling_rate": "<samples/second>",  # VIDEO ONLY
                "keyframe_mode": "frame",  # or "seconds" based on your use case
            }
            "frames": {
                "<key format>": {
                    # You can also add custom metadata and embeddings
                }
            }
        },
        "meta-text": "some-text"
    },
    "<data-unit-id-2>": {
        "$encord": {
            "frames": {
                "1": {},
                "2": {},
                "3": {},
            }
        },
    },
    "<data-unit-id-3>": {
        "$encord": {
            "frames": [1, 2, 3],
        },
    }
}

# Update the storage items based on the dictionary
for item_uuid, metadata_update in updates.items():
    item = user_client.get_storage_item(item_uuid=item_uuid)

    # make a copy of the current metadata and update it with the new metadata
    curr_metadata = item.client_metadata.copy()
    curr_metadata.update(metadata_update)

    # update the item with the new metadata
    item.update(client_metadata=curr_metadata)

Folders and custom metadata

List custom metadata (Folders)


from encord import EncordUserClient
from encord.orm.storage import StorageFolder, StorageItem, StorageItemType, FoldersSortBy

# Authentication
SSH_PATH = "<file-path-to-ssh-private-key-file>"
FOLDER_HASH = "<unique-folder-id>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

folder = user_client.get_storage_folder(FOLDER_HASH)
items = folder.list_items()

for item in items:
    print (item.uuid, item.client_metadata)


from encord import EncordUserClient
from encord.orm.storage import StorageFolder, StorageItem, StorageItemType, FoldersSortBy

# Authentication
SSH_PATH = "/Users/chris-encord/sdk-ssh-private-key.txt"
FOLDER_HASH = "2a838557-d6f4-4408-a980-64246dc5c56b"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

folder = user_client.get_storage_folder(FOLDER_HASH)
items = folder.list_items()

for item in items:
    print (item.uuid, item.client_metadata)

Import custom metadata (Folders)

❗️
CRITICAL INFORMATION
Before importing custom metadata to Encord, first import a metadata schema.
We strongly recommend that you upload your custom metadata to Folders, instead of importing using Datasets. Importing custom metadata to data in folders allows you to filter your data in Index by custom metadata.

❗️
CRITICAL INFORMATION
After importing or updating custom metadata, verify that your custom metadata (list the data units with custom metadata) applied correctly. Do not simply add a print command after importing or updating your custom metadata.

Import custom metadata to specific data units in a Folder

This code allows you to import custom metadata on specific data units in Index. This code OVERWRITES all existing custom metadata on a data unit.

# Import dependencies
from encord import EncordUserClient
from encord.orm.storage import StorageFolder, StorageItem, StorageItemType, FoldersSortBy


# Authentication
SSH_PATH = "<file-path-to-ssh-private-key-file>"
FOLDER_HASH = "<unique-folder-id>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

folder = user_client.get_storage_folder(FOLDER_HASH)

# Define a dictionary with item UUIDs and their respective metadata updates
updates = {
    # "<data-unit-id>": {"metadata": "metadata-value"},
    # "<data-unit-id>": {"metadata": False},
    # "<data-unit-id>": {"metadata": "metadata-value"},
    # "<data-unit-id>": {"metadata": True}
}

# Update the storage items based on the dictionary
for item_uuid, metadata in updates.items():
    item = user_client.get_storage_item(item_uuid=item_uuid)
    item.update(client_metadata=metadata)

# Import dependencies
from encord import EncordUserClient
from encord.orm.storage import StorageFolder, StorageItem, StorageItemType, FoldersSortBy

# Authentication
SSH_PATH = "/Users/chris-encord/sdk-ssh-private-key.txt"
FOLDER_HASH = "2a838557-d6f4-4408-a980-64246dc5c56b"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

folder = user_client.get_storage_folder(FOLDER_HASH)

# Define a dictionary with item UUIDs and their respective metadata updates
updates = {
    # "<data-unit-id>": {"metadata": "metadata-value"},
    # "<data-unit-id>": {"metadata": False}, # examples
    "8ad58157-ca74-4ae4-8f37-a0193430bcac": {"dark": True},
    "f165fe81-1956-4347-81ca-3a3b198f3f23": {"light": False},
    "9bd58157-ca74-4ae4-8f37-a0193431bcba": {"count": "100"},
    "62daaa33-195e-4faf-be5b-8335a239beb6": {"taken_at": "2024-02-24"}
}

# Update the storage items based on the dictionary
for item_uuid, metadata in updates.items():
    item = user_client.get_storage_item(item_uuid=item_uuid)
    item.update(client_metadata=metadata)

Import custom metadata to all data units in a Folder

This code allows you to update ALL custom metadata on ALL data units in a Folder in Index. This code OVERWRITES all existing custom metadata on a data unit.

# Import dependencies
from encord import EncordUserClient
from encord.orm.storage import StorageFolder, StorageItem, StorageItemType, FoldersSortBy

# Authentication
SSH_PATH = "<file-path-to-ssh-private-key-file>"
FOLDER_HASH = "<unique-folder-id>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

folder = user_client.get_storage_folder(FOLDER_HASH)
items = folder.list_items()

for item in items:
     item.update(client_metadata={"metadata": "value", "metadata": "value"})

# Import dependencies
from encord import EncordUserClient
from encord.orm.storage import StorageFolder, StorageItem, StorageItemType, FoldersSortBy

# Authentication
SSH_PATH = "/Users/chris-encord/sdk-ssh-private-key.txt"
FOLDER_HASH = "2a838557-d6f4-4408-a980-64246dc5c56b"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

folder = user_client.get_storage_folder(FOLDER_HASH)
items = folder.list_items()

for item in items:
     item.update(client_metadata={"dark": True, "captured_at": "2024-02-24"})

Update custom metadata to specific data units in a folder

This code allows you to update custom metadata on specific data units in Index. This code DOES NOT OVERWRITE all existing custom metadata on a data unit. It does overwrite custom metadata with existing values and adds new custom metadata to the data unit.

# Import dependencies
from encord import EncordUserClient

# Authentication
SSH_PATH = "<private_key_path>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

# Define a dictionary with item UUIDs and their respective metadata updates
updates = {
    # "<data-unit-id>": {"metadata": "metadata-value"},
    # "<data-unit-id>": {"metadata": False},
    # "<data-unit-id>": {"metadata": "metadata-value"},
    # "<data-unit-id>": {"metadata": True}
}

# Update the storage items based on the dictionary
for item_uuid, metadata_update in updates.items():
    item = user_client.get_storage_item(item_uuid=item_uuid)

    # make a copy of the current metadata and update it with the new metadata
    curr_metadata = item.client_metadata.copy()
    curr_metadata.update(metadata_update)

    # update the item with the new metadata
    item.update(client_metadata=curr_metadata)

# Import dependencies
from encord import EncordUserClient

# Authentication
SSH_PATH = "/Users/chris-encord/sdk-ssh-private-key.txt"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)


# Define a dictionary with item UUIDs and their respective metadata updates
updates = {
    # "<data-unit-id>": {"metadata": "metadata-value"},
    # "<data-unit-id>": {"metadata": False}, # examples
    "8ad58157-ca74-4ae4-8f37-a0193430bcac": {"dark": True},
    "f165fe81-1956-4347-81ca-3a3b198f3f23": {"light": False},
    "9bd58157-ca74-4ae4-8f37-a0193431bcba": {"count": "100"},
    "62daaa33-195e-4faf-be5b-8335a239beb6": {"taken_at": "2024-02-24"}
}

# Update the storage items based on the dictionary
for item_uuid, metadata_update in updates.items():
    item = user_client.get_storage_item(item_uuid=item_uuid)

    # make a copy of the current metadata and update it with the new metadata
    curr_metadata = item.client_metadata.copy()
    curr_metadata.update(metadata_update)

    # update the item with the new metadata
    item.update(client_metadata=curr_metadata)

Bulk custom metadata import to all data units in a Folder

This code allows you to update custom metadata on all data units in a Folder in Index. This code OVERWRITES all existing custom metadata on a data unit.

Using bundle allows you to update up to 1000 label rows at a time.

# Import dependencies
from encord import EncordUserClient
from encord.http.bundle import Bundle
from encord.orm.storage import StorageFolder, StorageItem, StorageItemType, FoldersSortBy

# Authentication
SSH_PATH = "<ssh-private-key>"
FOLDER_HASH = "<unique-folder-id>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

folder = user_client.get_storage_folder(FOLDER_HASH)
items = folder.list_items()

# Use the Bundle context manager
with Bundle() as bundle:
    for item in items:
        # Update each item with client metadata
        item.update(client_metadata={"metadata-1": "value", "metadata-2": False}, bundle=bundle)

Bulk custom metadata import on specific data units

This code allows you to update custom metadata on specific data units in a Folder in Index. This code DOES NOT OVERWRITE existing custom metadata on a data unit. It does overwrite custom metadata with existing values and adds new custom metadata to the data unit.

Using bundle allows you to update up to 1000 label rows at a time.

# Import dependencies
from encord import EncordUserClient
from encord.http.bundle import Bundle
from encord.orm.storage import StorageFolder, StorageItem, StorageItemType, FoldersSortBy

# Authentication
SSH_PATH = "<ssh-private-key>"
FOLDER_HASH = "<unique-folder-id>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

folder = user_client.get_storage_folder(FOLDER_HASH)
updates = {
    # "<data-unit-id>": {"metadata-1": "metadata-value"},
    # "<data-unit-id>": {"metadata-2": False},
    # "<data-unit-id>": {"metadata-1": "metadata-value"},
    # "<data-unit-id>": {"metadata-2": True}
}

# Use the Bundle context manager
with Bundle() as bundle:
    for storage_item in folder.list_items():
        # Update each item with client metadata
        update = updates[storage_item.uuid]
        storage_item.update(client_metadata=update, bundle=bundle)