Keyframes in Index

ℹ️

Note

Before reading any further, we recommend familiarizing yourself with custom metadata.

keyframes is a reserved keyword, in custom metadata, for use with frames of interest in videos. Specifying keyframes on specific frames ensures that those frames import into Index and Active. That means frames specified using keyframes are available to filter your frames and for calculating embeddings on your data.


 client_metadata = {
     "keyframes": [<frame_number>, <frame_number>, <frame_number>, <frame_number>, <frame_number>]
}
client_metadata = {
    "keyframes": [13, 17, 19, 23, 127, 149, 307, 557]
}

You can include keyframes while importing your videos or after you import your videos.

Import keyframes to specific data units in a folder

This code allows you to import keyframes on specific videos in Index. This code DOES NOT OVERWRITE all existing custom metadata on a data unit. It does overwrite custom metadata with existing values and adds new custom metadata to the data unit.

# Import dependencies
from encord import EncordUserClient

# Authentication
SSH_PATH = "<private_key_path>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

# Define a dictionary with item UUIDs and their keyframes updates
updates = {
    "<data-unit-id>": {"keyframes": [<frame_number>, <frame_number>, <frame_number>, <frame_number>, <frame_number>]},
    "<data-unit-id>": {"keyframes": [<frame_number>, <frame_number>, <frame_number>, <frame_number>, <frame_number>]},
    "<data-unit-id>": {"keyframes": [<frame_number>, <frame_number>, <frame_number>, <frame_number>, <frame_number>]},
    "<data-unit-id>": {"keyframes": [<frame_number>, <frame_number>, <frame_number>, <frame_number>, <frame_number>]}
}

# Update the storage items based on the dictionary
for item_uuid, metadata_update in updates.items():
    item = user_client.get_storage_item(item_uuid=item_uuid)

    # make a copy of the current metadata and update it with the new metadata
    curr_metadata = item.client_metadata.copy()
    curr_metadata.update(metadata_update)

    # update the item with the new metadata
    item.update(client_metadata=curr_metadata)
# Import dependencies
from encord import EncordUserClient

# Authentication
SSH_PATH = "/Users/chris-encord/sdk-ssh-private-key.txt"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

# Define a dictionary with item UUIDs and their keyframes updates
updates = {
    "8ad58157-ca74-4ae4-8f37-a0193430bcac": {"keyframes": [13, 17, 19, 23, 127, 149, 307, 557]},
    "f165fe81-1956-4347-81ca-3a3b198f3f23": {"keyframes": [10, 110, 210, 320, 430]},
    "9bd58157-ca74-4ae4-8f37-a0193431bcba": {"keyframes": [1087, 1549, 2029, 3527]},
    "62daaa33-195e-4faf-be5b-8335a239beb6": {"keyframes": [1000, 2001, 3241, 4637, 5431]}
}

# Update the storage items based on the dictionary
for item_uuid, metadata_update in updates.items():
    item = user_client.get_storage_item(item_uuid=item_uuid)

    # make a copy of the current metadata and update it with the new metadata
    curr_metadata = item.client_metadata.copy()
    curr_metadata.update(metadata_update)

    # update the item with the new metadata
    item.update(client_metadata=curr_metadata)

Filter by keyframes

Index supports filtering video frames by keyframes. This speeds up curation of your videos by focusing on frames of interest within your videos.

To filter data by keyframes:

  1. Log in to the Encord platform.
    The landing page for the Encord platform appears.

  2. Go to Index > Files.
    The All folders page appears with a list of all folders in Encord.

  3. Click in to a folder.
    The landing page for the folder appears and the Explorer button is enabled.

  4. Click the Explorer button.
    The Index Explorer page appears.

  5. Click Filter.
    The Filter tab displays.

  6. Click Add filter.
    A menu appears.

  7. Select Keyframe from the menu.
    The Keyframe filter appears in the filter list.

  8. Select True.
    The video frames wtih keyframes specified appear in the Explorer workspace.

Browse videos by keyframes

You can use keyframes to sort your data in the Frames Grid view, but you can also use keyframes to filter the frames displayed in the Videos view.

To filter videos by keyframes:

  1. Log in to the Encord platform.
    The landing page for the Encord platform appears.

  2. Go to Index > Files.
    The All folders page appears with a list of all folders in Encord.

  3. Click in to a folder.
    The landing page for the folder appears and the Explorer button is enabled.

  4. Click the Explorer button.
    The Index Explorer page appears.

  5. Click Video.
    All videos in the Folder/Folders appear in a list.

  6. Toggle the Show only Keyframes switch.
    Only video frames with keyframes appear when moving through the video.