Upload Cloud Data

Videos

Each object in the videos array is a JSON object with the key objectUrl specifying the full URL of where to find the video resource. The title field is optional. If omitted, the video file path and name are used as the default title. For example, if the file is located at https://encord-solutions-bucket.s3.eu-west-2.amazonaws.com/path/to/my/bucket/video23.mp4, the title defaults to /path/to/my/bucket/video23.mp4.

Video metadata

The JSON format allows you to specify videoMetadata for video files. videoMetadata is essential information used by the Label Editor and is crucial for aligning annotations to the correct frame.

{
    "videos": [
      {
        "objectUrl": "video_file.mp4",
        "videoMetadata": {
            "fps": 23.98,
            "duration": 29.09,
            "width": 1280,
            "height": 720,
            "file_size": 5468354,
            "mime_type": "video/mp4"
        }
      }
    ]
}

• fps: Frames per second.
• duration: Duration of the video (in seconds).
• width / height: Dimensions of the video (in pixels).
• file_size: The size of the file (in bytes).
• mime_type: Specifies the file type extension according to the MIME standard.

When videos are supplied with video metadata, Encord assumes the metadata to be correct and our servers will neither download nor pre-process your data. This may be a particularly useful feature for customers with strict data compliance concerns.

One way to find the necessary metadata is shown below. Run the following commands in your terminal.

• ffmpeg -i 'video_title.mp4' to retrieve fps, duration, width, and height - as highlighted below.

• ls -l 'video_title.mp4' to retrieve the file size - as highlighted below.

Audio Files

Each object in the audio file array is a JSON object with the key objectUrl specifying the full URL of where to find the audio resource. The title field is optional. If omitted, the audio file path and name are used as the default title. For example, if the file is located at https://encord-solutions-bucket.s3.eu-west-2.amazonaws.com/path/to/my/bucket/song23.mp3, the title defaults to /path/to/my/bucket/song23.mp3.

• Audio metadata is distinct from custom metadata.

• clientMetadata allows you to add metadata that can be used for filtering your data in Index. You can use text to import transcripts for your audio file.

Audio metadata

The JSON format allows you to specify audioMetadata for audio files. This is optional information.

{
    "audio": [
      {
        "objectUrl": "audio_file.mp3",
        "audioMetadata": {
            "duration": 23.98,
            "file_size": 2900000,
            "mime_type": "audio/mp3",
            "sample_rate": 44100,
            "bit_depth": 24,
            "codec": "mp3",
            "num_channels": 2
        }
      }
    ]
}

• duration_seconds: float - Audio duration in seconds.
• file_size: int - Size of the audio file in bytes.
• mime_type: str - MIME type of the audio file (for example: audio/mpeg or audio/wav).
• sample_rate: int - Sample rate (int) in Hz.
• bit_depth: int - Size of each sample (int) in bits.
• codec: str - Codec (for example: mp3, pcm).
• num_channels: int - Number of channels.

Single Images

The JSON structure for single images parallels that of videos. The title field is optional. If omitted, the image file path and name are used as the default title. For example, if the file is located at https://encord-solutions-bucket.s3.eu-west-2.amazonaws.com/path/to/my/bucket/image23.mp4, the title defaults to /path/to/my/bucket/image23.jpg.

Image metadata

The JSON format allows you to specify imageMetadata for image files. imageMetadata contains essential information used by the Label Editor and is crucial for aligning annotations to the correct image properties.

{
  "images": [
    {
      "objectUrl": "s3://my_image.jpg",
      "imageMetadata": {
        "mimeType": "image/jpg",
        "fileSize": 124,
        "width": 640,
        "height": 480
      }
    }
  ]
}

Image groups

• Image groups are collections of images that are processed as one annotation task.
• Images within image groups remain unaltered, meaning that images of different sizes and resolutions can form an image group without the loss of data.
• Image groups do not require ‘write’ permissions to your cloud storage.
• Custom client metadata is defined per image group, not per image. See our documentation here to learn how to add clientMetadata to images in an image group.

Image sequences

• Image sequences are collections of images that are processed as one annotation task and represented as a video.
• Images within image sequences may be altered as images of varying sizes are resolutions are made to match that of the first image in the sequence.
• Creating Image sequences from cloud storage requires ‘write’ permissions, as new files have to be created in order to be read as a video.
• Each object in the image_groups array with the createVideo flag set to true represents a single image sequence.
• Custom client metadata is defined per image sequence, not per image.

DICOM

• Each dicom_series element can contain one or more DICOM series.
• Each series requires a title and at least one object URL, as shown in the following example.

The following is an example JSON for uploading three DICOM series belonging to a study. Each title and object URL correspond to individual DICOM series.

• The first series contains only a single object URL, as it is composed of a single file.
• The second series contains 3 object URLs, as it is composed of three separate files.
• The third series contains 2 object URLs, as it is composed of two separate files.

NIfTI

Each series requires a title and at least one object URL.

The following is an example JSON file for uploading two NIfTI files to Encord.

Videos

A CSV file containing videos should contain two columns with the following mandatory column headings:
‘ObjectURL’ and ‘Video title’. All headings are case-insensitive.

• The ‘ObjectURL’ column containing the objectUrl. This field is mandatory for each file, as it specifies the full URL of the video resource.

• The ‘Video title’ column containing the video_title. If left blank, the original file name is used.

In the example below files 1, 2 and 4 will be assigned the names in the title column, while file 3 will keep its original file name.

A CSV file containing single images should contain two columns with the following mandatory headings:
‘ObjectURL’ and ‘Image title’. All headings are case-insensitive.

• The ‘ObjectURL’ column containing the objectUrl. This field is mandatory for each file, as it specifies the full URL of the image resource.

• The ‘Image title’ column containing the image_title. If left blank, the original file name is used.

In the example below files 1, 2 and 4 will be assigned the names in the title column, while file 3 will keep its original file name.

Image groups

A CSV file containing image groups should contain three columns with the following mandatory headings:
‘ObjectURL’, ‘Image group title’, and ‘Create video’. All three headings are case-insensitive.

• The ‘ObjectURL’ column containing the objectUrl. This field is mandatory for each file, as it specifies the full URL of the resource.

• The ‘Image group title’ column containing the image_group_title. This field is mandatory, as it determines which image group a file will be assigned to.

In the example below the first two URLs are grouped together into ‘Group 1’, while the following two files are grouped together into ‘Group 2’.

Image sequences

A CSV file containing image sequences should contain three columns with the following mandatory headings: ‘ObjectURL’, ‘Image group title’, and ‘Create video’. All three headings are case-insensitive.

• The ‘ObjectURL’ column containing the objectUrl. This field is mandatory for each file, as it specifies the full URL of the resource.

• The ‘Image group title’ column containing the image_group_title. This field is mandatory, as it determines which image sequence a file will be assigned to. The dimensions of the image sequence are determined by the first file in the sequence.

• The ‘Create video’ column. This can be left blank, as the default value is ‘true’.

In the example below the first two URLs are grouped together into ‘Sequence 1’, while the second two files are grouped together into ‘Sequence 2’.

A CSV file containing DICOM files should contain two columns with the following mandatory headings: ‘ObjectURL’ and ‘Dicom title’. Both headings are case-insensitive.

• The ‘ObjectURL’ column containing the objectUrl. This field is mandatory for each file, as it specifies the full URL of the resource.

• The ‘Series title’ column containing the dicom_title. When two files are given the same title they are grouped into the same DICOM series. If left blank, the original file name is used.

In the example below the first two files are grouped into ‘dicom series 1’, the next two files are grouped into ‘dicom series 2’, while the final file will remain separated as ‘dicom series 3’.

Multiple file types

You can upload multiple file types with a single CSV file by using a new header each time there is a change of file type. Three headings will be required if image sequences are included.

The example below shows a CSV file for the following:

• Two image sequences composed of 2 files each.
• One image group composed of 2 files.
• One single image.
• One video.

Import while importing images

This JSON file imports embeddings while importing your data into Index from a cloud integration.

Import specific images

The custom embeddings format for images follows the same format as importing custom metadata.

# Import dependencies
from encord import EncordUserClient
from encord.http.bundle import Bundle

# Authentication
SSH_PATH = "<file-path-to-ssh-private-key>"

# Authenticate with Encord using the path to your private key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
)

# Define a dictionary with item UUIDs and their respective metadata updates
updates = {
    "<data-ID-1>": {"<my-embedding>": [1.0, 2.0, 3.0]},
    "<data-ID-2>": {"<my-embedding>": [1.0, 2.0, 3.0]}
}

# Use the Bundle context manager
with Bundle() as bundle:
    # Update the storage items based on the dictionary
    for item_uuid, metadata_update in updates.items():
        item = user_client.get_storage_item(item_uuid=item_uuid)

        # Make a copy of the current metadata and update it with the new metadata
        curr_metadata = item.client_metadata.copy()
        curr_metadata.update(metadata_update)

        # Update the item with the new metadata and bundle
        item.update(client_metadata=curr_metadata, bundle=bundle)

Import while importing videos

This JSON file imports embeddings while importing your data into Index from a cloud integration.

config is optional when importing your custom embeddings:

"config": {
    "sampling_rate": "<samples-per-second>",
    "keyframe_mode": "frame" or "seconds",
},

If config is not specified, the sampling_rate is 1 frame per second, and the keyframe_mode is frame.

Update specific videos

AWS S3 object URLs can follow a few set patterns:

• Virtual-hosted style: https://<bucket-name>.s3.<region>.amazonaws.com/<key-name>
• Path-style: https://s3.<region>.amazonaws.com/<bucket-name>/<key-name>
• S3 protocol: S3://<bucket-name>/<key-name>
• Legacy: those without regions or those with S3-<region> in the URL

AWS best practice is to use Virtual-hosted style. Path-style is planned to be deprecated and the legacy URLs are already deprecated.

We support Virtual-hosted style, Path-style and S3 protocol object URLs. We recommend you use Virtual-hosted style object URLs wherever possible.

Object URLs can be found in the Properties tab of the object in question. Navigate to AWS S3 > bucket > object > Properties to find the Object URL.

Here’s a python script which creates a JSON file for single images by constructing the URLs from the specified path in a given S3 bucket. You’ll need to configure the following variables to match your setup.

1. region: the AWS region where your S3 bucket is.
2. aws_profile: the name of the profile in the AWS ~/.aws/credentials file. See AWS Credentials Documentation to properly set up the credentials file.
3. bucket_name: the name of your S3 bucket you want to pull files from.
4. s3_directory: the path to the directory in the S3 bucket where your files are stored.

In this Amazon S3 Virtual-hosted style URLs example, my-bucket is the bucket name, us-west-2 is the region, and images/dogs is the S3 directory:

https://my-bucket.s3.us-west-2.amazonaws.com/images/dogs/puppy.png

And the script itself:

import boto3
import json
from pathlib import Path
region = 'FILL_ME_IN'
aws_profile = 'FILL_ME_IN'
bucket_name = 'FILL_ME_IN'
s3_directory = Path('FILL_ME_IN')
root_url = f'https://{bucket_name}.s3.{region}.amazonaws.com'
session = boto3.Session(profile_name=aws_profile)
s3 = boto3.resource('s3')
bucket = s3.Bucket(bucket_name)
images = []
for object_summary in bucket.objects.filter(Prefix=s3_directory.as_posix() + '/'):
    object_file = Path(object_summary.key)
    if object_file.parent == s3_directory:
        object_url = f'{root_url}/{object_summary.key}'
        images.append({'objectUrl': object_url})
data_dict = {'images': images}
output_file = Path('upload_images.json')
output_file.write_text(json.dumps(data_dict, indent=2))