Data integrations

Suggest Edits

Create and manage data integrations from the Integrations section. Click Add integration to create a new integration.

Encord can integrate with the following cloud storage providers:

Sharing cloud integrations

Sharing cloud integrations allows other users to use the integration to upload data to the Encord platform. All integrations can be shared with other users with the following procedure, regardless of the cloud service provider the integration comes from.

To share an integration:

  1. Click the Share integration button on the integration.
  2. Type the emails of the users you want to share the integration with.
  3. Click Share.

Strict client-only access

Strict client-only access allows Encord to sign URLs without temporarily processing information on our servers. Strict client-only access is available for all types of private cloud integrations, including Direct access.

👍

Tip

For enhanced security, create a Direct Access integration with Strict client-only access. This combination ensures the highest level of data protection: Strict client-only access prevents Encord from accessing your data, while Direct Access keeps Encord from signing the URLs, maintaining your data's confidentiality.

There are two main consequences of making an integration client-only access:

  1. All features that rely on any kind of data conversion will not work. This includes features such as:
  1. Metadata must be specified for all data in the JSON file when adding data to a dataset. The video metadata section shows how to specify the metadata.

Enabling Strict client-only access

Strict client-only access is enabled by checking the Strict client-only access, server-side media features will not be available checkbox in the Advanced settings section when creating an integration.

A green tick and the words Client-only access are visible on all client-only access integrations.

Video metadata

The JSON format allows you to specify videoMetadata for video files. videoMetadata is essential information used by the Label Editor and is crucial for aligning annotations to the correct frame.

❗️

CRITICAL INFORMATION

When the videometadata flag is included in the JSON file, we directly use the supplied metadata without performing any additional validation, and do not store the file on our servers. To guarantee accurate labels, it is crucial that the metadata you provide is accurate.

ℹ️

Note

videoMetadata must be specified when a Strict client-only access integration is used. In all other cases videoMetdata is optional.
👍 Tip

Encord assumes the videoMetadata is correct and our servers do not download or pre-process your data. This is particularly useful for customers with strict data compliance concerns.

ℹ️

Note

videoMetadata is distinct from clientMetadata. videoMetadata is required for videos when using Strict client-only access, while clientMetadata is optional and can be included for all types of data.

One way to find the required metadata is shown below. Run the following commands in your terminal.

  • ffmpeg -i 'video_title.mp4' to retrieve fps, duration, width, and height:
  • ls -l 'video_title.mp4' to retrieve the file size:

Example JSON including video metadata 
{
    "videos": [
      {
        "objectUrl": "video_file.mp4",
        "videoMetadata": {
            "fps": 23.98,
            "duration": 29.09,
            "width": 1280,
            "height": 720,
            "file_size": 5468354,
            "mime_type": "video/mp4"
        }
      }
    ]
  }
  • fps: Frames per second.
  • duration: Duration of the video (in seconds).
  • width / height: Dimensions of the video (in pixels).
  • file_size: The size of the file (in bytes).
  • mime_type: Specifies the file type extension according to the MIME standard.

Custom signed URL expiration time

A custom signed URL expiration time allows you to set the duration that an integration's pre-signed URL remains valid.

ℹ️

Note

Custom signed URL expiration times are only available for AWS, GCP, and OTC integrations.

Shorter signed URL expiration times are more secure, but can negatively impact the annotation and review experiences in the Label Editor. Ensure that the URL expiration time exceeds the amount of time that files from the integration take to annotate and review. The default value for signed URL expiration times when creating an integration is 1 week.

Setting custom signed url expiration times

To set up a custom signed URL expiration time for a new integration:

  1. Click the Custom signed URL integration time checkbox in the Advanced settings when setting up your GCP, AWS, or OTC integration.

  2. Set the duration for which pre-signed URLs should remain valid. We recommend configuring the URL expiration time to exceed the time required to annotate and review files from the integration in the Label Editor.

Edit custom signed url expiration times

ℹ️

Note

Custom signed URL expiration times are only available for AWS, GCP, and OTC integrations. For Azure integrations,expired SAS tokens must be updated regularly.

To change the custom signed URL expiration time for an existing integration:

  1. Click the icon on your integration.

  2. Select Update Signed Url Expiration.

  1. Set the duration for which presigned URLs should remain valid. We recommend configuring the URL expiration time to exceed the time required to annotate and review files from the integration in the Label Editor.

Updated 6 days ago