Registering your cloud data using Cloud-synced folders provides you with a very quick way of getting up and running.

Custom Metadata needs to be uploaded separately if you use this method to register your data with Encord.

​
File Support and Limits

  • Cloud-synced folders support images, videos, PDFs, text, HTML, and audio files.
Image groups, image sequences, and DICOM series are not currently supported by Cloud-synced folders.
  • A single cloud synced folder can contain a maximum of 10 million files.

​
Register Cloud Data using Cloud-synced Folders

​
STEP 1: Create an Encord Integration

At least one data integration is required to register cloud data to Encord. Encord can integrate with the following cloud service providers:

​
STEP 2: Create a Cloud-synced Folder

  1. Go to Index > Files.

  2. Click New folder > Cloud-synced folder. The New Cloud-synced folder dialog appears.

    Create Cloud-synced Folder

  3. Provide the following:

    • Title: Provide a meaningful name for the Cloud-synced folder.
    • Description: OPTIONAL - Provide a meaningful description for the Cloud-synced folder.
    • Select your integration: Select the integration to use from the drop down.
    • Storage path: Specify the storage/file path to your cloud storage. For example: gs://encord-gcp-bucket/CloudSync/ or s3://encord-aws-bucket/CloudSync.

  4. Click Test to verify that Encord can commincate with your cloud storage.

  5. Click Create. The page for the new Cloud-synced folder appears.

​
Find Storage Path

Finding the Storage path for your folder or object varies across Cloud Storage platforms.

AWS

Find AWS storage path

GCP

Find GCP storage path

​
STEP 3: Sync Data Between Encord and Cloud Storage

  1. Go to Index > Files > [name of a Cloud-synced folder]. The Cloud-synced folder page appears.

    Sync your Data

  2. Click Initiate sync. The sync between the folder and your cloud storage begins.

​
Resync Data Between Encord and Cloud Storage

As you add or remove data from your cloud storage you need to resync the Cloud-synced folder with your cloud storage to keep it up to date.

  • You cannot change URI after folder creation.
  • Resyncing data to your Cloud-synced folder scans your bucket for any changes. New files in your cloud storage import to your Cloud-synced folder. Deleted files in your cloud storage are soft deleted in your Cloud-synced folder.

Soft deleted means that the data is not removed physically but is no longer visible to users. Soft deleted data is still available in any Projects where the data resides. Labels can be exported from soft deleted data.

  1. Go to Index > Files > [name of a Cloud-synced folder]. The Cloud-synced folder page appears.

    Resync your data

  2. Click Resync. The resync between the folder and your cloud storage begins.