ℹ️

Note

This documentation is only relevant for customers with early access to Encord Storage. Contact [email protected] to learn more and gain access.

Storage is your one-stop shop for everything related to data in Encord - offering robust cloud storage capabilities along with intuitive options for managing, organizing, and seamlessly integrating your data into Datasets.

ℹ️

Note

Data in Storage is structured into folders and subfolders. Data cannot be added directly to Storage, and must be placed within folders.

Standard folders vs synced folders

There are two types of folders in Storage:

  • Standard folders are folders to store your data and keep it organized. Data in standard folders can be added to multiple Datasets.
  • Synced folders are always synced to Datasets.
    • Creating a synced folder automatically creates a Dataset with the same name.
    • Any data added to a synced folder also appears in the Dataset.
    • Any data added directly to the Dataset also appears in the synced folder.

Upload data

Data can be added to existing or newly created folders. To add data to a folder:

  1. Navigate to Storage under the Index heading in the Encord platform.
  2. Click the + New folder button to create a new folder. A dialog to create a new folder appears. Alternatively, double-click an existing folder to open it.
  1. Give the folder a meaningful name and description. Enable the toggle to create a synced folder.
  1. Click Create to create the folder.
    The folder is listed in the Storage section.

  2. Right-click the folder and select Add data. Alternatively, double click the folder to open it and then click the +Add data button. A dialog appears.

  1. Select the type of data you want to add:
  • A - Upload: Drag and drop local images and / or videos. Click Upload to finish.
  • B - Batch images as: Create an image group or image sequence from local images. See the section on creating image groups and image sequences for more information. Click Upload and batch images to finish.
  • C - DICOM: Create a DICOM series from local DICOM files. Click Upload to finish.
  • D - Import from private cloud: Add any data stored in your cloud storage. At least one data integration is required to upload cloud data. Learn how to upload private cloud data here. Click Import to finish.

Create image groups and image sequences

Image groups and image sequences are created when adding data to a folder in Storage.

  1. Open the Add data dialog.
  2. Select the Batch images as tab.
  3. Select Image group to create an image group. Select Image sequence to create an image sequence.
  1. Drag-and-drop the local images you want to batch into an image group or image sequence. Learn how create image groups and image sequences from cloud data here.

  2. Click Upload and batch images to create the image group or image sequence.

  3. A file representing the image group or image sequence is created in the folder. A new folder appears containing all the constituent images of the image group or image sequence that was created.

ℹ️

Note

  • The new folder containing the constituent images is not synced.
  • Adding new images to this folder does not add them to the image group or image sequence.
  • Files contained in the image group or image sequence cannot be deleted from the folder until the image group or image sequence is deleted.

Upload cloud data

To add data from your private cloud you need the following:

See our documentation on data integrations, and adding cloud data for comprehensive guides.


Add data to Datasets

After data is uploaded to Storage it can be added to one or multiple Datasets.

👍

Tip

Creating a synced folder automatically creates a synced Dataset with the same name. All files added to the synced folder also appear in the Dataset.

  1. Locate and select the data you want to add to a Dataset.

👍

Tip

To add multiple files to a Dataset, bulk-select files by selecting the first file, and then selecting the last file while holding the Shift key on your keyboard.

  1. Right-click your selection and select Add to dataset.
    A dialog appears.
  1. You have the option to:
  • Add data to an existing Dataset by clicking Add next to the Dataset's name. You add data to multiple Datasets.
  • Create a new Dataset to add your data to. Select the +New dataset tab and give the Dataset a meaningful name and description. Enable the toggle to sync the new Dataset with the storage folder. Click Add to new dataset to add the files to the new Dataset.

View, search, and manage data

Storage allows you to search for data units, and view which Datasets each data unit is belongs to.

The global search bar searches all data in Storage, irrespective of which folder is currently open.

Click a data unit to see file specific details. Navigate to the Linked to tab to see which Datasets this file is added to. Click the icon next to the Dataset to:

  • View the Dataset.
  • Remove the selected file from Datasets.

Delete or move data and folders

Right-click a file, folder, or a selection of multiple files to see the options available to you.

  • To delete files and folders, right-click the selection and select Delete.

🚧

Caution

Deleting a file removes it from all Datasets it was added to.

  • To move files and folders to a different folder, right-click the selection and select Move to another folder. A dialog appears. Start typing the name of the folder, select the folder you want to move the selection to, and click Move to confirm.

🚧

Caution

Moving a file out of a synced folder removes it from the corresponding synced Dataset.

Folder permissions and activity

Folder details are automatically displayed whenever a folder is:

  • Selected
  • Open and no data unit is selected

Manage folder collaborators

Click Manage access to manage which users within your Organization can access the folder. A dialog appears with the the Invite tab open by default.

Select the role you want the users to have, and then start typing their email address into the search bar. Select the user(s) you want to give access to the folder and click Add to confirm your selection.

To delete a user from a folder, navigate to the Manage tab and click the button.

Change a user's access permissions by selecting a new role from the dropdown.

ℹ️

Note

Users with Admin permissions cannot be deleted, and their role cannot be changed.