Skip to main content

Working with datasets

Use the 'Datasets' tab in the Navigation bar to manage data across all your projects.

This is where you create, filter and manage datasets.

  • + New dataset button: this will initiate the dataset creation flow
  • Search bar: typing in the search bar will filter the datasets by title
  • List of datasets: clicking on a dataset item will take you to the Dataset dashboard for managing that dataset

Edit and view​

Clicking a dataset on the 'list and search' page takes you to the 'Edit and view' page.

This is where you manage your dataset. For example, you can upload new files, delete files you no longer need, view which projects and collaborators access this data etc. The dashboard is split into a 'Data' tab and 'Settings' tab

Data tab​

Use the data tab to upload more, and manage existing files.

Upload files​

You can upload new files on the left of the screen

  • Click the upload rectangle or drag video files into it to add your video files.
  • Click on the Image upload button to launch the 'New image group' pop-up to do the same for image files.

Manage files​

You can manage files under the 'Dataset data' heading on the right of the screen

  • Click the pencil icon to edit the filename
  • Click the Cloud download button to download the file to your local PC
  • Click on the checkbox next to a file name to select the file
  • Select an existing file to delete it
  • Select an existing video file to re-encode it

Settings tab​

Team​

The 'Team' pane shows a list of collaborators on the dataset.

Invite collaborators by clicking the + Invite collaborators button and adding their emails. New collaborators assume the 'Viewer' role by default. A 'Viewer' cannot make changes to the dataset, only an 'Admin' can. Collaborators can be upgraded to an 'Admin' using the 3 dots to the right of their name. Please note, once changed to an 'Admin', they cannot be reverted to a 'Viewer'. To do so you must delete and re-invite them. To delete a collaborator, use the bin icon.

Projects​

The 'Projects' pane shows a list of projects using the dataset.

Click on View to navigate to that project.

API access​

The 'API access' pane shows a list of API keys that have been generated for that dataset.

API keys are required to access resources such as projects and datasets using the Encord SDK or Encord API and are tied to specific projects or datasets if required. You can generate multiple keys for each project or dataset.

To create an API key:

  1. Click on the + New API key button
  2. Select the relevant read/write permissions
  3. Click the Create button
  4. Your new API key will be displayed, along with the dataset ID. Make a copy of these as the API key will only be displayed once

Danger zone​

The 'Danger zone' pane is where you can permanently delete your dataset.

Please note, once a dataset is deleted it cannot be retrieved.

Deleting files and datasets​

  • To delete a file from your dataset, simply select it from the list and click the Delete button that becomes available.
  • To delete the entire dataset you must navigate to Settings and select the Danger Zone tab. Here you will find the Delete Dataset button.
note

Deleting a dataset can't be undone. Please ensure you want to perform this action before continuing.

Data Optimization​

Re-encode your data​

If we detect that the video you uploaded can't be added to a dataset, it will have to be re-encoded. This simply means bringing your video in line with our requirements to avoid frame synchronization issues and ensure your labels are as accurate as possible.

Your video needs re-encoding if you see after successfully uploading your video. Hover over the icon to read what issues we have found with the video.

To automatically re-encode your video, simply select the file in question and click the Re-encode (auto) button to prevent any frame synchronization issues from arising. A notification telling you your video is re-encoded should appear at the top of your screen.

When in doubt, you can always check the progress of any process by clicking the icon. The number displayed in red signifies how many new notifications you have.

note

The way in which individual frames within different browsers are displayed may vary. To avoid browser-related frame synchronization issues we strongly recommend the Google Chrome browser when using our platform.

To learn more about frame synchronization issues and re-encoding your data please visit the relevant section on our Supported videos page.

Supported data formats​

We support all the major video and image formats, however not all video formats are supported by all browsers for playback. If you have trouble with playback in the 'Label editor' there may be an issue with your browser. We highly recommend using Google Chrome when using the Encord platform. If issues persist, refer to your browser documentation for more information.

For video data, the Label editor expects integer frame rates from 10-30 fps. Non-integer frame rates will generate a warning in the Label editor. Although labeling can continue with non-integer frame rates, please be aware this can introduce errors into your annotations.

For more information on supported video formats please see the Supported videos page.