> ## Documentation Index > Fetch the complete documentation index at: https://docs.encord.com/llms.txt > Use this file to discover all available pages before exploring further. # Getting Started with Data > Explore ## Data Management and Curation ![Data > Explore Get Started](https://storage.googleapis.com/docs-media.encord.com/static/img/index-data-curation-management-2.png) ## STEP 1: Register Cloud Data with Encord ### Create a Cloud Integration Select your cloud provider.
Clickable Div
AWS Integration GCP Integration Azure Integration OTC Integration Wasabi Integration Oracle Integration Direct Access Integration
*** ### Create a Folder You must create a folder in Data > Explore to store your files. 1. Navigate to **Data** > **Files & Folders** 2. Click the **+ New folder** button to create a new folder. Select the type of folder you want to create.
3. Give the folder a meaningful name and description. 4. Click **Create** to create the folder. The folder is listed in **Files & Folders**. *** ### Create JSON or CSV for Registration To register files from cloud storage with Encord, you must create a JSON or CSV file specifying the files you want to register. Find helpful scripts for creating JSON and CSV files for the cloud data registration process [here](/platform-documentation/Curate/add-files/index-register-cloud-data#helpful-scripts-and-examples). All types of data (videos, images, image groups, image sequences, and DICOM) from a private cloud are added to a Dataset in the same way, by using a JSON or CSV file. The file includes links to all images, image groups, videos and DICOM files in your cloud storage. For a list of supported file formats for each data type, go [here](/platform-documentation/General/general-supported-data) Encord supports file names up to 300 characters in length for any file or video for upload. Encord enforces the following upload limits for each JSON file used for file registration: * Up to 1 million URLs * A maximum of 500,000 items (e.g. images, image groups, videos, DICOMs) * URLs can be up to 16 KB in size Optimal upload chunking can vary depending on your data type and the amount of associated metadata. For tailored recommendations, contact Encord support. We recommend starting with smaller uploads and gradually increasing the size based on how quickly jobs are processed. Generally, smaller chunks result in faster data reflection within the platform. BEST PRACTICE: If you want to use Index or Active with your video data, we STRONGLY RECOMMEND using custom metadata (`clientMetadata`) to specify key frames, custom metadata, and custom embeddings. For more information go [here](/platform-documentation/Curate/custom-metadata/import-custom-metadata) or [here for information on using the SDK](/sdk-documentation/index-sdk/custom-metadata/sdk-import-custom-metadata). #### Create JSON file for Registration For detailed information about the JSON file format used for import go [here](/platform-documentation/Curate/add-files/index-register-cloud-data#json-format). The information provided about each of the following data types is designed to get you up and running as quickly as possible without going too deeply into the why or how. Look at the template for each data type, then the examples, and adjust the examples to suit your needs. If `skip_duplicate_urls` is set to `true`, all object URLs that exactly match existing images/videos in the dataset are skipped. #### AWS JSON #### [Videos](#videos) **Video Metadata** When the `videoMetadata` flag is present in the JSON file, we directly use the supplied metadata without performing any additional validation, and do not store the file on our servers. To guarantee accurate labels, it is crucial that the `videoMetadata` provided is accurate. ```json Template theme={"dark"} { "videos": [ { "objectUrl": "cloud-path-to-your-video-1" }, { "objectUrl": "cloud-path-to-your-video-2", "videoMetadata": { "fps": frames-per-second, "duration": duration-in-seconds, "width": frame-width, "height": frame-height, "file_size": file-size-in-bytes, "mime_type": "MIME-file-type-extension" } } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "videos": [ { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/videos/BerghouseLeopardJog.mp4" }, { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/videos/blue_bus_video.mp4" }, { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/videos/bluemlisalphutte_flyover.mp4" }, { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/videos/boat_lake_normalized.mp4" }, { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/videos/boat_lake.MP4" }, { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/videos/boat_lake.MP4" }, { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/videos/snow_sled.MOV" }, { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/videos/cyclists.MP4" } ], "skip_duplicate_urls": true } ``` ```json Video Metadata theme={"dark"} { "videos": [ { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/videos/video_file.mp4", "videoMetadata": { "fps": 23.98, "duration": 29.09, "width": 1280, "height": 720, "file_size": 5468354, "mime_type": "video/mp4" } } ], "skip_duplicate_urls": true } ``` #### [Audio Files](#audio-files) The following is an example JSON file for uploading two audio files to Encord. Waveform generation for long audio files can cause lag. To avoid this, generate the waveform offline and upload it with the audio file. For more information go [here](/platform-documentation/Curate/index-audio#long-audio-files). * **Template:** Imports audio files with an Encord title. * **Audio Metadata:** Imports one audio file with the `audiometadata` flag. When the `audiometadata` flag is present in the JSON file, we directly use the supplied metadata without performing any additional validation, and do not store the file on our servers. To guarantee accurate labels, it is crucial that the metadata you provide is accurate. ```json Template theme={"dark"} { "audio": [ { "objectUrl": "" }, { "objectUrl": "", "title": "my-custom-audio-file-title.mp3" } ], "skip_duplicate_urls": true } ``` ```json Audio Metadata theme={"dark"} { "audio": [ { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/videos/audio_file.mp3", "audioMetadata": { "duration": 23.98, "file_size": 2900000, "mime_type": "audio/mp3", "sample_rate": 44100, "bit_depth": 24, "codec": "mp3", "num_channels": 2 } } ], "skip_duplicate_urls": true } ``` #### [PDFs](#pdfs) The following are example JSON files for uploading PDFs to Encord. ```json Template theme={"dark"} { "pdfs": [ { "objectUrl": "" }, { "objectUrl": "", "title": "my-document-02.pdf" }, { "objectUrl": "", "title": "my-document-03.pdf", "pdfMetadata": { "fileSize": 300, "numPages": 5 } } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "pdfs": [ { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/pdfs/my-document-01.pdf", }, { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/pdfs/my-document-02.pdf", "title": "my-document-02.pdf" }, { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/pdfs/my-document-03.pdf", "title": "my-document-03.pdf", "pdfMetadata": { "fileSize": 300, "numPages": 5 } } ], "skip_duplicate_urls": true } ``` #### [Text Files](#text-files) The following are example JSON files for uploading text files to Encord. ```json Template theme={"dark"} { "text": [ { "objectUrl": "" }, { "objectUrl": "", "title": "my-file.html" }, { "objectUrl": "", "title": "my-plain-text-file.txt", "textMetadata": { "fileSize": 200, "mime_type": "text/plain" } }, { "objectUrl": "", "title": "my-csv-file.csv", "textMetadata": { "fileSize": 250, "mime_type": "text/csv" } }, { "objectUrl": "", "title": "my-xml-file.xml", "textMetadata": { "fileSize": 300, "mime_type": "application/xml" } }, { "objectUrl": "", "title": "my-html-file.html", "textMetadata": { "fileSize": 400, "mime_type": "text/html" } } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "text": [ { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/text/my-file-01.html" }, { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/text/my-plain-text-file-01.txt", "title": "my-plain-text-file-01.txt" }, { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/text/my-plain-text-file-02.txt", "title": "my-plain-text-file-02.txt", "textMetadata": { "fileSize": 200, "mime_type": "text/plain" } }, { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/text/my-csv-file-01.csv", "title": "my-csv-file-01.csv", "textMetadata": { "fileSize": 250, "mime_type": "text/csv" } }, { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/text/my-xml-file-01.xml", "title": "my-xml-file-01.xml", "textMetadata": { "fileSize": 300, "mime_type": "application/xml" } }, { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/text/my-html-file-01.html", "title": "my-html-file-01.html", "textMetadata": { "fileSize": 400, "mime_type": "text/html" } } ], "skip_duplicate_urls": true } ``` #### [Single Images](#single-images) For detailed information about the JSON file format used for import go [here](/platform-documentation/Curate/add-files/index-register-cloud-data#json-format). The JSON structure for single images parallels that of videos. **Template:** Provides the proper JSON format to import images into Encord. **Examples:** * **Data** Imports the images only. ```json Template theme={"dark"} { "images": [ { "objectUrl": "file/path/to/images/file-name-01.file-extension" }, { "objectUrl": "file/path/to/images/file-name-02.file-extension" }, { "objectUrl": "file/path/to/images/file-name-03.file-extension", "title": "image-title.file-extension" } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "images": [ { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/0001.jpg" }, { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/0002.jpg" }, { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/0003.jpg" }, { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/DALL%C2%B7E+2022-09-08+18.53.25+-+firefighter+extinguishing+flames+around+computer+in+software+office+overgrown+by+foliage.png" }, { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/DALL%C2%B7E+2022-12-08+22.16.52+-+steampunk+combustion+engine+that+is+fueled+by+data+and+produces+computer+vision+algorithms.png" }, { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/large_images/pexels-ivo-rainha-00057.png" } ] } ``` ```json Image Metadata theme={"dark"} { "images": [ { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/my_image.jpg", "imageMetadata": { "mimeType": "image/jpg", "fileSize": 124, "width": 640, "height": 480 } } ], "skip_duplicate_urls": true } ``` #### [Image groups](#image-groups) For detailed information about the JSON file format used for import go [here](/platform-documentation/Curate/add-files/index-register-cloud-data#json-format). * Image groups are collections of images that are processed as one annotation task. * Images within image groups remain unaltered, meaning that images of different sizes and resolutions can form an image group without the loss of data. * Image groups do NOT require 'write' permissions to your cloud storage. * If `skip_duplicate_urls` is set to `true`, all URLs exactly matching existing image groups in the dataset are skipped. The position of each image within the sequence needs to be specified in the key (`objectUrl_{position_number}`). **Template:** Provides the proper JSON format to import image groups into Encord. **Examples:** * **Data:** Imports the image groups only. ```json Template theme={"dark"} { "image_groups": [ { "title": "", "createVideo": false, "objectUrl_0": "file/path/to/images/file-name-01.file-extension", "objectUrl_1": "file/path/to/images/file-name-02.file-extension", "objectUrl_2": "file/path/to/images/file-name-03.file-extension", }, { "title": "<title 2>", "createVideo": false, "objectUrl_0": "file/path/to/images/file-name-01.file-extension", "objectUrl_1": "file/path/to/images/file-name-02.file-extension", "objectUrl_2": "file/path/to/images/file-name-03.file-extension" } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "image_groups": [ { "title": "Image group 01", "createVideo": false, "objectUrl_0": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/0001.jpg", "objectUrl_1": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/0002.jpg", "objectUrl_2": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/DALL%C2%B7E+2022-09-08+18.53.25+-+firefighter+extinguishing+flames+around+computer+in+software+office+overgrown+by+foliage.png" }, { "title": "Image group 02", "createVideo": false, "objectUrl_0": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/thing-0001.jpg", "objectUrl_1": "<https://encord-integration.s3.eu-west-2.amazonaws.com/images/thing-0002.jpg", "objectUrl_2": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/thing-0003.jpg" } ], "skip_duplicate_urls": true } ``` </CodeGroup> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Image sequences"> #### [Image sequences](#image-sequences) For detailed information about the JSON file format used for import go [here](/platform-documentation/Curate/add-files/index-register-cloud-data#json-format). * Image sequences are collections of images that are processed as one annotation task and represented as a video. * Images within image sequences may be altered as images of varying sizes and resolutions are made to match that of the first image in the sequence. * Creating Image sequences from cloud storage requires 'write' permissions, as new files have to be created in order to be read as a video. * Each object in the `image_groups` array with the `createVideo` flag set to `true` represents a single image sequence. * If `skip_duplicate_urls` is set to `true`, all URLs exactly matching existing image sequences in the dataset are skipped. <Tip> The only difference between adding image groups and image sequences using a JSON file is that image sequences require the `createVideo` flag to be set to `true`. Both use the key `image_groups`.</Tip> <Note>The position of each image within the sequence needs to be specified in the key (`objectUrl_{position_number}`).</Note> <Info>Encord supports up to 32,767 entries (21:50 minutes) for a single image sequence. We recommend up to 10,000 to 15,000 entries for a single image sequence for best performance. If you need a longer sequence, we recommend using video instead of an image sequence.</Info> **Template:** Provides the proper JSON format to import image groups into Encord. **Examples:** * **Data:** Imports the images groups only. <CodeGroup> ```json Template theme={"dark"} { "image_groups": [ { "title": "<title 1>", "createVideo": true, "objectUrl_0": "<object url>" }, { "title": "<title 2>", "createVideo": true, "objectUrl_0": "<object url>", "objectUrl_1": "<object url>", "objectUrl_2": "<object url>" } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "image_groups": [ { "title": "Image sequence 001", "createVideo": true, "objectUrl_0": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/01.jpg", "objectUrl_1": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/02.jpg", "objectUrl_2": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/DALL%C2%B7E+2022-09-08+18.53.25+-+firefighter+extinguishing+flames+around+computer+in+software+office+overgrown+by+foliage.png" }, { "title": "Image sequence 002", "createVideo": true, "objectUrl_0": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/thing-01.jpg", "objectUrl_1": "<https://encord-integration.s3.eu-west-2.amazonaws.com/images/thing-02.jpg", "objectUrl_2": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/thing-03.jpg" } ], "skip_duplicate_urls": true } ``` </CodeGroup> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="DICOM"> #### [DICOM](#dicom) For detailed information about the JSON file format used for import go [here](/platform-documentation/Curate/add-files/index-register-cloud-data#json-format). * Each `dicom_series` element can contain one or more DICOM series. * Each series requires a title and at least one object URL, as shown in the example below. * If `skip_duplicate_urls` is set to `true`, all object URLs exactly matching existing DICOM files in the dataset will be skipped. <Note>Custom metadata is distinct from patient metadata, which is included in the `.dcm` file and does not have to be specific during the upload to Encord. </Note> The following is an example JSON for uploading three DICOM series belonging to a study. Each title and object URL correspond to individual DICOM series. * The first series contains only a single object URL, as it is composed of a single file. * The second series contains 3 object URLs, as it is composed of three separate files. * The third series contains 2 object URLs, as it is composed of two separate files. <Info> For each DICOM upload, an additional `DicomSeries` file is created. This file represents the series file-set. Only `DicomSeries` are displayed in the Encord application. </Info> ```json Template theme={"dark"} { "dicom_series": [ { "title": "Series-1", "objectUrl_0": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/study1-series1-file.dcm" }, { "title": "Series-2", "objectUrl_0": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/study1-series2-file1.dcm", "objectUrl_1": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/study1-series2-file2.dcm", "objectUrl_2": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/study1-series2-file3.dcm", }, { "title": "Series-3", "objectUrl_0": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/study1-series3-file1.dcm", "objectUrl_1": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/study1-series3-file2.dcm", } ], "skip_duplicate_urls": true } ``` </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="NIfTI"> #### [NIfTI](#nifti) The following is an example JSON file for uploading two NIfTI files to Encord. ```json Template theme={"dark"} { "nifti": [ { "title": "<file-1>", "objectUrl": "https://my-bucket/.../nifti-file1.nii" }, { "title": "<file-2>", "objectUrl": "https://my-bucket/.../nifti-file2.nii.gz" } ], "skip_duplicate_urls": true } ``` </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title=" Multiple file types "> You can upload multiple file types using a single JSON file. The example below shows 1 image, 2 videos, 2 image sequences, and 1 image group. ```json Multiple file types theme={"dark"} { "images": [ { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/Image1.png" } ], "videos": [ { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/videos/Cooking.mp4" }, { "objectUrl": "https://encord-integration.s3.eu-west-2.amazonaws.com/videos/Oranges.mp4" } ], "image_groups": [ { "title": "apple-samsung-light", "createVideo": true, "objectUrl_0": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/1+(32).jpg", "objectUrl_1": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/1+(33).jpg", "objectUrl_2": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/1+(34).jpg", "objectUrl_3": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/1+(35).jpg" }, { "title": "apple-samsung-dark", "createVideo": true, "objectUrl_0": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/2+(32).jpg", "objectUrl_1": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/2+(33).jpg", "objectUrl_2": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/2+(34).jpg", "objectUrl_3": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/2+(35).jpg" } ], "image_groups": [ { "title": "apple-ios-light", "createVideo": false, "objectUrl_0": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/3+(32).jpg", "objectUrl_1": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/3+(33).jpg" } ], "skip_duplicate_urls": true } ``` </Accordion> </AccordionGroup> </Accordion> <Accordion title="JSON for GCP"> #### GCP JSON <AccordionGroup> <Accordion title="Videos"> #### [Videos](#videos) **Video Metadata** When the `videoMetadata` flag is present in the JSON file, we directly use the supplied metadata without performing any additional validation, and do not store the file on our servers. <Warning>To guarantee accurate labels, it is crucial that the `videoMetadata` provided is accurate.</Warning> <CodeGroup> ```json Template theme={"dark"} { "videos": [ { "objectUrl": "cloud-path-to-your-video-1" }, { "objectUrl": "cloud-path-to-your-video-2", "videoMetadata": { "fps": frames-per-second, "duration": duration-in-seconds, "width": frame-width, "height": frame-height, "file_size": file-size-in-bytes, "mime_type": "MIME-file-type-extension" } } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "videos": [ { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/eLeopardJog.mp4" }, { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/_video.mp4" }, { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/bluemlisalphutte_flyover.mp4" }, { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/boat_lake_normalized.mp4" }, { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/boat_lake.MP4" }, { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/boat_lake.MP4" }, { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/snow_sled.MOV" }, { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/cyclists.MP4" } ], "skip_duplicate_urls": true } ``` ```json Video Metadata theme={"dark"} { "videos": [ { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/es/video_file.mp4", "videoMetadata": { "fps": 23.98, "duration": 29.09, "width": 1280, "height": 720, "file_size": 5468354, "mime_type": "video/mp4" } } ], "skip_duplicate_urls": true } ``` </CodeGroup> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Audio files"> #### [Audio Files](#audio-files) The following is an example JSON file for uploading two audio files to Encord. <Tip> Waveform generation for long audio files can cause lag. To avoid this, generate the waveform offline and upload it with the audio file. For more information go [here](/platform-documentation/Curate/index-audio#long-audio-files). </Tip> * Example 1 imports audio files with an Encord title. * Example 2 imports one audio file with the `audiometadata` flag. When the `audiometadata` flag is present in the JSON file, we directly use the supplied metadata without performing any additional validation, and do not store the file on our servers. To guarantee accurate labels, it is crucial that the metadata you provide is accurate. <CodeGroup> ```json Template theme={"dark"} { "audio": [ { "objectUrl": "<object url_1>" }, { "objectUrl": "<object url_2>", "title": "my-custom-audio-file-title.mp3" } ], "skip_duplicate_urls": true } ``` ```json Audio Metadata theme={"dark"} { "audio": [ { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/images/audio_file.mp3", "audioMetadata": { "duration": 23.98, "file_size": 2900000, "mime_type": "audio/mp3", "sample_rate": 44100, "bit_depth": 24, "codec": "mp3", "num_channels": 2 } } ], "skip_duplicate_urls": true } ``` </CodeGroup> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="PDFs"> #### [PDFs](#pdfs) The following are example JSON files for uploading PDFs to Encord. <CodeGroup> ```json Template theme={"dark"} { "pdfs": [ { "objectUrl": "<object url_1>" }, { "objectUrl": "<object url_2>", "title": "my-document-02.pdf" }, { "objectUrl": "<object url_3>", "title": "my-document-03.pdf", "pdfMetadata": { "fileSize": 300, "numPages": 5 } } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "pdfs": [ { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/my-document-01.pdf", }, { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/my-document-02.pdf", "title": "my-document-02.pdf" }, { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/my-document-03.pdf", "title": "my-document-03.pdf", "pdfMetadata": { "fileSize": 300, "numPages": 5 } } ], "skip_duplicate_urls": true } ``` </CodeGroup> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Text Files"> #### [Text Files](#text-files) The following are example JSON files for uploading text files to Encord. <CodeGroup> ```json Template theme={"dark"} { "text": [ { "objectUrl": "<object url_1>" }, { "objectUrl": "<object url_2>", "title": "my-file.html" }, { "objectUrl": "<object url_3>", "title": "my-plain-text-file.txt", "textMetadata": { "fileSize": 200, "mime_type": "text/plain" } }, { "objectUrl": "<object url_4>", "title": "my-csv-file.csv", "textMetadata": { "fileSize": 250, "mime_type": "text/csv" } }, { "objectUrl": "<object url_5>", "title": "my-xml-file.xml", "textMetadata": { "fileSize": 300, "mime_type": "application/xml" } }, { "objectUrl": "<object url_6>", "title": "my-html-file.html", "textMetadata": { "fileSize": 400, "mime_type": "text/html" } } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "text": [ { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/my-file-01.html" }, { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/my-plain-text-file-01.txt", "title": "my-plain-text-file-01.txt" }, { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/my-plain-text-file-02.txt", "title": "my-plain-text-file-02.txt", "textMetadata": { "fileSize": 200, "mime_type": "text/plain" } }, { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/my-csv-file-01.csv", "title": "my-csv-file-01.csv", "textMetadata": { "fileSize": 250, "mime_type": "text/csv" } }, { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/my-xml-file-01.xml", "title": "my-xml-file-01.xml", "textMetadata": { "fileSize": 300, "mime_type": "application/xml" } }, { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/my-html-file-01.html", "title": "my-html-file-01.html", "textMetadata": { "fileSize": 400, "mime_type": "text/html" } } ], "skip_duplicate_urls": true } ``` </CodeGroup> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Single images"> #### [Single Images](#single-images) For detailed information about the JSON file format used for import go [here](/platform-documentation/Curate/add-files/index-register-cloud-data#json-format). The JSON structure for single images parallels that of videos. **Template:** Provides the proper JSON format to import images into Encord. **Examples:** * **Data** Imports the images only. * **Image Metadata:** Imports images with image metadata. This improves the import speed for your images. <CodeGroup> ```json Template theme={"dark"} { "images": [ { "objectUrl": "file/path/to/images/file-name-01.file-extension" }, { "objectUrl": "file/path/to/images/file-name-02.file-extension" }, { "objectUrl": "file/path/to/images/file-name-03.file-extension", "title": "image-title.file-extension" } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "images": [ { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/es/0001.jpg" }, { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/images/0002.jpg" }, { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/images/0003.jpg" }, { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/images/DALL%C2%B7E+2022-09-08+18.53.25+-+firefighter+extinguishing+flames+around+computer+in+software+office+overgrown+by+foliage.png" }, { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/images/DALL%C2%B7E+2022-12-08+22.16.52+-+steampunk+combustion+engine+that+is+fueled+by+data+and+produces+computer+vision+algorithms.png" }, { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/images/large_images/pexels-ivo-rainha-00057.png" } ], "skip_duplicate_urls": true } ``` ```json Image Metadata theme={"dark"} { "images": [ { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/images/my_image.jpg", "imageMetadata": { "mimeType": "image/jpg", "fileSize": 124, "width": 640, "height": 480 } } ], "skip_duplicate_urls": true } ``` </CodeGroup> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Image groups"> #### [Image groups](#image-groups) For detailed information about the JSON file format used for import go [here](/platform-documentation/Curate/add-files/index-register-cloud-data#json-format). * Image groups are collections of images that are processed as one annotation task. * Images within image groups remain unaltered, meaning that images of different sizes and resolutions can form an image group without the loss of data. * Image groups do NOT require 'write' permissions to your cloud storage. * If `skip_duplicate_urls` is set to `true`, all URLs exactly matching existing image groups in the dataset are skipped. <Note>The position of each image within the sequence needs to be specified in the key (`objectUrl_{position_number}`).</Note> **Template:** Provides the proper JSON format to import image groups into Encord. **Examples:** * **Data:** Imports the image groups only. <CodeGroup> ```json Template theme={"dark"} { "image_groups": [ { "title": "<title 1>", "createVideo": false, "objectUrl_0": "file/path/to/images/file-name-01.file-extension", "objectUrl_1": "file/path/to/images/file-name-02.file-extension", "objectUrl_2": "file/path/to/images/file-name-03.file-extension", }, { "title": "<title 2>", "createVideo": false, "objectUrl_0": "file/path/to/images/file-name-01.file-extension", "objectUrl_1": "file/path/to/images/file-name-02.file-extension", "objectUrl_2": "file/path/to/images/file-name-03.file-extension" } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "image_groups": [ { "title": "Image group 01", "createVideo": false, "objectUrl_0": "https://storage.cloud.google.com/encord-image-bucket/images/0001.jpg", "objectUrl_1": "https://storage.cloud.google.com/encord-image-bucket/images/0002.jpg", "objectUrl_2": "https://storage.cloud.google.com/encord-image-bucket/images/DALL%C2%B7E+2022-09-08+18.53.25+-+firefighter+extinguishing+flames+around+computer+in+software+office+overgrown+by+foliage.png" }, { "title": "Image group 02", "createVideo": false, "objectUrl_0": "https://storage.cloud.google.com/encord-image-bucket/images/thing-0001.jpg", "objectUrl_1": "<https://storage.cloud.google.com/encord-image-bucket/images/thing-0002.jpg", "objectUrl_2": "https://storage.cloud.google.com/encord-image-bucket/images/thing-0003.jpg" } ], "skip_duplicate_urls": true } ``` </CodeGroup> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Image sequences"> #### [Image sequences](#image-sequences) For detailed information about the JSON file format used for import go [here](/platform-documentation/Curate/add-files/index-register-cloud-data#json-format). * Image sequences are collections of images that are processed as one annotation task and represented as a video. * Images within image sequences may be altered as images of varying sizes and resolutions are made to match that of the first image in the sequence. * Creating Image sequences from cloud storage requires 'write' permissions, as new files have to be created in order to be read as a video. * Each object in the `image_groups` array with the `createVideo` flag set to `true` represents a single image sequence. * If `skip_duplicate_urls` is set to `true`, all URLs exactly matching existing image sequences in the dataset are skipped. <Tip> The only difference between adding image groups and image sequences using a JSON file is that image sequences require the `createVideo` flag to be set to `true`. Both use the key `image_groups`.</Tip> <Note>The position of each image within the sequence needs to be specified in the key (`objectUrl_{position_number}`).</Note> <Info>Encord supports up to 32,767 entries (21:50 minutes) for a single image sequence. We recommend up to 10,000 to 15,000 entries for a single image sequence for best performance. If you need a longer sequence, we recommend using video instead of an image sequence.</Info> **Template:** Provides the proper JSON format to import image groups into Encord. **Examples:** * **Data:** Imports the images groups only. <CodeGroup> ```json Template theme={"dark"} { "image_groups": [ { "title": "<title 1>", "createVideo": true, "objectUrl_0": "<object url>" }, { "title": "<title 2>", "createVideo": true, "objectUrl_0": "<object url>", "objectUrl_1": "<object url>", "objectUrl_2": "<object url>" } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "image_groups": [ { "title": "Image sequence 001", "createVideo": true, "objectUrl_0": "https://storage.cloud.google.com/encord-image-bucket/images/01.jpg", "objectUrl_1": "https://storage.cloud.google.com/encord-image-bucket/images/02.jpg", "objectUrl_2": "https://storage.cloud.google.com/encord-image-bucket/images/DALL%C2%B7E+2022-09-08+18.53.25+-+firefighter+extinguishing+flames+around+computer+in+software+office+overgrown+by+foliage.png" }, { "title": "Image sequence 002", "createVideo": true, "objectUrl_0": "https://storage.cloud.google.com/encord-image-bucket/images/thing-01.jpg", "objectUrl_1": "<https://storage.cloud.google.com/encord-image-bucket/images/thing-02.jpg", "objectUrl_2": "https://storage.cloud.google.com/encord-image-bucket/images/thing-03.jpg" } ], "skip_duplicate_urls": true } ``` </CodeGroup> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="DICOM"> #### [DICOM](#dicom) For detailed information about the JSON file format used for import go [here](/platform-documentation/Curate/add-files/index-register-cloud-data#json-format). * Each `dicom_series` element can contain one or more DICOM series. * Each series requires a title and at least one object URL, as shown in the example below. * If `skip_duplicate_urls` is set to `true`, all object URLs exactly matching existing DICOM files in the dataset will be skipped. <Note>Custom metadata is distinct from patient metadata, which is included in the `.dcm` file and does not have to be specific during the upload to Encord. </Note> The following is an example JSON for uploading three DICOM series belonging to a study. Each title and object URL correspond to individual DICOM series. * The first series contains only a single object URL, as it is composed of a single file. * The second series contains 3 object URLs, as it is composed of three separate files. * The third series contains 2 object URLs, as it is composed of two separate files. <Info> For each DICOM upload, an additional `DicomSeries` file is created. This file represents the series file-set. Only `DicomSeries` are displayed in the Encord application. </Info> ```json JSON for DICOM theme={"dark"} { "dicom_series": [ { "title": "Series-1", "objectUrl_0": "https://storage.cloud.google.com/encord-image-bucket/images/study1-series1-file.dcm" }, { "title": "Series-2", "objectUrl_0": "https://storage.cloud.google.com/encord-image-bucket/images/study1-series2-file1.dcm", "objectUrl_1": "https://storage.cloud.google.com/encord-image-bucket/images/study1-series2-file2.dcm", "objectUrl_2": "https://storage.cloud.google.com/encord-image-bucket/images/study1-series2-file3.dcm", }, { "title": "Series-3", "objectUrl_0": "https://storage.cloud.google.com/encord-image-bucket/images/study1-series3-file1.dcm", "objectUrl_1": "https://storage.cloud.google.com/encord-image-bucket/images/study1-series3-file2.dcm", } ], "skip_duplicate_urls": true } ``` </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="NIfTI"> #### [NIfTI](#nifti) The following is an example JSON file for uploading two NIfTI files to Encord. ```json theme={"dark"} { "nifti": [ { "title": "<file-1>", "objectUrl": "https://my-bucket/.../nifti-file1.nii" }, { "title": "<file-2>", "objectUrl": "https://my-bucket/.../nifti-file2.nii.gz" } ], "skip_duplicate_urls": true } ``` </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title=" Multiple file types "> You can upload multiple file types using a single JSON file. The example below shows 1 image, 2 videos, 2 image sequences, and 1 image group. ```json Multiple file types theme={"dark"} { "images": [ { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/images/Image1.png" } ], "videos": [ { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/videos/Cooking.mp4" }, { "objectUrl": "https://storage.cloud.google.com/encord-image-bucket/videos/Oranges.mp4" } ], "image_groups": [ { "title": "apple-samsung-light", "createVideo": true, "objectUrl_0": "https://storage.cloud.google.com/encord-image-bucket/images/1+(32).jpg", "objectUrl_1": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/1+(33).jpg", "objectUrl_2": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/1+(34).jpg", "objectUrl_3": "https://encord-integration.s3.eu-west-2.amazonaws.com/images/1+(35).jpg" }, { "title": "apple-samsung-dark", "createVideo": true, "objectUrl_0": "https://storage.cloud.google.com/encord-image-bucket/images/2+(32).jpg", "objectUrl_1": "https://storage.cloud.google.com/encord-image-bucket/images/2+(33).jpg", "objectUrl_2": "https://storage.cloud.google.com/encord-image-bucket/images/2+(34).jpg", "objectUrl_3": "https://storage.cloud.google.com/encord-image-bucket/images/2+(35).jpg" } ], "image_groups": [ { "title": "apple-ios-light", "createVideo": false, "objectUrl_0": "https://storage.cloud.google.com/encord-image-bucket/images/3+(32).jpg", "objectUrl_1": "https://storage.cloud.google.com/encord-image-bucket/images/3+(33).jpg" } ], "skip_duplicate_urls": true } ``` </Accordion> </AccordionGroup> </Accordion> <Accordion title="JSON for Azure"> #### Azure JSON <AccordionGroup> <Accordion title="Videos"> #### [Videos](#videos) **Video Metadata** When the `videoMetadata` flag is present in the JSON file, we directly use the supplied metadata without performing any additional validation, and do not store the file on our servers. <Warning>To guarantee accurate labels, it is crucial that the `videoMetadata` provided is accurate.</Warning> <CodeGroup> ```json Template theme={"dark"} { "videos": [ { "objectUrl": "cloud-path-to-your-video-1" }, { "objectUrl": "cloud-path-to-your-video-2", "videoMetadata": { "fps": frames-per-second, "duration": duration-in-seconds, "width": frame-width, "height": frame-height, "file_size": file-size-in-bytes, "mime_type": "MIME-file-type-extension" } } { "objectUrl": "cloud-path-to-your-video-3", "title": "title-for-your-video-3" } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "videos": [ { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/BerghouseLeopardJog.mp4" }, { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/blue_bus_video.mp4" }, { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/bluemlisalphutte_flyover.mp4" }, { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/boat_lake_normalized.mp4" }, { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/boat_lake.MP4" }, { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/boat_lake.MP4" }, { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/snow_sled.MOV" }, { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/cyclists.MP4" } ], "skip_duplicate_urls": true } ``` ```json Video Metadata theme={"dark"} { "videos": [ { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/video_file.mp4", "videoMetadata": { "fps": 23.98, "duration": 29.09, "width": 1280, "height": 720, "file_size": 5468354, "mime_type": "video/mp4" } } ], "skip_duplicate_urls": true } ``` </CodeGroup> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Audio files"> #### [Audio Files](#audio-files) The following is an example JSON file for uploading two audio files to Encord. <Tip> Waveform generation for long audio files can cause lag. To avoid this, generate the waveform offline and upload it with the audio file. For more information go [here](/platform-documentation/Curate/index-audio#long-audio-files). </Tip> * **Template:** Imports audio files with an Encord title. * **Audio Metadata:** Imports one audio file with the `audiometadata` flag. When the `audiometadata` flag is present in the JSON file, we directly use the supplied metadata without performing any additional validation, and do not store the file on our servers. To guarantee accurate labels, it is crucial that the metadata you provide is accurate. <CodeGroup> ```json Template theme={"dark"} { "audio": [ { "objectUrl": "<object url_1>" }, { "objectUrl": "<object url_2>", "title": "my-custom-audio-file-title.mp3" } ], "skip_duplicate_urls": true } ``` ```json Audio Metadata theme={"dark"} { "audio": [ { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/audio_file.mp3", "audioMetadata": { "duration": 23.98, "file_size": 2900000, "mime_type": "audio/mp3", "sample_rate": 44100, "bit_depth": 24, "codec": "mp3", "num_channels": 2 } } ], "skip_duplicate_urls": true } ``` </CodeGroup> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="PDFs"> #### [PDFs](#pdfs) The following are example JSON files for uploading PDFs to Encord. <CodeGroup> ```json Template theme={"dark"} { "pdfs": [ { "objectUrl": "<object url_1>" }, { "objectUrl": "<object url_2>", "title": "my-document-02.pdf" }, { "objectUrl": "<object url_3>", "title": "my-document-03.pdf", "pdfMetadata": { "fileSize": 300, "numPages": 5 } } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "pdfs": [ { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/my-document-01.pdf", }, { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/my-document-02.pdf", "title": "my-document-02.pdf" }, { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/my-document-03.pdf", "title": "my-document-03.pdf", "pdfMetadata": { "fileSize": 300, "numPages": 5 } } ], "skip_duplicate_urls": true } ``` </CodeGroup> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Text Files"> #### [Text Files](#text-files) The following are example JSON files for uploading text files to Encord. <CodeGroup> ```json Template theme={"dark"} { "text": [ { "objectUrl": "<object url_1>" }, { "objectUrl": "<object url_2>", "title": "my-file.html" }, { "objectUrl": "<object url_3>", "title": "my-plain-text-file.txt", "textMetadata": { "fileSize": 200, "mime_type": "text/plain" } }, { "objectUrl": "<object url_4>", "title": "my-csv-file.csv", "textMetadata": { "fileSize": 250, "mime_type": "text/csv" } }, { "objectUrl": "<object url_5>", "title": "my-xml-file.xml", "textMetadata": { "fileSize": 300, "mime_type": "application/xml" } }, { "objectUrl": "<object url_6>", "title": "my-html-file.html", "textMetadata": { "fileSize": 400, "mime_type": "text/html" } } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "text": [ { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/my-file-01.html" }, { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/my-plain-text-file-01.txt", "title": "my-plain-text-file-01.txt" }, { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/my-plain-text-file-02.txt", "title": "my-plain-text-file-02.txt", "textMetadata": { "fileSize": 200, "mime_type": "text/plain" } }, { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/my-csv-file-01.csv", "title": "my-csv-file-01.csv", "textMetadata": { "fileSize": 250, "mime_type": "text/csv" } }, { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/my-xml-file-01.xml", "title": "my-xml-file-01.xml", "textMetadata": { "fileSize": 300, "mime_type": "application/xml" } }, { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/my-html-file-01.html", "title": "my-html-file-01.html", "textMetadata": { "fileSize": 400, "mime_type": "text/html" } } ], "skip_duplicate_urls": true } ``` </CodeGroup> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Single images"> #### [Single Images](#single-images) For detailed information about the JSON file format used for import go [here](/platform-documentation/Curate/add-files/index-register-cloud-data#json-format). The JSON structure for single images parallels that of videos. **Template:** Provides the proper JSON format to import images into Encord. **Examples:** * **Data** Imports the images only. * **Image Metadata:** Imports images with image metadata. This improves the import speed for your images. <CodeGroup> ```json Template theme={"dark"} { "images": [ { "objectUrl": "file/path/to/images/file-name-01.file-extension" }, { "objectUrl": "file/path/to/images/file-name-02.file-extension" }, { "objectUrl": "file/path/to/images/file-name-03.file-extension", "title": "image-title.file-extension" } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "images": [ { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/0001.jpg" }, { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/0002.jpg" }, { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/0003.jpg" }, { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/DALL%C2%B7E+2022-09-08+18.53.25+-+firefighter+extinguishing+flames+around+computer+in+software+office+overgrown+by+foliage.png" }, { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/DALL%C2%B7E+2022-12-08+22.16.52+-+steampunk+combustion+engine+that+is+fueled+by+data+and+produces+computer+vision+algorithms.png" }, { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/large_images/pexels-ivo-rainha-00057.png" } ], "skip_duplicate_urls": true } ``` ```json Image Metadata theme={"dark"} { "images": [ { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/my_image.jpg", "imageMetadata": { "mimeType": "image/jpg", "fileSize": 124, "width": 640, "height": 480 } } ], "skip_duplicate_urls": true } ``` </CodeGroup> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Image groups"> #### [Image groups](#image-groups) For detailed information about the JSON file format used for import go [here](/platform-documentation/Curate/add-files/index-register-cloud-data#json-format). * Image groups are collections of images that are processed as one annotation task. * Images within image groups remain unaltered, meaning that images of different sizes and resolutions can form an image group without the loss of data. * Image groups do NOT require 'write' permissions to your cloud storage. * If `skip_duplicate_urls` is set to `true`, all URLs exactly matching existing image groups in the dataset are skipped. <Note>The position of each image within the sequence needs to be specified in the key (`objectUrl_{position_number}`).</Note> **Template:** Provides the proper JSON format to import image groups into Encord. **Examples:** * **Data:** Imports the image groups only. <CodeGroup> ```json Template theme={"dark"} { "image_groups": [ { "title": "<title 1>", "createVideo": false, "objectUrl_0": "file/path/to/images/file-name-01.file-extension", "objectUrl_1": "file/path/to/images/file-name-02.file-extension", "objectUrl_2": "file/path/to/images/file-name-03.file-extension" }, { "title": "<title 2>", "createVideo": false, "objectUrl_0": "file/path/to/images/file-name-01.file-extension", "objectUrl_1": "file/path/to/images/file-name-02.file-extension", "objectUrl_2": "file/path/to/images/file-name-03.file-extension" } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "image_groups": [ { "title": "Image group 01", "createVideo": false, "objectUrl_0": "https://myaccount.blob.core.windows.net/encordcontainer/0001.jpg", "objectUrl_1": "https://myaccount.blob.core.windows.net/encordcontainer/0002.jpg", "objectUrl_2": "https://myaccount.blob.core.windows.net/encordcontainer/DALL%C2%B7E+2022-09-08+18.53.25+-+firefighter+extinguishing+flames+around+computer+in+software+office+overgrown+by+foliage.png" }, { "title": "Image group 02", "createVideo": false, "objectUrl_0": "https://myaccount.blob.core.windows.net/encordcontainer/images/thing-0001.jpg", "objectUrl_1": "https://myaccount.blob.core.windows.net/encordcontainer/images/thing-0002.jpg", "objectUrl_2": "https://myaccount.blob.core.windows.net/encordcontainer/thing-0003.jpg" } ], "skip_duplicate_urls": true } ``` </CodeGroup> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Image sequences"> #### [Image sequences](#image-sequences) For detailed information about the JSON file format used for import go [here](/platform-documentation/Curate/add-files/index-register-cloud-data#json-format). * Image sequences are collections of images that are processed as one annotation task and represented as a video. * Images within image sequences may be altered as images of varying sizes and resolutions are made to match that of the first image in the sequence. * Creating Image sequences from cloud storage requires 'write' permissions, as new files have to be created in order to be read as a video. * Each object in the `image_groups` array with the `createVideo` flag set to `true` represents a single image sequence. * If `skip_duplicate_urls` is set to `true`, all URLs exactly matching existing image sequences in the dataset are skipped. <Tip> The only difference between adding image groups and image sequences using a JSON file is that image sequences require the `createVideo` flag to be set to `true`. Both use the key `image_groups`.</Tip> <Note>The position of each image within the sequence needs to be specified in the key (`objectUrl_{position_number}`).</Note> <Info>Encord supports up to 32,767 entries (21:50 minutes) for a single image sequence. We recommend up to 10,000 to 15,000 entries for a single image sequence for best performance. If you need a longer sequence, we recommend using video instead of an image sequence.</Info> **Template:** Provides the proper JSON format to import image groups into Encord. **Examples:** * **Data:** Imports the images groups only. <CodeGroup> ```json Template theme={"dark"} { "image_groups": [ { "title": "<title 1>", "createVideo": true, "objectUrl_0": "<object url>" }, { "title": "<title 2>", "createVideo": true, "objectUrl_0": "<object url>", "objectUrl_1": "<object url>", "objectUrl_2": "<object url>" } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "image_groups": [ { "title": "Image sequence 001", "createVideo": true, "objectUrl_0": "https://myaccount.blob.core.windows.net/encordcontainer/01.jpg", "objectUrl_1": "https://myaccount.blob.core.windows.net/encordcontainer/02.jpg", "objectUrl_2": "https://myaccount.blob.core.windows.net/encordcontainer/DALL%C2%B7E+2022-09-08+18.53.25+-+firefighter+extinguishing+flames+around+computer+in+software+office+overgrown+by+foliage.png" }, { "title": "Image sequence 002", "createVideo": true, "objectUrl_0": "https://myaccount.blob.core.windows.net/encordcontainer/thing-01.jpg", "objectUrl_1": "<https://myaccount.blob.core.windows.net/encordcontainer/thing-02.jpg", "objectUrl_2": "https://myaccount.blob.core.windows.net/encordcontainer/thing-03.jpg" } ], "skip_duplicate_urls": true } ``` </CodeGroup> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="DICOM"> #### [DICOM](#dicom) For detailed information about the JSON file format used for import go [here](/platform-documentation/Curate/add-files/index-register-cloud-data#json-format). * Each `dicom_series` element can contain one or more DICOM series. * Each series requires a title and at least one object URL, as shown in the example below. * If `skip_duplicate_urls` is set to `true`, all object URLs exactly matching existing DICOM files in the dataset will be skipped. <Note>Custom metadata is distinct from patient metadata, which is included in the `.dcm` file and does not have to be specific during the upload to Encord. </Note> The following is an example JSON for uploading three DICOM series belonging to a study. Each title and object URL correspond to individual DICOM series. * The first series contains only a single object URL, as it is composed of a single file. * The second series contains 3 object URLs, as it is composed of three separate files. * The third series contains 2 object URLs, as it is composed of two separate files. <Info> For each DICOM upload, an additional `DicomSeries` file is created. This file represents the series file-set. Only `DicomSeries` are displayed in the Encord application. </Info> ```json Template theme={"dark"} { "dicom_series": [ { "title": "Series-1", "objectUrl_0": "https://myaccount.blob.core.windows.net/encordcontainer/study1-series1-file.dcm" }, { "title": "Series-2", "objectUrl_0": "https://myaccount.blob.core.windows.net/encordcontainer/study1-series2-file1.dcm", "objectUrl_1": "https://myaccount.blob.core.windows.net/encordcontainer/study1-series2-file2.dcm", "objectUrl_2": "https://myaccount.blob.core.windows.net/encordcontainer/study1-series2-file3.dcm", }, { "title": "Series-3", "objectUrl_0": "https://myaccount.blob.core.windows.net/encordcontainer/study1-series3-file1.dcm", "objectUrl_1": "https://myaccount.blob.core.windows.net/encordcontainer/study1-series3-file2.dcm", } ], "skip_duplicate_urls": true } ``` </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="NIfTI"> #### [NIfTI](#nifti) The following is an example JSON file for uploading two NIfTI files to Encord. ```json Template theme={"dark"} { "nifti": [ { "title": "<file-1>", "objectUrl": "https://my-bucket/.../nifti-file1.nii" }, { "title": "<file-2>", "objectUrl": "https://my-bucket/.../nifti-file2.nii.gz" } ], "skip_duplicate_urls": true } ``` </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title=" Multiple file types "> You can upload multiple file types using a single JSON file. The example below shows 1 image, 2 videos, 2 image sequences, and 1 image group. ```json theme={"dark"} { "images": [ { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/Image1.png" } ], "videos": [ { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/Cooking.mp4" }, { "objectUrl": "https://myaccount.blob.core.windows.net/encordcontainer/Oranges.mp4" } ], "image_groups": [ { "title": "apple-samsung-light", "createVideo": true, "objectUrl_0": "https://myaccount.blob.core.windows.net/encordcontainer/1-Samsung-S4-Light+Environment/1+(32).jpg", "objectUrl_1": "https://myaccount.blob.core.windows.net/encordcontainer/1-Samsung-S4-Light+Environment/1+(33).jpg", "objectUrl_2": "https://myaccount.blob.core.windows.net/encordcontainer/1-Samsung-S4-Light+Environment/1+(34).jpg", "objectUrl_3": "https://myaccount.blob.core.windows.net/encordcontainer/1-Samsung-S4-Light+Environment/1+(35).jpg" }, { "title": "apple-samsung-dark", "createVideo": true, "objectUrl_0": "https://myaccount.blob.core.windows.net/encordcontainer/2-samsung-S4-Dark+Environment/2+(32).jpg", "objectUrl_1": "https://myaccount.blob.core.windows.net/encordcontainer/2-samsung-S4-Dark+Environment/2+(33).jpg", "objectUrl_2": "https://myaccount.blob.core.windows.net/encordcontainer/2-samsung-S4-Dark+Environment/2+(34).jpg", "objectUrl_3": "https://myaccount.blob.core.windows.net/encordcontainer/2-samsung-S4-Dark+Environment/2+(35).jpg" } ], "image_groups": [ { "title": "apple-ios-light", "createVideo": false, "objectUrl_0": "https://myaccount.blob.core.windows.net/encordcontainer/3-IOS-4-Light+Environment/3+(32).jpg", "objectUrl_1": "https://myaccount.blob.core.windows.net/encordcontainer/3-IOS-4-Light+Environment/3+(33).jpg" } ], "skip_duplicate_urls": true } ``` </Accordion> </AccordionGroup> </Accordion> <Accordion title="JSON for OTC"> #### OTC JSON <AccordionGroup> <Accordion title="Videos"> #### [Videos](#videos) **Video Metadata** When the `videoMetadata` flag is present in the JSON file, we directly use the supplied metadata without performing any additional validation, and do not store the file on our servers. <Warning>To guarantee accurate labels, it is crucial that the `videoMetadata` provided is accurate.</Warning> <CodeGroup> ```json Template theme={"dark"} { "videos": [ { "objectUrl": "cloud-path-to-your-video-1" }, { "objectUrl": "cloud-path-to-your-video-2", "videoMetadata": { "fps": frames-per-second, "duration": duration-in-seconds, "width": frame-width, "height": frame-height, "file_size": file-size-in-bytes, "mime_type": "MIME-file-type-extension" } } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "videos": [ { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/og.mp4" }, { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/videos/blue_bus_video.mp4" }, { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/videos/bluemlisalphutte_flyover.mp4" }, { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/videos/boat_lake_normalized.mp4" }, { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/videos/boat_lake.MP4" }, { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/videos/boat_lake.MP4" }, { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/videos/snow_sled.MOV" }, { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/videos/cyclists.MP4" } ], "skip_duplicate_urls": true } ``` ```json Video Metadata theme={"dark"} { "videos": [ { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/videos/video_file.mp4", "videoMetadata": { "fps": 23.98, "duration": 29.09, "width": 1280, "height": 720, "file_size": 5468354, "mime_type": "video/mp4" } } ], "skip_duplicate_urls": true } ``` </CodeGroup> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Audio files"> #### [Audio Files](#audio-files) The following is an example JSON file for uploading two audio files to Encord. <Tip> Waveform generation for long audio files can cause lag. To avoid this, generate the waveform offline and upload it with the audio file. For more information go [here](/platform-documentation/Curate/index-audio#long-audio-files). </Tip> * **Template:** Imports audio files with an Encord title. * **Audio Metadata:** Imports one audio file with the `audiometadata` flag. When the `audiometadata` flag is present in the JSON file, we directly use the supplied metadata without performing any additional validation, and do not store the file on our servers. To guarantee accurate labels, it is crucial that the metadata you provide is accurate. <CodeGroup> ```json Template theme={"dark"} { "audio": [ { "objectUrl": "<object url_1>" }, { "objectUrl": "<object url_2>", "title": "my-custom-audio-file-title.mp3" } ], "skip_duplicate_urls": true } ``` ```json Audio Metadata theme={"dark"} { "audio": [ { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/audio/audio_file.mp3", "audioMetadata": { "duration": 23.98, "file_size": 2900000, "mime_type": "audio/mp3", "sample_rate": 44100, "bit_depth": 24, "codec": "mp3", "num_channels": 2 } } ], "skip_duplicate_urls": true } ``` </CodeGroup> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="PDFs"> #### [PDFs](#pdfs) The following are example JSON files for uploading PDFs to Encord. <CodeGroup> ```json Template theme={"dark"} { "pdfs": [ { "objectUrl": "<object url_1>" }, { "objectUrl": "<object url_2>", "title": "my-document-02.pdf" }, { "objectUrl": "<object url_3>", "title": "my-document-03.pdf", "pdfMetadata": { "fileSize": 300, "numPages": 5 } } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "pdfs": [ { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/my-document-01.pdf", }, { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/my-document-02.pdf", "title": "my-document-02.pdf" }, { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/my-document-3.pdf", "title": "my-document-03.pdf", "pdfMetadata": { "fileSize": 300, "numPages": 5 } } ], "skip_duplicate_urls": true } ``` </CodeGroup> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Text Files"> #### [Text Files](#text-files) The following are example JSON files for uploading text files to Encord. <CodeGroup> ```json Template theme={"dark"} { "text": [ { "objectUrl": "<object url_1>" }, { "objectUrl": "<object url_2>", "title": "my-file.html" }, { "objectUrl": "<object url_3>", "title": "my-plain-text-file.txt", "textMetadata": { "fileSize": 200, "mime_type": "text/plain" } }, { "objectUrl": "<object url_4>", "title": "my-csv-file.csv", "textMetadata": { "fileSize": 250, "mime_type": "text/csv" } }, { "objectUrl": "<object url_5>", "title": "my-xml-file.xml", "textMetadata": { "fileSize": 300, "mime_type": "application/xml" } }, { "objectUrl": "<object url_6>", "title": "my-html-file.html", "textMetadata": { "fileSize": 400, "mime_type": "text/html" } } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "text": [ { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/my-file-01.html" }, { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/my-plain-text-file-01.txt", "title": "my-plain-text-file-01.txt" }, { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/my-plain-text-file-02.txt", "title": "my-plain-text-file-02.txt", "textMetadata": { "fileSize": 200, "mime_type": "text/plain" } }, { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/my-csv-file-01.csv", "title": "my-csv-file-01.csv", "textMetadata": { "fileSize": 250, "mime_type": "text/csv" } }, { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/my-xml-file-01.xml", "title": "my-xml-file-01.xml", "textMetadata": { "fileSize": 300, "mime_type": "application/xml" } }, { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/my-html-file-01.html", "title": "my-html-file-01.html", "textMetadata": { "fileSize": 400, "mime_type": "text/html" } } ], "skip_duplicate_urls": true } ``` </CodeGroup> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Single images"> #### [Single Images](#single-images) For detailed information about the JSON file format used for import go [here](/platform-documentation/Curate/add-files/index-register-cloud-data#json-format). The JSON structure for single images parallels that of videos. **Template:** Provides the proper JSON format to import images into Encord. **Examples:** * **Data** Imports the images only. * **Image Metadata:** Imports images with image metadata. This improves the import speed for your images. <CodeGroup> ```json Template theme={"dark"} { "images": [ { "objectUrl": "file/path/to/images/file-name-01.file-extension" }, { "objectUrl": "file/path/to/images/file-name-02.file-extension" }, { "objectUrl": "file/path/to/images/file-name-03.file-extension", "title": "image-title.file-extension" } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "images": [ { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/0001.jpg" }, { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/0002.jpg" }, { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/0003.jpg" }, { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/DALL%C2%B7E+2022-09-08+18.53.25+-+firefighter+extinguishing+flames+around+computer+in+software+office+overgrown+by+foliage.png" }, { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/DALL%C2%B7E+2022-12-08+22.16.52+-+steampunk+combustion+engine+that+is+fueled+by+data+and+produces+computer+vision+algorithms.png" }, { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/large_images/pexels-ivo-rainha-00057.png" } ], "skip_duplicate_urls": true } ``` ```json Image Metadata theme={"dark"} { "images": [ { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/my_image.jpg", "imageMetadata": { "mimeType": "image/jpg", "fileSize": 124, "width": 640, "height": 480 } } ], "skip_duplicate_urls": true } ``` </CodeGroup> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Image groups"> #### [Image groups](#image-groups) For detailed information about the JSON file format used for import go [here](/platform-documentation/Curate/add-files/index-register-cloud-data#json-format). * Image groups are collections of images that are processed as one annotation task. * Images within image groups remain unaltered, meaning that images of different sizes and resolutions can form an image group without the loss of data. * Image groups do NOT require 'write' permissions to your cloud storage. * If `skip_duplicate_urls` is set to `true`, all URLs exactly matching existing image groups in the dataset are skipped. <Note>The position of each image within the sequence needs to be specified in the key (`objectUrl_{position_number}`).</Note> **Template:** Provides the proper JSON format to import image groups into Encord. **Examples:** * **Data:** Imports the image groups only. <CodeGroup> ```json Template theme={"dark"} { "image_groups": [ { "title": "<title 1>", "createVideo": false, "objectUrl_0": "file/path/to/images/file-name-01.file-extension", "objectUrl_1": "file/path/to/images/file-name-02.file-extension", "objectUrl_2": "file/path/to/images/file-name-03.file-extension", }, { "title": "<title 2>", "createVideo": false, "objectUrl_0": "file/path/to/images/file-name-01.file-extension", "objectUrl_1": "file/path/to/images/file-name-02.file-extension", "objectUrl_2": "file/path/to/images/file-name-03.file-extension", "clientMetadata": {"optional": "metadata"} } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "image_groups": [ { "title": "Image group 01", "createVideo": false, "objectUrl_0": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/0001.jpg", "objectUrl_1": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/0002.jpg", "objectUrl_2": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/DALL%C2%B7E+2022-09-08+18.53.25+-+firefighter+extinguishing+flames+around+computer+in+software+office+overgrown+by+foliage.png" }, { "title": "Image group 02", "createVideo": false, "objectUrl_0": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/thing-0001.jpg", "objectUrl_1": "<https://encord-bucket.obs.eu-de.otc.t-systems.com/images/thing-0002.jpg", "objectUrl_2": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/thing-0003.jpg" } ], "skip_duplicate_urls": true } ``` </CodeGroup> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Image sequences"> #### [Image sequences](#image-sequences) For detailed information about the JSON file format used for import go [here](/platform-documentation/Curate/add-files/index-register-cloud-data#json-format). * Image sequences are collections of images that are processed as one annotation task and represented as a video. * Images within image sequences may be altered as images of varying sizes and resolutions are made to match that of the first image in the sequence. * Creating Image sequences from cloud storage requires 'write' permissions, as new files have to be created in order to be read as a video. * Each object in the `image_groups` array with the `createVideo` flag set to `true` represents a single image sequence. * If `skip_duplicate_urls` is set to `true`, all URLs exactly matching existing image sequences in the dataset are skipped. <Tip> The only difference between adding image groups and image sequences using a JSON file is that image sequences require the `createVideo` flag to be set to `true`. Both use the key `image_groups`.</Tip> <Note>The position of each image within the sequence needs to be specified in the key (`objectUrl_{position_number}`).</Note> <Info>Encord supports up to 32,767 entries (21:50 minutes) for a single image sequence. We recommend up to 10,000 to 15,000 entries for a single image sequence for best performance. If you need a longer sequence, we recommend using video instead of an image sequence.</Info> **Template:** Provides the proper JSON format to import image groups into Encord. **Examples:** * **Data:** Imports the images groups only. <CodeGroup> ```json Template theme={"dark"} { "image_groups": [ { "title": "<title 1>", "createVideo": true, "objectUrl_0": "<object url>" }, { "title": "<title 2>", "createVideo": true, "objectUrl_0": "<object url>", "objectUrl_1": "<object url>", "objectUrl_2": "<object url>" } ], "skip_duplicate_urls": true } ``` ```json Data theme={"dark"} { "image_groups": [ { "title": "Image sequence 001", "createVideo": true, "objectUrl_0": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/01.jpg", "objectUrl_1": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/02.jpg", "objectUrl_2": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/DALL%C2%B7E+2022-09-08+18.53.25+-+firefighter+extinguishing+flames+around+computer+in+software+office+overgrown+by+foliage.png" }, { "title": "Image sequence 002", "createVideo": true, "objectUrl_0": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/thing-01.jpg", "objectUrl_1": "<https://encord-bucket.obs.eu-de.otc.t-systems.com/images/thing-02.jpg", "objectUrl_2": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/thing-03.jpg" } ], "skip_duplicate_urls": true } ``` </CodeGroup> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="DICOM"> #### [DICOM](#dicom) For detailed information about the JSON file format used for import go [here](/platform-documentation/Curate/add-files/index-register-cloud-data#json-format). * Each `dicom_series` element can contain one or more DICOM series. * Each series requires a title and at least one object URL, as shown in the example below. * If `skip_duplicate_urls` is set to `true`, all object URLs exactly matching existing DICOM files in the dataset will be skipped. <Note>Custom metadata is distinct from patient metadata, which is included in the `.dcm` file and does not have to be specific during the upload to Encord. </Note> The following is an example JSON for uploading three DICOM series belonging to a study. Each title and object URL correspond to individual DICOM series. * The first series contains only a single object URL, as it is composed of a single file. * The second series contains 3 object URLs, as it is composed of three separate files. * The third series contains 2 object URLs, as it is composed of two separate files. <Info> For each DICOM upload, an additional `DicomSeries` file is created. This file represents the series file-set. Only `DicomSeries` are displayed in the Encord application. </Info> ```json JSON for DICOM theme={"dark"} { "dicom_series": [ { "title": "Series-1", "objectUrl_0": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/study1-series1-file.dcm" }, { "title": "Series-2", "objectUrl_0": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/study1-series2-file1.dcm", "objectUrl_1": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/study1-series2-file2.dcm", "objectUrl_2": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/study1-series2-file3.dcm", }, { "title": "Series-3", "objectUrl_0": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/study1-series3-file1.dcm", "objectUrl_1": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/study1-series3-file2.dcm", } ], "skip_duplicate_urls": true } ``` </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="NIfTI"> #### [NIfTI](#nifti) The following is an example JSON file for uploading two NIfTI files to Encord. ```json theme={"dark"} { "nifti": [ { "title": "<file-1>", "objectUrl": "https://my-bucket/.../nifti-file1.nii" }, { "title": "<file-2>", "objectUrl": "https://my-bucket/.../nifti-file2.nii.gz" } ], "skip_duplicate_urls": true } ``` </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title=" Multiple file types "> You can upload multiple file types using a single JSON file. The example below shows 1 image, 2 videos, 2 image sequences, and 1 image group. ```json Multiple file types theme={"dark"} { "images": [ { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/Image1.png" } ], "videos": [ { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/videos/Cooking.mp4" }, { "objectUrl": "https://encord-bucket.obs.eu-de.otc.t-systems.com/videos/Oranges.mp4" } ], "image_groups": [ { "title": "apple-samsung-light", "createVideo": true, "objectUrl_0": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/1+(32).jpg", "objectUrl_1": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/1+(33).jpg", "objectUrl_2": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/1+(34).jpg", "objectUrl_3": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/1+(35).jpg" }, { "title": "apple-samsung-dark", "createVideo": true, "objectUrl_0": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/2+(32).jpg", "objectUrl_1": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/2+(33).jpg", "objectUrl_2": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/2+(34).jpg", "objectUrl_3": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/2+(35).jpg" } ], "image_groups": [ { "title": "apple-ios-light", "createVideo": false, "objectUrl_0": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/3+(32).jpg", "objectUrl_1": "https://encord-bucket.obs.eu-de.otc.t-systems.com/images/3+(33).jpg" } ], "skip_duplicate_urls": true } ``` </Accordion> </AccordionGroup> </Accordion> <Accordion title="JSON for AWS Multi-Region Access Point"> #### Use a Multi-Region Access Point When using a Multi-Region Access Point for your AWS S3 buckets the JSON file has to be slightly different from the examples provided. Instead of an object's URL, objects are specified using the ARN of the Multi-Region Access Point followed by the object name. The example below shows how video files from a Multi-Region Access Point would be specified. ```json theme={"dark"} { "videos": [ { "objectUrl": "Multi-Region-Access-Point-ARN + <object name_1>" }, { "objectUrl": "Multi-Region-Access-Point-ARN + <object name_2>", "title": "my-custom-video-title.mp4" } ], "skip_duplicate_urls": true } ``` ```json MRAP Example theme={"dark"} { "videos": [ { "objectUrl": "https://arn:aws:s3::123123123:accesspoint/frf28frarf9.mrap.s3-accesspoint.amazonaws.com/Videos/2022/video_1.mp4" }, { "objectUrl": "https://arn:aws:s3::123123123:accesspoint/frf28frarf9.mrap.s3-accesspoint.amazonaws.com/Videos/2022/video_2.mp4", "title": "many-cute-cats.mp4" } ], "skip_duplicate_urls": true } ``` </Accordion> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="CSV Format"> #### Create CSV File for Registration In the CSV file format, the column headers specify which type of data is being uploaded. You can add and single file format at a time, or combine multiple data types in a single CSV file. Details for each data format are given in the sections below. <Info>Encord supports up to 10,000 entries for upload in the CSV file.</Info> <Warning> * Object URLs can't contain whitespace. * For backwards compatibility reasons, a single column CSV is supported. A file with the single `ObjectUrl` column is interpreted as a request for video upload. If your objects are of a different type (for example, images), this error displays: "Expected a video, got a file of type XXX". </Warning> <AccordionGroup> <Accordion title="Videos"> #### Videos A CSV file containing videos should contain two columns with the following mandatory column headings:\ 'ObjectURL' and 'Video title'. All headings are case-insensitive. * The 'ObjectURL' column containing the `objectUrl`. This field is mandatory for each file, as it specifies the full URL of the video resource. * The 'Video title' column containing the `video_title`. If left blank, the original file name is used. In the example below files 1, 2 and 4 will be assigned the names in the title column, while file 3 will keep its original file name. | ObjectUrl | Video title | | ----------------------------------- | ----------- | | path/to/storage-location/frame1.mp4 | Video 1 | | path/to/storage-location/frame2.mp4 | Video 2 | | path/to/storage-location/frame3.mp4 | | | path/to/storage-location/frame4.mp4 | Video 3 | </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Single images"> A CSV file containing single images should contain two columns with the following mandatory headings:\ 'ObjectURL' and 'Image title'. All headings are case-insensitive. * The 'ObjectURL' column containing the `objectUrl`. This field is mandatory for each file, as it specifies the full URL of the image resource. * The 'Image title' column containing the `image_title`. If left blank, the original file name is used. In the example below files 1, 2 and 4 will be assigned the names in the title column, while file 3 will keep its original file name. | ObjectUrl | Image title | | ----------------------------------- | ----------- | | path/to/storage-location/frame1.jpg | Image 1 | | path/to/storage-location/frame2.jpg | Image 2 | | path/to/storage-location/frame3.jpg | | | path/to/storage-location/frame4.jpg | Image 3 | </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Image groups"> #### Image groups A CSV file containing image groups should contain three columns with the following mandatory headings:\ 'ObjectURL', 'Image group title', and 'Create video'. All three headings are case-insensitive. * The 'ObjectURL' column containing the `objectUrl`. This field is mandatory for each file, as it specifies the full URL of the resource. * The 'Image group title' column containing the `image_group_title`. This field is mandatory, as it determines which image group a file will be assigned to. In the example below the first two URLs are grouped together into 'Group 1', while the following two files are grouped together into 'Group 2'. | ObjectUrl | Image group title | Create video | | ----------------------------------- | ----------------- | ------------ | | path/to/storage-location/frame1.jpg | Group 1 | false | | path/to/storage-location/frame2.jpg | Group 1 | false | | path/to/storage-location/frame3.jpg | Group 2 | false | | path/to/storage-location/frame4.jpg | Group 2 | false | <Note>Image groups do not require 'write' permissions.</Note> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Image sequences"> #### Image sequences A CSV file containing image sequences should contain three columns with the following mandatory headings: 'ObjectURL', 'Image group title', and 'Create video'. All three headings are case-insensitive. * The 'ObjectURL' column containing the `objectUrl`. This field is mandatory for each file, as it specifies the full URL of the resource. * The 'Image group title' column containing the `image_group_title`. This field is mandatory, as it determines which image sequence a file will be assigned to. The dimensions of the image sequence are determined by the first file in the sequence. * The 'Create video' column. This can be left blank, as the default value is 'true'. In the example below the first two URLs are grouped together into 'Sequence 1', while the second two files are grouped together into 'Sequence 2'. | ObjectUrl | Image group title | Create video | | ----------------------------------- | ----------------- | ------------ | | path/to/storage-location/frame1.jpg | Sequence 1 | true | | path/to/storage-location/frame2.jpg | Sequence 1 | true | | path/to/storage-location/frame3.jpg | Sequence 2 | true | | path/to/storage-location/frame4.jpg | Sequence 2 | true | <Tip>Image groups and image sequences are only distinguished by the presence of the 'Create video' column.</Tip> <Note>Image sequences require 'write' permissions against your storage bucket to save the compressed video.</Note> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="DICOM"> A CSV file containing DICOM files should contain two columns with the following mandatory headings: 'ObjectURL' and 'Dicom title'. Both headings are case-insensitive. * The 'ObjectURL' column containing the `objectUrl`. This field is mandatory for each file, as it specifies the full URL of the resource. * The 'Series title' column containing the `dicom_title`. When two files are given the same title they are grouped into the same DICOM series. If left blank, the original file name is used. In the example below the first two files are grouped into 'dicom series 1', the next two files are grouped into 'dicom series 2', while the final file will remain separated as 'dicom series 3'. | ObjectUrl | Series title | | ----------------------------------- | -------------- | | path/to/storage-location/frame1.dcm | dicom series 1 | | path/to/storage-location/frame2.dcm | dicom series 1 | | path/to/storage-location/frame3.dcm | dicom series 2 | | path/to/storage-location/frame4.dcm | dicom series 2 | | path/to/storage-location/frame5.dcm | dicom series 3 | </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title=" Multiple file types "> #### Multiple file types You can upload multiple file types with a single CSV file by using a new header each time there is a change of file type. Three headings will be required if image sequences are included. <Warning>Since the 'Create video' column defaults to `true` all files that **are not** image sequences must contain the value `false`</Warning> The example below shows a CSV file for the following: * Two image sequences composed of 2 files each. * One image group composed of 2 files. * One single image. * One video. | ObjectUrl | Image group title | Create video | | ----------------------------------- | ----------------- | ------------ | | path/to/storage-location/frame1.jpg | Sequence 1 | true | | path/to/storage-location/frame2.jpg | Sequence 1 | true | | path/to/storage-location/frame3.jpg | Sequence 2 | true | | path/to/storage-location/frame4.jpg | Sequence 2 | true | | path/to/storage-location/frame5.jpg | Group 1 | false | | path/to/storage-location/frame6.jpg | Group 1 | false | | ObjectUrl | Image title | Create video | | path/to/storage-location/frame1.jpg | Image 1 | false | | ObjectUrl | Image title | Create video | | full/storage/path/video.mp4 | Video 1 | false | </Accordion> </AccordionGroup> </Accordion> </AccordionGroup> *** ### Import or Register your Data <AccordionGroup> <Accordion title="Register Cloud Data"> #### Register Cloud Data <Tip>To ensure smoother uploads and faster completion times, and avoid hitting absolute file limits, we recommend adding smaller batches of data. Limit uploads to 100 videos or up to 1,000 images at a time. You can also create multiple Datasets, all of which can be linked to a single Project. Familiarize yourself with our [limits and best practices for data import/registration](/platform-documentation/General/best-practices#import-into-encord) before adding data to Encord. </Tip> 1. Navigate to **Data** > **Files & Folders** in the Encord platform. 2. Click into a Folder. 3. Click **+ Upload files**. A dialog appears. <div class="flex justify-center"> <img src="https://storage.googleapis.com/docs-media.encord.com/static/img/upload-files-new.png" width="800" /> </div> 4. Click **Import from cloud data**. <div class="flex justify-center"> <img src="https://storage.googleapis.com/docs-media.encord.com/static/img/Index/upload-data-dialog-04.png" width="550" /> </div> <Tip>We recommend turning on the **Ignore individual file errors** feature. This ensures that individual file errors do not lead to the whole upload process being aborted. </Tip> 6. Click **Add JSON or CSV files** to add a [JSON](/platform-documentation/Curate/add-files/index-register-cloud-data#json-format) or [CSV](/platform-documentation/Curate/add-files/index-register-cloud-data#csv-format) file specifying cloud data that is to be added. </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Import Local Data"> #### Import Local Data <Tip>To ensure smoother uploads and faster completion times, and avoid hitting absolute file limits, we recommend adding smaller batches of data. Limit uploads to 100 videos or up to 1,000 images at a time. You can also create multiple Datasets, all of which can be linked to a single Project. Familiarize yourself with our [limits and best practices for data import/registration](/platform-documentation/General/best-practices#import-into-encord) before adding data to Encord. </Tip> 1. Navigate to **Data** > **Files & Folders** in the Encord platform. 2. Click into a Folder. 3. Click **+ Upload files**. A dialog appears. <img src="https://storage.googleapis.com/docs-media.encord.com/static/img/upload-files-new.png" width="600" /> 4. Click one of the following: * **Upload:** Upload images, videos, and audio files. ![Upload dialog](https://storage.googleapis.com/docs-media.encord.com/static/img/Index/upload-data-dialog-001.png) * **Batch images as:** Upload image batches as image groups or image sequences. ![Batch images as](https://storage.googleapis.com/docs-media.encord.com/static/img/Index/upload-data-dialog-02.png) * **DICOM/NifTi:** Upload DICOM or NifTi series. ![DICOM/NifTi](https://storage.googleapis.com/docs-media.encord.com/static/img/Index/upload-data-dialog-03.png) 5. Click **Upload** after selecting your images or series. Your files upload into the Folder in Encord. </Accordion> </AccordionGroup> ## STEP 2: Curate your Data Curation of your data consists of sorting, filtering, using analytics, and embedding plots to create one or more Collections. Collections can then be made into a Dataset for use with your Annotate Projects. <div style={{ height: '0', paddingBottom: '54.93387589013224%', position: 'relative' }} > <iframe allowFullScreen frameBorder="0" mozallowfullscreen="" src="https://www.loom.com/embed/eb71850421204a72a0902f0e0db3d2e6?sid=8dbd2b96-6463-4970-b644-e7bda64e8477" style={{ height: '100%', left: '0', position: 'absolute', top: '0', width: '100%' }} webkitallowfullscreen="" /> </div> <Steps titleSize="h3"> <Step title="Filter and Sort Data"> You can sort your data using a number of data unit metrics. The metrics available to sort your data depends on the data available in the Folder. You can sort your data in ascending and descending order. Data > Explore has a number of criteria available to filter your data. You can only access filtering criteria that is available from the top-level Folder. For example, if you do not have any video or audio data in the top-level Folder or any sub-Folders, you cannot filter your data using those filtering criteria. <Tip>Adding [key frames (for videos), custom metadata](/platform-documentation/Curate/custom-metadata/index-custom-metadata), and [custom embeddings](/platform-documentation/Curate/custom-embeddings)) to your data can all significantly improve the speed and effectiveness when filtering your data. We recommend importing customer metadata and custom embeddings with your data, but you can import custom metadata and embeddings after importing your data.</Tip> ![Sort and Filter](https://storage.googleapis.com/docs-media.encord.com/static/img/Index/sort-filtering-using-02.gif) <AccordionGroup> <Accordion title="File Quality Metrics"> <table> <thead> <tr> <th>Criteria</th> <th>Description</th> </tr> </thead> <tbody> <tr> <td> File Size </td> <td> Filters files (video) based on their size in bytes. </td> </tr> <tr> <td> Random Value </td> <td> Filters files (video) based on a random value assigned as the file is added to Data > Explore. </td> </tr> </tbody> </table> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Video Quality Metrics"> <table> <thead> <tr> <th>Criteria</th> <th>Description</th> </tr> </thead> <tbody> <tr> <td> Area </td> <td> Filters videos based on the area of the frame. </td> </tr> <tr> <td> Aspect Ratio </td> <td> Filters videos based on the aspect ratio of the video. </td> </tr> <tr> <td> Clip Duration </td> <td> Filters videos based on the duration of the video by hundredths of seconds. </td> </tr> <tr> <td> Frame per Second </td> <td> Filters videos based on the frames per second of the video. </td> </tr> <tr> <td> Height </td> <td> Filters videos based on the height of the video frame. </td> </tr> <tr> <td> No. of Video Frames </td> <td> Filters videos based on the number of frames in the video. </td> </tr> <tr> <td> Random Value </td> <td> Filters videos based on a random value assigned as the file is added to Data > Explore. </td> </tr> <tr> <td> Width </td> <td> Filters videos based on the width of the video. </td> </tr> </tbody> </table> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Frame Quality Metrics"> <table> <thead> <tr> <th>Criteria</th> <th>Description</th> </tr> </thead> <tbody> <tr> <td> Area </td> <td> Filters images/video frames based on the area of the frame. </td> </tr> <tr> <td> Aspect Ratio </td> <td> Filters images/video frames based on the aspect ratio of the video. </td> </tr> <tr> <td> Frame Number </td> <td> Filters images/video frames based on the height of the video frame. </td> </tr> <tr> <td> Height </td> <td> Filters images/video frames based on the height of the video frame. </td> </tr> <tr> <td> Random Value </td> <td> Filters images/video frames based on a random value assigned as the file is added to Data > Explore. </td> </tr> <tr> <td> Width </td> <td> Filters images/video frames based on the width of the video. </td> </tr> </tbody> </table> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Audio Quality Metrics"> <table> <thead> <tr> <th>Criteria</th> <th>Description</th> </tr> </thead> <tbody> <tr> <td> Bit Depth </td> <td> Filters audio files based on the size of each sample (int) in bits. </td> </tr> <tr> <td> Duration </td> <td> Filters audio files based on the duration of the audio file by hundredths of seconds. </td> </tr> <tr> <td> Number of Channels </td> <td> Filters audio files based on number of audio channels in the audio file. </td> </tr> <tr> <td> Random Value </td> <td> Filters audio files based on a random value assigned as the file is added to Data > Explore. </td> </tr> <tr> <td> Samples per Second </td> <td> Filters audio files based on the number of samples per second. </td> </tr> </tbody> </table> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Custom Metadata, Custom Embeddings, Key Frames"> <table> <thead> <tr> <th>Criteria</th> <th>Description</th> </tr> </thead> <tbody> <tr> <td> Custom Metadata </td> <td> Filters files by any custom metadata available on your data.<br /> <Tip>For more information about custom metadata with Data > Explore, [refer to our documentation](/platform-documentation/Curate/custom-metadata/index-custom-metadata)</Tip> </td> </tr> <tr> <td> Custom Embeddings </td> <td> Filters files by any custom embeddings available on your data.<br /> <Tip>For more information about custom embeddings with Data > Explore, [refer to our documentation](/platform-documentation/Curate/custom-embeddings)</Tip> </td> </tr> <tr> <td> Key Frames </td> <td> **VIDEO ONLY** <br /> Filters files by any key frames specified on frames in your video data. <Note>Specifying key frames on videos significantly speeds up importing videos in to Data > Explore. <br /> Data > Explore imports videos at 1 FPS by default.</Note> </td> </tr> </tbody> </table> </Accordion> </AccordionGroup> <AccordionGroup> <Accordion title="Other Filtering Criteria"> <table> <thead> <tr> <th>Criteria</th> <th>Description</th> </tr> </thead> <tbody> <tr> <td> Collection </td> <td> Filters files by the Collection the files belong to. </td> </tr> <tr> <td> Data Title </td> <td> Filters files their file name. </td> </tr> <tr> <td> Data Type </td> <td> Filters files their file type: Video, Audio, Image, Image Group, Image Sequence, DICOM Series </td> </tr> <tr> <td> File ID </td> <td> Filters files their unique file ID. </td> </tr> <tr> <td> Folder </td> <td> Filters files by the Folders that contain the files. </td> </tr> <tr> <td> Integration </td> <td> Filters files by the Integrations that contain the files. </td> </tr> <tr> <td> Mime </td> <td> Filters files the file's Mime type. </td> </tr> <tr> <td> Storage Location </td> <td> Filters files by the storage location that contains the files. </td> </tr> <tr> <td> Uploaded At </td> <td> Filters files the time and date of their upload. </td> </tr> </tbody> </table> </Accordion> </AccordionGroup> *** </Step> <Step title="Use Custom Analytic Dashboards"> You can create custom analytic dashboards using Distribution and Correlation charts. For more information on analytics dashboards, [refer to our documentation](/platform-documentation/Curate/index-custom-analytics-dashboard). <Tip>Adding [custom metadata](/platform-documentation/Curate/custom-metadata/index-custom-metadata) to your data can significantly improve the insights you can gather from your data. We recommend importing customer metadata with your data, but you can import custom metadata after importing your data.</Tip> ![Custom Analytics Dashboard](https://storage.googleapis.com/docs-media.encord.com/static/img/Index/custom-analytics-dashboard-using-02.gif) *** </Step> <Step title="Custom Embeddings and Similarity Search"> <Tip>Adding [custom embeddings](/platform-documentation/Curate/custom-embeddings) to your data can significantly improve the insights you can gather from your data. We recommend importing customer embeddings with your data, but you can import custom embeddings after importing your data.</Tip> <Warning>You must **Upgrade** the top-level Folder and select the custom embeddings to use embedding plot graphs and similarity search.</Warning> For information on upgrading your Folder and importing custom embeddings, [refer to our documentation](/platform-documentation/Curate/custom-embeddings). **Custom Embeddings** ![Custom Embedding](https://storage.googleapis.com/docs-media.encord.com/static/img/Index/custom-embeddings-using-02.gif) **Similarity Search** ![Similarity Search](https://storage.googleapis.com/docs-media.encord.com/static/img/Index/similarity-search-using-02.gif) *** </Step> <Step title="Create a Collection"> Looking at the existing snippet content and the feature changes, I need to update the `CreateCollection.mdx` snippet to reflect the new collection actions submenu behaviour — specifically the copyable collection ID and grouped actions. The snippet covers the workflow for creating a collection in the Explorer, so I'll add information about the actions menu that users encounter after a collection is created. The existing snippet content shows steps for creating a collection. The new feature adds UI changes to the `...` actions submenu on collections in the Explorer filter panel. Since this snippet is reused across multiple pages, I'll add a concise section about the collection actions submenu that is broadly applicable. ```mdx theme={"dark"} 1. Log in to the Encord platform. The landing page for the Encord platform appears. 2. Navigate to **Data** > **Explore**. The _All folders_ page appears with a list of all folders in Encord. 3. Open a folder. 4. Click the **Explorer** button. The _Index Explorer_ page appears. ![Access Index Explorer](https://storage.googleapis.com/docs-media.encord.com/static/img/Index/index-access-index.gif) 5. Search, sort, and filter your data until you have the subset of the data you need. ![Sort and Filter](https://storage.googleapis.com/docs-media.encord.com/static/img/Index/sort-and-filter.gif) 6. Select one or more of the images/frames/audio files in the Explorer workspace. A ribbon appears at the top of the Explorer workspace. <Note>Selecting a video frame selects the entire video. Specific frames from a video cannot be selected.</Note> ![Create Collection in Index](https://storage.googleapis.com/docs-media.encord.com/static/img/Index/index-create-collection.gif) 7. Click **Select all** to select all the files in the subset. <Note>Use <kbd>Command + A</kbd> as a short cut to select all files.</Note> 8. Click **Add to a Collection**. 9. Click **New Collection**. 10. Specify a meaningful title and description for the Collection. <Note> The title specified here is applied as a tag/label to every selected image.</Note> 11. Click **Collections** to verify the Collection appears in the Collections list. ## Collection actions menu Each collection in the Explorer filter panel has a **...** (horizontal dots) button that opens a collection actions submenu. The submenu provides quick access to all available actions for that collection. The submenu displays: - The **collection name** in bold at the top of the menu. - The **collection ID** beneath the name. Click the copy icon next to the ID to copy it to your clipboard. - **Actions grouped into sections** separated by dividers, making it easier to find the right action. Actions are organised into the following groups: - **Navigate** — for example, View in Explorer. - **Manage** — for example, Edit Details. - **Data** — for example, Add to Dataset and Download CSV. - **Workflow** — for example, Task Actions, Bulk Classify, Convert Collection, and Edit Object Class. These actions appear only when you are working within a project. - **Danger** — for example, Delete. <Note>Sections with no available actions for your current context are hidden automatically — no empty dividers appear.</Note> <Note>The grouped layout and copyable collection ID apply only to the collection actions submenu in the Explorer filter panel. The collections table row menu is unchanged.</Note> ``` </Step> </Steps> ## STEP 3: Create a Dataset Once you have a Collection, you can create a Dataset from your Collection.