Label Ingestion
Label ingestion lets you store annotation data as sidecar files alongside your files in a cloud storage folder. When a folder is configured for label ingestion, Encord automatically detects sidecar files during bucket scans, pairs them with their corresponding data files, and ingests their contents as labels against a chosen ontology.
Metadata ingestion (suffix .metadata.json) is a separate configuration on the same folder and operates independently. Both can be active simultaneously.
How it Works
During each scheduled bucket scan, Encord classifies every file in the folder. Files whose names match the configured suffix are identified as label sidecars and excluded from the data file list. Each sidecar is paired with its corresponding data file, and the label payload is downloaded, validated against the ontology, and written to the database.
Under the hood, enabling label ingestion provisions labels linked to the Folder and the chosen Ontology.
The ingestion pipeline using a sidecar file:
- Labels check: If an existing sidecar file matches the last ingestion skip (no change).
- Download sidecar: Sidecar files are imported from cloud storage (max 300 MB).
Labels are imported as ground truth labels.
- UUID substitution: Placeholder UUIDs sent by the SDK are replaced with server-allocated values.
- Ontology resolution: The specified Ontology is loaded and Encord converts the payload into the Ontology’s label format.
- Label write: Labels appear on the data units.
If a sidecar is removed from the bucket, the corresponding label are removed from the data unit.
Configuration
Label ingestion is configured per folder using Storage > [Folder Name] > Folder details > Label ingestion.
Enabling Label Ingestion
When creating or editing a Cloud-Synced folder:
- Toggle Label ingestion on.
- Set the file suffix that marks a file as a label sidecar (default:
.labels.json).
- Select the Ontology to validate and store labels against.
Once saved, subsequent scans begin ingesting labels matching sidecars.
Configuration Options
| Option | Default | Description |
|---|
| Suffix | .labels.json | File suffix that identifies a sidecar. Any bucket key ending in this string is treated as a label file; the paired data key is the full key with the suffix removed. |
| Ontology | (required) | The Ontology used to validate ingested annotations. Cannot be changed after label ingestion is enabled. |
| Paused | false | When paused, sidecars are classified and excluded from data ingestion but are not downloaded or ingested. No new labels are written. Existing labels are preserved. |
Ontology lock-in
The ontology is fixed at the point of enabling label ingestion. Changing ontology invalidates all already-ingested label rows. To use a different Ontology, delete the label ingestion configuration and reconfigure the label ingestion settings. Existing ingested labels are preserved but no new sidecars are ingested under the new Ontology until the folder is reconfigured.
File Pairing
Suffix mode (default)
Files are paired by stripping the suffix from the sidecar key to derive the data key.
Example with suffix .labels.json:
| Data file | Label sidecar |
|---|
dataset/image001.jpg | dataset/image001.jpg.labels.json |
videos/clip_42.mp4 | videos/clip_42.mp4.labels.json |
scans/ct_scan.dcm | scans/ct_scan.dcm.labels.json |
Pausing and Resuming
Pausing stops new sidecars from being downloaded and ingested without removing existing labels. This is the recommended way to temporarily halt label ingestion. For example, while updating your sidecar generation pipeline.
To pause or resume: Folder details > Label ingestion > Pause / Resume.
Disabling Label Ingestion
Deleting the label ingestion configuration removes the config from the folder. Already-ingested labels are preserved. No new sidecars are picked up after deletion.
To disable: Folder details > Label ingestion > Delete (trash icon) > Confirm.
Supported Data Types
Label sidecars can be paired with any data type supported by cloud-synced folders:
- Images
- Videos
- Audio
- Text
- PDFs
- Time series
