Metadata Schema
Before importing your custom metadata to Encord, we recommend that you import a metadata schema. Encord uses metadata schemas to validate custom metadata uploaded to Encord and to instruct Index and Active how to display your metadata.
video.description
, while team B could use audio.description
. Another example could be TeamName.MetadataKey
. This approach maintains clarity and avoids key collisions across departments.Benefits of Using a Metadata Schema
Using a metadata schema provides several benefits:
- Validation: Ensures that all custom metadata conforms to predefined data types, reducing errors during data import and processing.
- Consistency: Maintains uniformity in data types across different datasets and projects, which simplifies data management and analysis.
- Filtering and Sorting: Enhances the ability to filter and sort data efficiently in the Encord platform, enabling more accurate and quick data retrieval.
Metadata Schema Table
varchar
as a versatile default. Use .add_scalar() to add a scalar key to your metadata schema.
Scalar Key | Description | Display Benefits |
---|---|---|
boolean | Binary data type with values “true” or “false”. | Filtering by binary values |
datetime | ISO 8601 formatted date and time. | Filtering by time and date |
number | Numeric data type supporting float values. | Filtering by numeric values |
uuid | Customer specified unique identifier for a data unit. | Filtering by customer specified unique identifier |
varchar | Textual data type. Formally string . string can be used as an alias for varchar , but we STRONGLY RECOMMEND that you use varchar . | Filtering by string. |
text | Text data with unlimited length (example: transcripts for audio). Formally long_string . long_string can be used as an alias for text , but we STRONGLY RECOMMEND that you use text . | Storing and filtering large amounts of text. |
Use add_enum
and add_enum_options
to add an enum and enum options to your metadata schema.
Key | Description | Display Benefits |
---|---|---|
enum | Enumerated type with predefined set of values. | Facilitates categorical filtering and data validation |
Use add_embedding
to add an embedding to your metadata schema.
Key | Description | Display Benefits |
---|---|---|
embedding | 1 to 4096 for Index. 1 to 2000 for Active. | Filtering by embeddings, similarity search, 2D scatter plot visualization (Coming Soon) |
Incorrectly specifying a data type in the schema can cause errors when filtering your data in Index or Active. If you encounter errors while filtering, verify your schema is correct. If your schema has errors, correct the errors, re-import the schema, and then re-sync your Active Project.
Import Your Metadata Schema to Encord
Verify Your Schema
After importing your schema to Encord we recommend that you verify that the import is successful. Run the following code to verify your metadata schema imported and that the schema is correct.
Edit Schema Keys
You can change the data type of schema keys using the .set_scalar()
method. The example below shows how to update the data type for multiple metadata fields.
Delete Schema Keys
You can delete schema keys using the .delete() method.
There are two types of deletion: hard delete and soft delete. A hard delete permanently removes the key, making it impossible to restore. A soft delete allows you to restore the key later using the .restore_key() method.
The following examples show hard delete and soft deletion of a schema key called Fruit
.
Restore Schema Keys
Keys that have been soft deleted can be restored using the .restore_key() method. The following example restores a schema key called Fruit
.
Was this page helpful?