TLDR;
Do you already know what you are doing and only want to look over a Jupyter Notebook example? We provide one here.Active’s Default Embeddings
Currently in Active, we calculate and display embeddings using a generic CLIP model. This model is excellent for a wide range of tasks and across the board performance. However, a generic CLIP model may struggle with highly specialized tasks. The embeddings generated from the generic model are used for: Natural Language Search (not supported for custom embeddings), image similarity search and the embeddings view (where you view reduced embeddings).Custom Embeddings Support
We currently support embeddings of dimensions ranging from 1 to 4096 for Index, and 1 to 2000 for Active, following on from our in-house clip Embeddings. We support embeddings for images, image sequences and image groups. Support for Videos is coming soon.Use Custom Embeddings in Active
To use custom embeddings in Active:- Add an Embedding type to your custom metadata schema.
- Upload your embeddings as custom metadata.
- Create an Annotate Project.
- Select your custom embeddings as you import the Project to Active.
Step 1: Create an Embedding Field
A key is required in your custom metadata schema for your embeddings. You can use any string as the key for your embeddings. We strongly recommend that you use a string that is meaningful. If you do not include a key in your metadata schema, your imported embeddings are treated as strings.Embedding key names can contain alphanumeric (a-z, A-Z, 0-1) characters, hyphens, and underscores.
add_embedding
to add an embedding to your metadata schema.
Key | Description | Display Benefits |
---|---|---|
embedding | 1 to 4096 for Index. 1 to 2000 for Active | Filtering by embeddings, similarity search, 2D scatter plot visualization (Coming Soon) |
Step 2: Upload Embeddings
With the key in the custom metadata schema ready, we can now import our embeddings. Custom embedding sizes are flexible and can be set anywhere between 1 and 4096. You can import embeddings after you have added your data or during your data registration.Your key frames (frames specified with or without embeddings) always appear in Index, regardless of what sampling rate you specify.
Embedding key names can contain alphanumeric (a-z, A-Z, 0-1) characters, hyphens, and underscores.
config
is not specified, the sampling_rate
is 1 frame per second, and the keyframe_mode
is frame
.
Specifying a
sampling_rate
of 0
only imports the first frame and all keyframes of your video into Index.Import while registering videos
Import while registering videos
Import while importing videos
This JSON file imports embeddings while registering your data with Index from a cloud integration.config
is optional when importing your custom embeddings:config
is not specified, the sampling_rate
is 1 frame per second, and the keyframe_mode
is frame
.Specifying a
sampling_rate
of 0
only imports the first frame and all keyframes of your video into Index.Update specific videos
Update specific videos
Update specific videos
Import while registering data units
Import while registering data units
Import while importing data units
This JSON file imports embeddings while registering your data with Index from a cloud integration.Update specific data units
Update specific data units
Import specific data units
The custom embeddings format for images, text files, PDFs, and audio files follows the same format as importing custom metadata.Step 3 : Create a Project in Annotate
The following task outlines the basics of creating an Annotate Project. For detailed instructions, refer to the documentation here. To create an Annotate Project:- Go to Annotate > Projects. The Annotate Projects list appears.
- Click New annotation project. The Create New Project page appears.
- Provide a meaningful title and description.
- Select an Ontology.
- Select one or more Datasets.
- Load a Workflow template.
- Click Create Project.
Step 4: Import Project with Custom Embeddings
Before you can use your custom embeddings in Encord Active Projects, you need to import the custom embeddings. This is performed while you import your Annotate Project into Active. For existing Active Projects, you can import custom embeddings for your Project import if your Project imported to Data or Labels. Importing to Metrics & Embeddings requires deleting the Project in Active and re-importing the Project with your custom embeddings.After updating your embeddings, sync your Active Project to automatically apply the new embeddings.
- Log in to the Encord platform. The landing page for the Encord platform appears.
- Create a Project (Annotation Project or Training Project) in Encord Annotate.
- Click Active from the main menu. The landing page for Active appears.
- Click the Import Annotate Project button. The Import an Annotate Project to Encord Active dialog appears.
-
Click the Import button for the Annotate Project you want to import.
The Confirm Project Import dialog appears.
.
- Select the custom embeddings you want to import to the Project, under Metrics & Embeddings.
- Select Import under Metrics & Embeddings.
- Click Proceed. The Annotate Project imports with your custom embeddings.