This documentation provides a comprehensive guide to using the Label Editor for annotation, covering its components, features, and the display of various elements.
For information on how to label, see our documentation here.
The following video goes through the basics of annotating images in the Label Editor.
The following video goes through the basics of annotating videos in the Label Editor.
Annotation mode components:
Using the copy and paste features in the Label Editor can significantly save time and effort. Here’s how you can efficiently use these features:
Copy: To copy both the label and the instance, use the keyboard shortcut CMD + C on Mac or Ctrl + C on Windows.
Paste instance: To paste the copied instance, use CMD + V on Mac or Ctrl + V on Windows. Instances can only be pasted onto a different frame from where they were copied.
Paste Label: To paste just the label / shape, use CMD + Shift + V on Mac or Ctrl + Shift + V on Windows. Labels can be pasted onto any frame.
Fixing incorrect labels is an important step in producing high-quality datasets. With bulk label operations, annotators can merge labels which should really be one instance, or efficiently delete multiple labels in bulk.
Bulk label operations are accessed in the editor menu.
Merge operations
Merge one or more label objects into a single object.
Delete operations
Encord lets you perform bulk delete operations as follows:
Remove labels of a specific class:
Delete all labels within a frame range:
Delete labels below a specified confidence level:
In annotation mode, the Classes section of the Label Editor shows you the available Ontology classes, both objects and classifications. You can create a new instance by either clicking the desired class, or by using the indicated instantiation hotkeys. Learn more about Ontologies in general here or learn more about the details of Ontology structure on Encord.
The canvas is your labeling area to annotate the instances of the Ontology classes.
An instance is a specific occurrence of a class. For example, if you have a class called “Car”, an instance of this class could be “Car(0)”, which might represent a specific black sedan. This single instance can appear in a single frame or a range of frames, and therefore, instances may contain multiple labels across frames. You can think of an instance as a unique object that belongs to a certain class.
A frame label is a frame-specific annotation of an instance. For example, the annotation of “Car(0)” on frame 201 is a label. Labels are used to annotate instances in specific frames, providing more detailed information about the instance in the context of that frame.
Submit tasks
Labels are applied in the Label Editor. Click the Submit button to submit your labels for review.
Skip tasks
Users have the option to skip tasks in cases where, for example, the file is blurred, or if they suspect the task was erroneously added.
Skipping a task automatically creates an Issue and is therefore counted in the Issue Analytics.
When a user skips a task, it is automatically unassigned from them, and the task’s status is updated to Skipped. If another task is available, it automatically opens in the Label Editor and is assigned to the user.
Turbo mode simplifies the standard annotation workflow, enabling annotators to rapidly assign or confirm attributes. Activate turbo mode and choose an instance from the top-left dropdown menu to set the instance and its relevant attributes.
Enable turbo mode:
The Annotator instructions button is available in the top-left corner of the Label Editor if annotation instructions are provided by the Admin or Team Manager.
Each instance or label is assigned a confidence score (denoted with the symbol α) that appears next to the label name.
The confidence score is a measure of a machine learning model’s certainty that a given prediction is accurate. The higher the confidence score, the more certain a model is about its prediction.
Manual labels are always assigned α = 100%, while label predictions created via models and automated methods such as interpolation will have a confidence score below 100% (α < 100%).
Models allow you to select a Minimum confidence threshold that determines the lowest confidence that label predictions can have to appear as annotations. It ranges from 0 to 1.
A Minimum confidence threshold of 0.6, as seen in the image above, states that only labels above a confidence score α = 60% will be accepted as labels.
The Labels pane shows all the instantiated objects and classifications for a data unit. Objects are grouped first by class, and sorted by creation time within each class, with earlier objects on top. They are named in an “ABC (0)” format, where ABC is the Ontology class and 0 is the first instance annotation.
Click the three dots icon to access additional actions:
Copy URL: Copies the URL for the particular object.
Copy identifier: Each object instance is automatically assigned a unique alphanumeric ID, which can be used to identify it within the project. This ID is sometimes known alternately as a feature hash, instance hash, or instance identifier. For objects and classifications, it may also be known as the object hash or classification hash. Use this button to copy the instance identifier to your clipboard — which may help in workflows using specific instances in source code or for other purposes.
Track from this frame: Initiates single object tracking using SAM 2. This type of automated labeling creates labels for the specified object from the current frame, according to the Single object tracking settings section in the editor settings.
Interpolate: Interpolates this object’s location for a range between specified frames. See our documentation on interpolation for more details.
Go to object: Navigate to the specified object in the data asset. If it is present in more than one frame, the first frame the object appears is shown.
Split object track: In some cases, particularly when many objects overlap, labels may be associated with the correct instance in several frames, but later become associated with a different instance. In those cases, use the Split object track feature, which will split the labels from one instance into two at the frame specified.
Change class: Use the drop-down in the modal to change the current class of an instance. You can only switch within classes of the same annotation type. For example, you cannot switch from a bounding box to a polygon.
You can select multiple labels and bulk-change their classes by right-clicking and choosing Change Class.
Delete object: Delete an object from a single frame or a range of frames. To delete labels between frame ranges, move the frame slider to the desired frames, or input the From and To (exclusive) values in the relevant form.
The details pane allows you to modify attributes efficiently. For dynamic attributes, use the ‘Preserve chosen state’ feature. When this feature is toggled ‘ON’, any subsequent labels for that instance automatically inherit the same dynamic attribute value as the frame where ‘Preserve chosen state’ was enabled.
If you have numerous attributes, use the search bar to quickly find specific attributes or attribute values.
Automated labeling includes several techniques to quickly create labels automatically.
Alternatively, run object tracking or interpolation on videos, image groups, and image sequences by:
Encord provides multiple tools to edit polygon and polyline labels after they have been created. See our tutorial video for an quick introduction on working with polygons in Encord.
Watch a tutorial for working with polygons
Move Vertex
To move vertices:
Click and drag a vertex to move it to a new location.
Add Vertex
To add a vertex:
You can add more vertices to a polygon after it has been created.
Remove Vertex
To remove a vertex:
You can remove the last vertex by pressing the Backspace key while still in draw mode.
Extension Tool
Use the polygon freehand tool to modify the edge of a polygon or polyline in a free-hand manner and create complex polygons.
To use the freehand tool:
Eraser Tool
Use the eraser tool to remove any part of a polygon in a free-hand manner.
To use the eraser tool:
Merge Polygons
Merging polygons combines two polygons into a single polygon
To merge Polygons:
The polygons have been merged.
Subtract Polygons
Subtracting polygons removes the area one polygon from that of another.
To subtract polygons:
The part of the second polygon that intersected the first is now removed from the first polygon.
Polygon Brush
The Polygon brush allows you to create polygons in the same manner that bitmasks can be created with the brush tool.
To create polygons with the brush:
Freehand Edge Touchup
Freehand edge touchup lets you refine a polygon by drawing a new edge between two vertices to adjust and close its shape.
To use freehand edge touchup:
Shape-specific interactions:
Shape | Interactions | Modalities |
---|---|---|
Bitmask | Ability to move (one or multiple) | Video / Image Group / Image Sequence / DICOM |
Ability to ‘select multiple’ with Shift + click | Video / Image Group / Image Sequence / DICOM | |
Draw using polygon ‘pen’ | Video / Image Group / Image Sequence / DICOM | |
Panoptic mode | Video / Image Group / Image Sequence / DICOM | |
Polygon | Video / Image Group / Image Sequence / DICOM | |
Add vertex with left-click | Video / Image Group / Image Sequence / DICOM | |
Remove vertex with right-click | Video / Image Group / Image Sequence / DICOM | |
Use the Brush tool by pressing F | Video / Image Group / Image Sequence / DICOM | |
Polyline | Video / Image Group / Image Sequence / DICOM | |
Add vertex with left-click | Video / Image Group / Image Sequence / DICOM | |
Remove vertex with right-click | Video / Image Group / Image Sequence / DICOM |
Right-click interactions:
Shape | Right-click Interactions | Modalities |
---|---|---|
Bounding Box | Track from this frame | Videos, image groups, and image sequences only |
Bounding Box | Interpolate | Videos, image groups, and image sequences only |
Bounding Box | Split object track | All |
Bounding Box | Change class | All |
Bounding Box | Copy identifier | All |
Bounding Box | Delete object | All |
Bounding Box | Copy URL | All |
Rotatable Bounding Box | Track from this frame | Videos, image groups, and image sequences only |
Rotatable Bounding Box | Interpolate | Videos, image groups, and image sequences only |
Rotatable Bounding Box | Split object track | All |
Rotatable Bounding Box | Change class | All |
Rotatable Bounding Box | Copy identifier | All |
Rotatable Bounding Box | Delete object | All |
Rotatable Bounding Box | Copy URL | All |
Polygon | Interpolate | All |
Polygon | Split object track | All |
Polygon | Change class | All |
Polygon | Copy identifier | All |
Polygon | Delete object | All |
Polygon | Copy URL | All |
Polygon | Track from this frame | Videos |
Polygon | Edit with brush | All |
Polygon | Freehand edge touchup | All |
Polygon | Merge with another polygon | All |
Polygon | Subtract another polygon | All |
Polyline | Track from this frame | Videos, image groups, and image sequences only |
Polyline | Interpolate | Videos, image groups, and image sequences only |
Polyline | Split object track | All |
Polyline | Change class | All |
Polyline | Copy identifier | All |
Polyline | Delete object | All |
Polyline | Copy URL | All |
Polyline | Freehand edge touchup | All |
Keypoint | Track from this frame | Videos, image groups, and image sequences only |
Keypoint | Interpolate | Videos, image groups, and image sequences only |
Keypoint | Split object track | All |
Keypoint | Change class | All |
Keypoint | Copy identifier | All |
Keypoint | Delete object | All |
Keypoint | Copy URL | All |
Bitmask | Interpolate | Videos, image groups, and image sequences only |
Bitmask | Change class | All |
Bitmask | Copy identifier | All |
Bitmask | Delete object | All |
Bitmask | Copy URL | All |
Bitmask | Select more bitmasks to combine | All |
Bitmask | Track from this frame | Videos |
Object Primitive | Track from this frame | Videos, image groups, and image sequences only |
Object Primitive | Interpolate | Videos, image groups, and image sequences only |
Object Primitive | Change class | All |
Object Primitive | Copy identifier | All |
Object Primitive | Delete object | All |
Object Primitive | Copy URL | All |
You can customize the default Label Editor layout by uploading a JSON file that defines your preferred arrangement. These JSON files can be added in the Project settings and are applied to all tasks in a Project.
Custom editor layouts depend on a file’s client_metadata
or their DICOM tags. This means that custom editor layouts only work for files that contain client_metadata
or DICOM tags. Watch the video tutorials below to learn how it fits together.
Introduction to custom editor layouts for DICOM - Video Tutorial
Introduction to custom editor layouts (Client metadata) - Video Tutorial
To upload an editor layout:
JSON file requirements:
The JSON file must follow the JSON schema defined here. The following fields are required:
topLevelGridFilter
In Workflow Projects, tasks with matching values in the field specified by the topLevelGridFilter
progress through the Workflow as a unit. When encord-LayoutGroup
is set as the topLevelGridFilter
, for example, tasks sharing the same encord-LayoutGroup
value are opened, submitted, reviewed, and reopened together, enabling seamless coordination across the Workflow. This behavior does not apply in Manual QA Projects.
The operator value must be =
. This means the value must be equal to.
Another example is displaying images from the same patient together in the Label Editor using the topLevelGridFilter
to group tasks based on a patient-id
.
grid
The grid
key defines the basic layout of the Label Editor. The grid
is a list of lists, where each value represents a window.
In the following example, the values 0, 1, 2, and 3 represent individual windows. The layout is defined as two separate lists [0, 1] and [2, 3], resulting in a 2x2 grid configuration:
This configuration translates to the following visual layout:
gridContent
The gridContent
key specifies what content to display in each window defined in the grid
. It includes the following elements:
Grid Values: Assigns content to each window based on its grid value.
Content Query: Defines the criteria for displaying content, which can be either dicom_tags_query
or client_metadata_query
.
Conditions: Specifies the conditions for displaying the content, which can be either and_conditions
or or_conditions
.
Operator value can only be =
. This means the value must be equal to.
In the following example, window 0
displays content based on a dicom_tags_query
with specific conditions.
dicom_tags_query
specifies that the content is determined by DICOM tags.and_conditions
key specifies that all conditions listed must be met for the content to be displayed.00185101
must equal CC
.00200062
must equal R
.This setup ensures that the content for window 0 is only displayed if both conditions are satisfied.
In the following example, window 1
displays content based on a client_metadata_query
with specific conditions.
client_metadata_query
specifies that the content is determined by client metadata.or_conditions
key specifies that any of the conditions listed must be met for the content to be displayed.group-id
must equal A
.group-id
must equal B
.This setup ensures that the content for window 1 is only displayed if the group-id
is either A or B.
Complete example
Add client metadata to files
Editor layouts are based on DICOM tags for DICOM files or on client_metadata
for all other use cases. We provide templates for common mammography layouts based on DICOM tags here.
This example uses client_metadata
. The following SDK script can be used to add client metadata to specific data units in a Dataset.
Create a JSON file
Create a JSON for specifying the editor layout that suits your needs.
In the JSON file below:
The grid
arrangement is configured to display two files side by side in the Label Editor.
The gridContent
section specifies that tasks in either position (0 or 1) can have client metadata values for encord-EditorGridPosition
set to either A
or B
.
The topLevelGridFilter
is defined as encord-LayoutGroup
, meaning tasks with matching encord-LayoutGroup
metadata values are displayed together in the Label Editor, ensuring they appear side by side when they share the same encord-LayoutGroup
values.
Create a JSON file
Upload the JSON file to your Project in Encord.
Create a JSON file
Open any task in the task queue of your Project.
The JSON file in this example results in the following Label Editor layout.
Open the editor menu by clicking the menu icon in the top left corner of the Label Editor.
The editor menu contains the following controls:
The Editor header at the top-right of the navigation bar contains buttons and menu items to perform labeling activities as well as manage general app settings and notifications.
Icon | Name | Description |
---|---|---|
![]() | Help | Links to the: quick start guide, documentation, learning hub, and Encord support team. |
![]() | Notifications | Shows the current status and history of actions taken in the application. This includes uploading datasets, training models, and exporting labels. |
![]() | Share | Copies the URL of the asset currently being labeled or reviewed. |
![]() | Save labels | Manually saves labels on the Encord server. |
![]() | Label Editor settings | Settings controlling your labeling and reviewing experience. |
![]() | Issues | Shows any issues and comments for the current task, such as any objects or classifications marked as Required, or comments added by other users. |
A toolbar with handy tools to:
Click the Rotate toolbar icon on the quick toolbar to bring up a pop-up that enables you to rotate the label editor by using the slider, as shown below.
The Pixelated scaling feature enhances the visibility of individual pixels in an image or frame by disabling the browser’s automatic edge smoothing. This increased pixel clarity aids annotators in creating precise, pixel-perfect annotations, thereby improving their annotation accuracy, especially when zooming into an image.
Pixelated Scaling turned on | Pixelated Scaling turned off |
---|---|
![]() | ![]() |
Enable Pixelated scaling by selecting the Adjust filters icon ()in the Quick toolbar and enabling the Pixelated scaling checkbox, as shown below.
Pixelated scaling is a browser-specific feature. The table below shows which types of files Pixelated scaling works for Firefox and Chrome.
File type | Chrome | Firefox |
---|---|---|
Image | ✅ | ✅ |
Image group | ✅ | ✅ |
Video | ❌ | ✅ |
Image sequence | ❌ | ✅ |
The Grayscale filter feature enhances the visibility across an image or frame by applying an RGB grayscale to the image/frame. The filter applies across the pixels of an image/frame using the value for Red, Green, or Blue as a base then converting to grayscale.
Grayscale Image/Frame
Grayscale DICOM/NIfTi
The video frame slider lets you quickly scroll through frames in a video asset. Below the frame slider are a range of controls to let you play, skip and navigate through media. Notice that the navigation controls are not displayed in single images files, as well as any media with only 1 frame.
Icon descriptions:
The Time and Frame rate information for each frame is displayed in the far lower right corner.
Add Issue Tags to a Project to flexibly categorize and group your issues.
Use the Issues tab of your Project Analytics to learn about issues in your Project.
A complete walkthrough of all things Issues and Comments related can be found here.
Issues can be added to annotation and review tasks to communicate with other users. All issues contain comment threads relating to the entire task or specific sections of the task. A small notification badge on the Issues icon displays the number of unresolved issues in a task.
Types of Issues:
Label rejection: Created when a reviewer rejects labels. The reviewer is prompted to provide a reason for the rejection in the form of a comment. If all instances in a frame or image are rejected, a separate rejection issue is created for each label. Labels with rejection issues display a red Issues icon next to the instance name, which turns green once the issue is resolved.
File: Apply to the entire file and can be created at any stage within the Label Editor.
Pinned: Apply to a specific part of the file or frame and can be created at any stage within the Label Editor. Red Pin icons indicate unresolved issues and remain red until they are marked as resolved.
Frame: Apply to a specific frame and can be created at any stage within the Label Editor. Frame issues can only be created when multiple frames are present; such as in a video, image sequence, or DICOM volume.
Frame range: Apply to a range of frames and can be created at any stage within the Label Editor. Frame range issues can only be created when multiple frames are present; such as in a video, image sequence, or DICOM volume. Frame range issues can only be created in Encord Active.
Task requirement: Are created for each object or classification marked Required in the Ontology. The issue is resolved when the required object or classification is applied. Task requirement issues are the only issues visible in a Consensus annotate stage.
To Create a new Issue:
Issues cannot be created in a Consensus annotate stage.
To Resolve an Issue:
Resolved issues remain visible but can be filtered out. All users can resolve issues.
To Delete an Issue Thread:
Deleted threads are NOT counted in the Issues analytics.
Users can only delete issues that they created. Label rejections cannot be deleted.
To Reopen an Issue Thread:
Use Cases:
The ‘Editor lock’ is a safety feature we implemented to prevent several people making changes to the same file - annotators labeling, or reviewers reviewing the same data unit.
A warning is triggered in two cases:
Our Label Editor fully supports Wacom tablets and pens. You get the best experience if you enable freehand drawing mode.
The following actions vary significantly between the legacy Label Editor V1 and the new Label Editor. To see how to perform these in the new Label Editor, see our documentation here.
Adding a polygon vertex - Label Editor V1
To add more vertices to a polygon after it has been created:
Removing a polygon or polyline vertex - Label Editor V1
To remove vertices from a polygon::
Freehand edge touchup - Label Editor V1
If you need to modify the edge of a polygon to account for complicated shapes or other requirements, the freehand edge touchup feature allows you edit the edge of the polygon in a free-hand manner, from one vertex to another.