Encord provides native support for Digital Imaging and Communications in Medicine (DICOM) browser rendering and data annotations. The editor also supports NIfTI (Neuroimaging Informatics Technology Initiative). With the DICOM editor, you can:
The following video provides a summary of DICOM Label Editor capabilities.
DICOM editor components:
If you have questions regarding a specific modality or datatype, please reach out to us at support@encord.com.
Before we get into the details of the label editor and making annotations, a quick note about scheduling annotation work. Annotation is done in the annotation phase of the task management system. In order to ensure annotation work is saved properly, it’s crucial that each labeling task is properly assigned to an annotator, annotators always access the task through the Queue tab in the Task Management System, and only the assigned annotator works on any given task at any given time.
There are a number of different ways to skip through slices in a DICOM file:
The number of slices skipped is determined by the specified Frameskip interval. Frameskip intervals can be set using the predefined values, or customized by entering a value in the New input and clicking the Plus icon.
You don’t have to wait for your file to finish loading before navigating to your desired frame! Simply drag the cursor to, or click the progress bar on, the location of the frame you would like to view while it is still loading, and the file will continue loading from the desired location, as seen in the following video. The light-blue color on the progress bar indicates how much of your file has been loaded.
Crosshair mode can be activated using the Crosshair icon in the floating bar.
Left-clicking in a view updates the other views automatically so that the displayed image planes match the selected location.
Seamlessly copy the URL of the slice currently being viewed by clicking the Share icon, shown below.
Share the URL to guide users to a specific slice or frame.
The Encord DICOM viewer renders DICOM files natively in the browser, enabling the display of the full range of the intensity values (i.e., 20,000+ pixel intensities).
Intensity values are projected directly on the image. The example below has the cursor pointing to Hounsfield Value 1591 o.
Toggle the DICOM windowing settings menu by clicking the sliders icon in the viewer pane. Adjust window width and level using the sliders or setting your required input values. Save custom levels as presets for quick access later.
Use the windowing adjustment functionality by clicking the Adjust windowing button in the toolbar or pressing alt + w on your keyboard. Next, press the left mouse button and drag the mouse to change the window width or level:
The VOI (Value Of Interest) LUT (Look-Up Table) Function, specified in the DICOM metadata Tag (0028,1056), defines how pixel values are mapped from Modality LUT output to Presentation LUT input, affecting image contrast and brightness.
The Encord DICOM viewer supports Linear and Sigmoid VOI LUT transfer functions.
For a detailed explanation of VOI LUT Transfer Functions, refer to the official DICOM documentation here.
Encord’s MPR display reconstructs 2D orthogonal images (coronal, sagittal, axial) for enhanced visualization, analysis, and annotation of anatomy. Annotations made in the one view automatically appear in all other views, and cross-reference lines are projected across all three views, assisting with precise anatomy annotation.
MPR views are toggled on or off in the DICOM editor settings.
By default, the primary view is shown in the main annotation window on the left of the Label Editor, and reconstructed images are shown in smaller windows on the right. Use the MPR dropdown to switch between different views in any window - the primary view is always clearly labeled with the word PRIMARY.
Labeling reconstructed views
Labels can be applied to primary and reconstructed views. When a label is created it automatically shows up in all other views. You can change the tile currently being annotated using the Annotate from this tile button, however we recommend creating labels in the main annotation window.
How to disable MPR
MPR views are automatically displayed when a 3D volume is detected. To turn this off, disable the Default to MPR layout toggle in the DICOM settings section of the Editor settings.
Medical scans can be explored in 3 dimensions, allowing for a more intuitive assessment of anatomical structures.
.dcm
file tags.To enable 3D volume rendering:
Use the drop-down menu in the viewport to seamlessly switch between any series in the Project.
Click a Series title to open the series in the Label Editor.
In the drop-down menu of the viewport you can add other series to be viewed at the same time. Annotations will be seen on all the series being viewed, as seen in the screenshot below.
Only series that are part of the same Dataset are available for selection. If you want to view multiple series in one session, ensure they are part of the same Dataset.
Clicking the sliders icon in the corresponding viewer pane and then changing the slab thickness (ST) to something greater than 0 enables MIP mode. MIP can be switched on/off for each individual view. Note that annotations are not shown in MIP mode. To disable MIP mode change the ST back to 0.
Hanging protocols allow you to select how many entities within a study are displayed at the same time. This allows you to, for example, view the same CT scan slice at different moments in time.
You can save presets for a given layout by typing a meaningful name after making a protocol selection, and clicking the + button. Click the tick icon next to a preset name to select the preset as your default layout.
You can specify any preset, including Auto layout, as the default layout for studies in the Label Editor.
Click the tick icon next to a preset in the hanging protocols menu, to select the preset as your default layout.
You can reset the Label Editor to the default layout by clicking the Reset layout to default icon.
Auto layouts are common layouts for viewing DICOM studies in the Label Editor. Auto layouts are set in the DICOM editor settings. This includes the MPR layout setting and the Mammography layout setting for mammography files.
Selecting the Auto layout option resets the Label Editor to your auto layout. You can specify the Auto layout option as your default layout.
The Encord DICOM metadata viewer renders DICOM files natively in the browser, allowing you to access all defined metadata fields and sequences associated with the DICOM file.
The metadata viewer is displayed by toggling the View metadata button above the series being annotated. The metadata viewer contains a search bar allowing you to search for specific types of metadata.
The metadata is organized into the following columns for easy interpretation:
Metadata sequences provide additional layers of information and can be identified by the presence of the + icon. Clicking the + icon expands the sequence, revealing nested fields and changing to a - icon. Clicking the - icon collapses the sequence.
The Tag column plays a vital role in identifying the type of metadata field:
The Data overlay toggle, located in the Additional controls, allows you to display or hide metadata overlay. When toggled on, metadata displays in the bottom left corner of the series being annotated.
Create a custom overlay to display specific metadata in the Label Editor by default.
To set a custom overlay:
Specifying custom overlays:
A JSON file is used to define which elements are displayed as overlays in the Label Editor. The following JSON schema includes all possible overlay elements you can configure.
You can seamlessly switch the series you’d like to annotate.
To select a tile for annotation
To move a tile to the one currently being annotated
The ‘Editor switch’ button ( ) enables you to select any series being displayed to be annotated without having to return to the task management system.
The Label Editor displays how many series are in part of the study currently being viewed. The View full study button allows you to view all series belonging to a study side-by-side.
You can toggle the View full study button. When you toggle that button it turns into the Hide full study button.
Click the Rotate toolbar icon at the top of the label editor to bring up a pop-up enabling you to rotate the label editor by using the slider, as shown below.
The distance measurement tool can be used to measure real-world distances between any two points on an image. It uses the ‘Pixel spacing’ parameter - contained in an entity’s metadata - to accurately convert pixels into real-world distances that are displayed in millimeters.
The Polyline Ontology displays the total length of the line in millimeters, rather than simply the distance between two points.
Enable the distance measurement functionality by clicking the button on the floating toolbar, or by pressing alt + m on your keyboard.
To delete existing measurements the distance measurement functionality has to be enabled. Erase measurements by selecting a measurement and pressing backspace or del on your keyboard.
Encord supports copying measured distances using the polyline annotation type, separate from using the measurement tool as detailed above.
Polygon and polyline angles can be displayed in the label editor to provide useful insights into your labels.
Objects are annotated using bounding boxes, polygons, polylines, keypoints or primitives. Slices are annotated using classifications. Instantiating objects or classifications in the DICOM editor generates a UUID that uniquely identifies that instance across a range of slices (i.e., in the spatial dimension).
The identifier is sometimes called a “track,” with numerical IDs supplementing UUIDs for objects for visual aid.
Object instances can be assigned with static and dynamic attributes (i.e., spatial classifications that may change through the volume) as defined in the specified ontology. Static classifications define the global properties of an object (e.g., the surgical tool has three forks) whereas dynamic attributes change during the volume (e.g., the brain has a tumour present from slices 0 to 100 but none from frame 101 to 150).
Supported annotation types & corresponding features are listed in the table below. You can read more about the listed automation features here.
Annotation type | Dynamic attribute | Automated tracking | Interpolation | Import predictions | Model training & inference |
---|---|---|---|---|---|
Bounding box | ✅ | ✅ | ✅ | ✅ | ✅ |
Rotatable bounding box | ✅ | ✅ | ✅ | ✅ | Coming soon |
Polygon | ✅ | Coming soon | ✅ | ✅ | ✅ |
Polyline | ✅ | ✅ | ✅ | ✅ | Coming Soon |
Keypoint | ✅ | ✅ | ✅ | ✅ | ✅ |
Bitmask | ✅ | N/A | ✅ | ✅ | N/A |
Object primitive | N/A | ✅ | N/A | ✅ | N/A |
Frame classification | N/A | N/A | N/A | ✅ | ✅ |
Creating a bounding box requires an Ontology with a bounding box annotation type. Instantiate a new bounding box instance by clicking on the specified class in the ‘Classes’ menu, or by using the specified hotkey (e.g., 1, 2, 3).
Instantiating existing bounding box instances (i.e., bounding boxes that should keep the same identifier in preceding or succeeding slices) can be done by clicking on the Highlight icon for the specified object or using the assigned hotkey (e.g., q, w, e).
Bounding boxes can be copy/pasted between slices using cmd (MacOS)/ctrl (Windows) + c to copy the object to the clipboard and cmd/ctrl + v to paste it on the desired slice. See this visual illustration.
Bounding box instances can be assigned with static and dynamic attributes should they be defined in your Ontology.
Highlight an object by clicking on it in the DICOM editor canvas or by clicking on the Plus icon or start/end slice in the range overview for the specified bounding box.
Assign classifications by clicking on the relevant buttons or using the set hotkeys (for example q, u).
Interpolation is run via the Automated labeling drawer. Click the Automated labeling button to open the drawer. Select Interpolation from the drop-down menu, select the desired objects to interpolate, set the interpolation range (i.e., the range of slices) and click the Run interpolation button. You need a minimum two labels of an instance to run interpolation.
Learn how to use interpolation for DICOM using the video tutorial below.
Video Tutorial - Interpolation (DICOM)
You can read more about interpolation here.
Creating a polygon requires an Ontology with a polygon annotation type. Enable or disable free-hand drawing mode by pressing d on your keyboard. Polygon coarseness (for polygons drawn free-hand) is set in the Drawing settings section of the Settings ().
Instantiate a new polygon instance by clicking on the specified class in the ‘Classes’ menu or using the specified hotkey (e.g., 1, 2, 3).
A polygon can be closed by double-clicking anywhere on the canvas. Doing so will “snap” creating an edge between the last vertex to the first vertex drawn.
You can create polygons using SAM 2, by placing vertices, or by using the polygon brush (F).
Instantiate existing polygon instances (i.e., polygons that should keep the same identifier in preceding or succeeding slices) by clicking on the Highlight icon for the specified object or using the assigned hotkey (e.g., q, w, e). See the keypoint documentation for visual instructions on how to instantiate existing polygons.
Polygons can be copy/pasted between slices using cmd (MacOS)/ctrl (Windows) + c to copy the object to the clipboard and cmd/ctrl + v to paste it on the desired slice.
Polygon instances can be assigned with static and dynamic attributes should they be defined in your Ontology:
Highlight an object by clicking on it in the editor canvas or by clicking on the Plus icon or start/end slice in the range overview for the specified polygon.
Assign classifications by clicking on the relevant buttons or using the set hotkeys (for example q, u).
Run polygon interpolation following the instructions for bounding box, which you can find here. Unlike other linear interpolation methods, Encord’s polygon interpolation algorithm does not require a matching number of vertices between polygon objects in set keyframes. Our algorithm also allows you to draw polygons in arbitrary directions (e.g., clockwise, counterclockwise, and otherwise).
You can read more about interpolation here.
You can edit existing polygons with the brush tool. This allows you to create complex shapes in a much faster way than going vertex by vertex. To use the brush tool, select an existing polygon and then click on the brush tool button. This will show the brush tool settings panel where you can choose the brush or eraser and change its size. Once you are done editing, click the “x” to go back to the label editor.
Creating a polyline requires an Ontology with a polyline annotation type. Enable or disable free-hand drawing mode by pressing d on your keyboard. Polyline coarseness (for polylines drawn free-hand) is set in the Drawing settings section of the Settings ().
Instantiate a new polyline instance by clicking on the specified class in the ‘Classes’ menu or using the specified hotkey (e.g., 1, 2, 3).
Polylines can be copy/pasted between slices using cmd (MacOS)/ctrl (Windows) + c to copy the object to the clipboard and cmd/ctrl + v to paste it on the desired slice.
Complete a polyline by double-clicking anywhere on the canvas.
Instantiate existing polyline instances (i.e., polylines that should keep the same identifier in preceding or succeeding slices) by clicking on the Highlight icon for the specified object or using the assigned hotkey (e.g., q, w, e).
Polyline instances can be assigned with static and dynamic attributes should they be defined in your Ontology:
Highlight an object by clicking on it in the editor canvas or by clicking on the Plus icon or start/end frame in the range overview for the specified polyline.
Assign a classification by clicking on the relevant buttons or using the set hotkeys (for example q, u).
Creating a keypoint requires an Ontology with a keypoint annotation type. Instantiate a new keypoint instance by clicking on the specified class in the ‘Classes’ menu or using the specified hotkey (e.g., 1, 2, 3).
Instantiate existing keypoint instances (i.e., keypoints that should keep the same identifier in preceding or succeeding slices) by clicking on the Highlight icon for the specified object or using the assigned hotkey (e.g., q, w, e).
Keypoints can be copy/pasted between slices using cmd (MacOS)/ctrl (Windows) + c to copy the object to the clipboard and cmd/ctrl + v to paste it on the desired slice.
Keypoints instances can be assigned with static and dynamic attributes should they be defined in your Ontology:
Run keypoint interpolation following the instructions for bounding box, which you can find here.
Creating a primitive (f.k.a. skeleton templates) requires an Ontology with a primitive annotation type. Use primitives to templatize shapes (example, 3D cuboids, pose estimation skeletons, rotated bounding boxes) commonly used by your annotation team.
Create a primitive ontology for data labeling
To instantiate a new primitive instance, click on the specified class in the ‘Classes’ menu or use the specified hotkey (e.g., 1, 2, 3).
Instantiate existing primitive instances (i.e., primitives that should keep the same identifier in preceding or succeeding slices) by clicking on the Highlight icon for the specified object or using the assigned hotkey (e.g., q, w, e).
Primitives can be copy/pasted between slices using cmd (MacOS)/ctrl (Windows) + c to copy the object to the clipboard and cmd/ctrl + v to paste it on the desired slice.
Primitives allow you to define properties of edges defined in your template as visible, occluded, or invisible. Toggle the edge property settings for a primitive by highlighting the primitive and clicking the Show controls button.
Object primitive instances can be assigned with static and dynamic attributes should they be defined in your Ontology:
Highlight an object by clicking on it in the editor canvas or by clicking on the Plus icon or start/end frame in the range overview for the specified primitive.
Assign classifications by clicking on the relevant buttons or using the set hotkeys (for example q, u).
Run primitive interpolation following the instructions for bounding box, which you can find here.
Bitmasks allow you to create labels using a brush tool to select parts of an image. This can be useful when creating labels for vessel outlines or labeling topologically separate regions belonging to the same frame classification.
When creating a bitmask, the process continues until you press the ENTER or ESC key. This allows you to easily create complex bitmask labels without interruption.
The brush tool is selected by clicking the brush icon or by pressing f on your keyboard while the popup is open.
When using the brush tool, the 2D brush tool is selected by default. The 2D brush tool on creates bitmasks on the slice they are drawn on. In comparison, the 3D cylinder brush is applied to multiple slices, giving it depth in addition to width.
You can adjust the brush tool by:
Click Apply label or press Enter to apply the Bitmask.
The 3D cylinder brush lets you apply a Bitmask in 3 dimensions, which means it is applied to multiple slices in a volume. The following two settings are available:
Adjust the slider to set your desired Size and Height.
After the Bitmask is applied, click Apply label or press Enter to apply the Bitmask.
We recommend enabling 3D view in one of the windows of the Label Editor when using the 3D cylinder brush. This allows you to see the Bitmask label in 3D.
Panoptic settings allow you to determine how different bitmasks interact with one another.
The Thresholding tool enables you to set a threshold that determines the parts of the image or frame that is labeled by the Bitmask. Consequently, only the parts of the image falling within th predefined range are labeled upon selection with the Thresholding tool, ensuring precise and targeted labeling.
When Thresholding is enabled, a mask covering the parts of the image that fall above the set threshold appears on the slice. This allows you to preview which parts of the image are labeled when a Bitmask label is applied. The mask color can be changed by clicking the circle icon.
The eyedropper tool enables you to pick a color or intensity value directly from the image.
Three different kinds of threshold can be selected using a dropdown:
When your label is ready, click Apply label, or press Enter.
Polygon mode lets you draw bitmasks the same way you draw polygons.
To draw a bitmask using Polygon mode :
The Eraser tool allows you to erase parts, or the entirety of your Bitmask selection, if the Apply label button has not been clicked yet.
To select the threshold brush, click the eraser icon, or hit h on your keyboard while the popup is open.
Combining bitmasks on an image or frame allows you to label objects that are split/separated in the image/video frame.
When creating a bitmask, the process continues until you press the ENTER or ESC key. This allows you to easily create complex bitmask labels without interruption.
To combine two or more bitmasks on an image / video frame:
Hold SHIFT.
Click the bitmasks you want to combine in the Label Editor workspace.
Right-click (on Mac press Cmd). A menu appears.
Select Combine bitmasks into. The bitmasks are now a single bitmask.
It is possible to prevent a Bitmask label from being overlapped by subsequent Bitmasks after the label is created. Use the toggle in the Labels section of the Label Editor to set the overlap behavior.
Choose between the following settings:
To update a Bitmask label:
Bitmask labels can be moved to another location after being created.
Encord’s interpolation feature uses a proprietary linear interpolation algorithm that runs without using a representational model or matching pixel information in neighboring frames.
Our interpolation algorithm has been built with pragmatic usage in mind. For example, unlike other linear interpolation methods, Encord’s interpolation algorithm does not require a matching number of vertices between objects in set keyframes. Our algorithm also allows you to draw object vertices in arbitrary directions (e.g., clockwise, counterclockwise, and otherwise).
You can read more about interpolation here.
Encord’s Python SDK & APIs allow you to import model predictions programmatically. Importing model predictions helps to pre-annotate your data to save annotation costs.
The Label Editor has several DICOM specific settings.
Encord provides native support for Digital Imaging and Communications in Medicine (DICOM) browser rendering and data annotations. The editor also supports NIfTI (Neuroimaging Informatics Technology Initiative). With the DICOM editor, you can:
The following video provides a summary of DICOM Label Editor capabilities.
DICOM editor components:
If you have questions regarding a specific modality or datatype, please reach out to us at support@encord.com.
Before we get into the details of the label editor and making annotations, a quick note about scheduling annotation work. Annotation is done in the annotation phase of the task management system. In order to ensure annotation work is saved properly, it’s crucial that each labeling task is properly assigned to an annotator, annotators always access the task through the Queue tab in the Task Management System, and only the assigned annotator works on any given task at any given time.
There are a number of different ways to skip through slices in a DICOM file:
The number of slices skipped is determined by the specified Frameskip interval. Frameskip intervals can be set using the predefined values, or customized by entering a value in the New input and clicking the Plus icon.
You don’t have to wait for your file to finish loading before navigating to your desired frame! Simply drag the cursor to, or click the progress bar on, the location of the frame you would like to view while it is still loading, and the file will continue loading from the desired location, as seen in the following video. The light-blue color on the progress bar indicates how much of your file has been loaded.
Crosshair mode can be activated using the Crosshair icon in the floating bar.
Left-clicking in a view updates the other views automatically so that the displayed image planes match the selected location.
Seamlessly copy the URL of the slice currently being viewed by clicking the Share icon, shown below.
Share the URL to guide users to a specific slice or frame.
The Encord DICOM viewer renders DICOM files natively in the browser, enabling the display of the full range of the intensity values (i.e., 20,000+ pixel intensities).
Intensity values are projected directly on the image. The example below has the cursor pointing to Hounsfield Value 1591 o.
Toggle the DICOM windowing settings menu by clicking the sliders icon in the viewer pane. Adjust window width and level using the sliders or setting your required input values. Save custom levels as presets for quick access later.
Use the windowing adjustment functionality by clicking the Adjust windowing button in the toolbar or pressing alt + w on your keyboard. Next, press the left mouse button and drag the mouse to change the window width or level:
The VOI (Value Of Interest) LUT (Look-Up Table) Function, specified in the DICOM metadata Tag (0028,1056), defines how pixel values are mapped from Modality LUT output to Presentation LUT input, affecting image contrast and brightness.
The Encord DICOM viewer supports Linear and Sigmoid VOI LUT transfer functions.
For a detailed explanation of VOI LUT Transfer Functions, refer to the official DICOM documentation here.
Encord’s MPR display reconstructs 2D orthogonal images (coronal, sagittal, axial) for enhanced visualization, analysis, and annotation of anatomy. Annotations made in the one view automatically appear in all other views, and cross-reference lines are projected across all three views, assisting with precise anatomy annotation.
MPR views are toggled on or off in the DICOM editor settings.
By default, the primary view is shown in the main annotation window on the left of the Label Editor, and reconstructed images are shown in smaller windows on the right. Use the MPR dropdown to switch between different views in any window - the primary view is always clearly labeled with the word PRIMARY.
Labeling reconstructed views
Labels can be applied to primary and reconstructed views. When a label is created it automatically shows up in all other views. You can change the tile currently being annotated using the Annotate from this tile button, however we recommend creating labels in the main annotation window.
How to disable MPR
MPR views are automatically displayed when a 3D volume is detected. To turn this off, disable the Default to MPR layout toggle in the DICOM settings section of the Editor settings.
Medical scans can be explored in 3 dimensions, allowing for a more intuitive assessment of anatomical structures.
.dcm
file tags.To enable 3D volume rendering:
Use the drop-down menu in the viewport to seamlessly switch between any series in the Project.
Click a Series title to open the series in the Label Editor.
In the drop-down menu of the viewport you can add other series to be viewed at the same time. Annotations will be seen on all the series being viewed, as seen in the screenshot below.
Only series that are part of the same Dataset are available for selection. If you want to view multiple series in one session, ensure they are part of the same Dataset.
Clicking the sliders icon in the corresponding viewer pane and then changing the slab thickness (ST) to something greater than 0 enables MIP mode. MIP can be switched on/off for each individual view. Note that annotations are not shown in MIP mode. To disable MIP mode change the ST back to 0.
Hanging protocols allow you to select how many entities within a study are displayed at the same time. This allows you to, for example, view the same CT scan slice at different moments in time.
You can save presets for a given layout by typing a meaningful name after making a protocol selection, and clicking the + button. Click the tick icon next to a preset name to select the preset as your default layout.
You can specify any preset, including Auto layout, as the default layout for studies in the Label Editor.
Click the tick icon next to a preset in the hanging protocols menu, to select the preset as your default layout.
You can reset the Label Editor to the default layout by clicking the Reset layout to default icon.
Auto layouts are common layouts for viewing DICOM studies in the Label Editor. Auto layouts are set in the DICOM editor settings. This includes the MPR layout setting and the Mammography layout setting for mammography files.
Selecting the Auto layout option resets the Label Editor to your auto layout. You can specify the Auto layout option as your default layout.
The Encord DICOM metadata viewer renders DICOM files natively in the browser, allowing you to access all defined metadata fields and sequences associated with the DICOM file.
The metadata viewer is displayed by toggling the View metadata button above the series being annotated. The metadata viewer contains a search bar allowing you to search for specific types of metadata.
The metadata is organized into the following columns for easy interpretation:
Metadata sequences provide additional layers of information and can be identified by the presence of the + icon. Clicking the + icon expands the sequence, revealing nested fields and changing to a - icon. Clicking the - icon collapses the sequence.
The Tag column plays a vital role in identifying the type of metadata field:
The Data overlay toggle, located in the Additional controls, allows you to display or hide metadata overlay. When toggled on, metadata displays in the bottom left corner of the series being annotated.
Create a custom overlay to display specific metadata in the Label Editor by default.
To set a custom overlay:
Specifying custom overlays:
A JSON file is used to define which elements are displayed as overlays in the Label Editor. The following JSON schema includes all possible overlay elements you can configure.
You can seamlessly switch the series you’d like to annotate.
To select a tile for annotation
To move a tile to the one currently being annotated
The ‘Editor switch’ button ( ) enables you to select any series being displayed to be annotated without having to return to the task management system.
The Label Editor displays how many series are in part of the study currently being viewed. The View full study button allows you to view all series belonging to a study side-by-side.
You can toggle the View full study button. When you toggle that button it turns into the Hide full study button.
Click the Rotate toolbar icon at the top of the label editor to bring up a pop-up enabling you to rotate the label editor by using the slider, as shown below.
The distance measurement tool can be used to measure real-world distances between any two points on an image. It uses the ‘Pixel spacing’ parameter - contained in an entity’s metadata - to accurately convert pixels into real-world distances that are displayed in millimeters.
The Polyline Ontology displays the total length of the line in millimeters, rather than simply the distance between two points.
Enable the distance measurement functionality by clicking the button on the floating toolbar, or by pressing alt + m on your keyboard.
To delete existing measurements the distance measurement functionality has to be enabled. Erase measurements by selecting a measurement and pressing backspace or del on your keyboard.
Encord supports copying measured distances using the polyline annotation type, separate from using the measurement tool as detailed above.
Polygon and polyline angles can be displayed in the label editor to provide useful insights into your labels.
Objects are annotated using bounding boxes, polygons, polylines, keypoints or primitives. Slices are annotated using classifications. Instantiating objects or classifications in the DICOM editor generates a UUID that uniquely identifies that instance across a range of slices (i.e., in the spatial dimension).
The identifier is sometimes called a “track,” with numerical IDs supplementing UUIDs for objects for visual aid.
Object instances can be assigned with static and dynamic attributes (i.e., spatial classifications that may change through the volume) as defined in the specified ontology. Static classifications define the global properties of an object (e.g., the surgical tool has three forks) whereas dynamic attributes change during the volume (e.g., the brain has a tumour present from slices 0 to 100 but none from frame 101 to 150).
Supported annotation types & corresponding features are listed in the table below. You can read more about the listed automation features here.
Annotation type | Dynamic attribute | Automated tracking | Interpolation | Import predictions | Model training & inference |
---|---|---|---|---|---|
Bounding box | ✅ | ✅ | ✅ | ✅ | ✅ |
Rotatable bounding box | ✅ | ✅ | ✅ | ✅ | Coming soon |
Polygon | ✅ | Coming soon | ✅ | ✅ | ✅ |
Polyline | ✅ | ✅ | ✅ | ✅ | Coming Soon |
Keypoint | ✅ | ✅ | ✅ | ✅ | ✅ |
Bitmask | ✅ | N/A | ✅ | ✅ | N/A |
Object primitive | N/A | ✅ | N/A | ✅ | N/A |
Frame classification | N/A | N/A | N/A | ✅ | ✅ |
Creating a bounding box requires an Ontology with a bounding box annotation type. Instantiate a new bounding box instance by clicking on the specified class in the ‘Classes’ menu, or by using the specified hotkey (e.g., 1, 2, 3).
Instantiating existing bounding box instances (i.e., bounding boxes that should keep the same identifier in preceding or succeeding slices) can be done by clicking on the Highlight icon for the specified object or using the assigned hotkey (e.g., q, w, e).
Bounding boxes can be copy/pasted between slices using cmd (MacOS)/ctrl (Windows) + c to copy the object to the clipboard and cmd/ctrl + v to paste it on the desired slice. See this visual illustration.
Bounding box instances can be assigned with static and dynamic attributes should they be defined in your Ontology.
Highlight an object by clicking on it in the DICOM editor canvas or by clicking on the Plus icon or start/end slice in the range overview for the specified bounding box.
Assign classifications by clicking on the relevant buttons or using the set hotkeys (for example q, u).
Interpolation is run via the Automated labeling drawer. Click the Automated labeling button to open the drawer. Select Interpolation from the drop-down menu, select the desired objects to interpolate, set the interpolation range (i.e., the range of slices) and click the Run interpolation button. You need a minimum two labels of an instance to run interpolation.
Learn how to use interpolation for DICOM using the video tutorial below.
Video Tutorial - Interpolation (DICOM)
You can read more about interpolation here.
Creating a polygon requires an Ontology with a polygon annotation type. Enable or disable free-hand drawing mode by pressing d on your keyboard. Polygon coarseness (for polygons drawn free-hand) is set in the Drawing settings section of the Settings ().
Instantiate a new polygon instance by clicking on the specified class in the ‘Classes’ menu or using the specified hotkey (e.g., 1, 2, 3).
A polygon can be closed by double-clicking anywhere on the canvas. Doing so will “snap” creating an edge between the last vertex to the first vertex drawn.
You can create polygons using SAM 2, by placing vertices, or by using the polygon brush (F).
Instantiate existing polygon instances (i.e., polygons that should keep the same identifier in preceding or succeeding slices) by clicking on the Highlight icon for the specified object or using the assigned hotkey (e.g., q, w, e). See the keypoint documentation for visual instructions on how to instantiate existing polygons.
Polygons can be copy/pasted between slices using cmd (MacOS)/ctrl (Windows) + c to copy the object to the clipboard and cmd/ctrl + v to paste it on the desired slice.
Polygon instances can be assigned with static and dynamic attributes should they be defined in your Ontology:
Highlight an object by clicking on it in the editor canvas or by clicking on the Plus icon or start/end slice in the range overview for the specified polygon.
Assign classifications by clicking on the relevant buttons or using the set hotkeys (for example q, u).
Run polygon interpolation following the instructions for bounding box, which you can find here. Unlike other linear interpolation methods, Encord’s polygon interpolation algorithm does not require a matching number of vertices between polygon objects in set keyframes. Our algorithm also allows you to draw polygons in arbitrary directions (e.g., clockwise, counterclockwise, and otherwise).
You can read more about interpolation here.
You can edit existing polygons with the brush tool. This allows you to create complex shapes in a much faster way than going vertex by vertex. To use the brush tool, select an existing polygon and then click on the brush tool button. This will show the brush tool settings panel where you can choose the brush or eraser and change its size. Once you are done editing, click the “x” to go back to the label editor.
Creating a polyline requires an Ontology with a polyline annotation type. Enable or disable free-hand drawing mode by pressing d on your keyboard. Polyline coarseness (for polylines drawn free-hand) is set in the Drawing settings section of the Settings ().
Instantiate a new polyline instance by clicking on the specified class in the ‘Classes’ menu or using the specified hotkey (e.g., 1, 2, 3).
Polylines can be copy/pasted between slices using cmd (MacOS)/ctrl (Windows) + c to copy the object to the clipboard and cmd/ctrl + v to paste it on the desired slice.
Complete a polyline by double-clicking anywhere on the canvas.
Instantiate existing polyline instances (i.e., polylines that should keep the same identifier in preceding or succeeding slices) by clicking on the Highlight icon for the specified object or using the assigned hotkey (e.g., q, w, e).
Polyline instances can be assigned with static and dynamic attributes should they be defined in your Ontology:
Highlight an object by clicking on it in the editor canvas or by clicking on the Plus icon or start/end frame in the range overview for the specified polyline.
Assign a classification by clicking on the relevant buttons or using the set hotkeys (for example q, u).
Creating a keypoint requires an Ontology with a keypoint annotation type. Instantiate a new keypoint instance by clicking on the specified class in the ‘Classes’ menu or using the specified hotkey (e.g., 1, 2, 3).
Instantiate existing keypoint instances (i.e., keypoints that should keep the same identifier in preceding or succeeding slices) by clicking on the Highlight icon for the specified object or using the assigned hotkey (e.g., q, w, e).
Keypoints can be copy/pasted between slices using cmd (MacOS)/ctrl (Windows) + c to copy the object to the clipboard and cmd/ctrl + v to paste it on the desired slice.
Keypoints instances can be assigned with static and dynamic attributes should they be defined in your Ontology:
Run keypoint interpolation following the instructions for bounding box, which you can find here.
Creating a primitive (f.k.a. skeleton templates) requires an Ontology with a primitive annotation type. Use primitives to templatize shapes (example, 3D cuboids, pose estimation skeletons, rotated bounding boxes) commonly used by your annotation team.
Create a primitive ontology for data labeling
To instantiate a new primitive instance, click on the specified class in the ‘Classes’ menu or use the specified hotkey (e.g., 1, 2, 3).
Instantiate existing primitive instances (i.e., primitives that should keep the same identifier in preceding or succeeding slices) by clicking on the Highlight icon for the specified object or using the assigned hotkey (e.g., q, w, e).
Primitives can be copy/pasted between slices using cmd (MacOS)/ctrl (Windows) + c to copy the object to the clipboard and cmd/ctrl + v to paste it on the desired slice.
Primitives allow you to define properties of edges defined in your template as visible, occluded, or invisible. Toggle the edge property settings for a primitive by highlighting the primitive and clicking the Show controls button.
Object primitive instances can be assigned with static and dynamic attributes should they be defined in your Ontology:
Highlight an object by clicking on it in the editor canvas or by clicking on the Plus icon or start/end frame in the range overview for the specified primitive.
Assign classifications by clicking on the relevant buttons or using the set hotkeys (for example q, u).
Run primitive interpolation following the instructions for bounding box, which you can find here.
Bitmasks allow you to create labels using a brush tool to select parts of an image. This can be useful when creating labels for vessel outlines or labeling topologically separate regions belonging to the same frame classification.
When creating a bitmask, the process continues until you press the ENTER or ESC key. This allows you to easily create complex bitmask labels without interruption.
The brush tool is selected by clicking the brush icon or by pressing f on your keyboard while the popup is open.
When using the brush tool, the 2D brush tool is selected by default. The 2D brush tool on creates bitmasks on the slice they are drawn on. In comparison, the 3D cylinder brush is applied to multiple slices, giving it depth in addition to width.
You can adjust the brush tool by:
Click Apply label or press Enter to apply the Bitmask.
The 3D cylinder brush lets you apply a Bitmask in 3 dimensions, which means it is applied to multiple slices in a volume. The following two settings are available:
Adjust the slider to set your desired Size and Height.
After the Bitmask is applied, click Apply label or press Enter to apply the Bitmask.
We recommend enabling 3D view in one of the windows of the Label Editor when using the 3D cylinder brush. This allows you to see the Bitmask label in 3D.
Panoptic settings allow you to determine how different bitmasks interact with one another.
The Thresholding tool enables you to set a threshold that determines the parts of the image or frame that is labeled by the Bitmask. Consequently, only the parts of the image falling within th predefined range are labeled upon selection with the Thresholding tool, ensuring precise and targeted labeling.
When Thresholding is enabled, a mask covering the parts of the image that fall above the set threshold appears on the slice. This allows you to preview which parts of the image are labeled when a Bitmask label is applied. The mask color can be changed by clicking the circle icon.
The eyedropper tool enables you to pick a color or intensity value directly from the image.
Three different kinds of threshold can be selected using a dropdown:
When your label is ready, click Apply label, or press Enter.
Polygon mode lets you draw bitmasks the same way you draw polygons.
To draw a bitmask using Polygon mode :
The Eraser tool allows you to erase parts, or the entirety of your Bitmask selection, if the Apply label button has not been clicked yet.
To select the threshold brush, click the eraser icon, or hit h on your keyboard while the popup is open.
Combining bitmasks on an image or frame allows you to label objects that are split/separated in the image/video frame.
When creating a bitmask, the process continues until you press the ENTER or ESC key. This allows you to easily create complex bitmask labels without interruption.
To combine two or more bitmasks on an image / video frame:
Hold SHIFT.
Click the bitmasks you want to combine in the Label Editor workspace.
Right-click (on Mac press Cmd). A menu appears.
Select Combine bitmasks into. The bitmasks are now a single bitmask.
It is possible to prevent a Bitmask label from being overlapped by subsequent Bitmasks after the label is created. Use the toggle in the Labels section of the Label Editor to set the overlap behavior.
Choose between the following settings:
To update a Bitmask label:
Bitmask labels can be moved to another location after being created.
Encord’s interpolation feature uses a proprietary linear interpolation algorithm that runs without using a representational model or matching pixel information in neighboring frames.
Our interpolation algorithm has been built with pragmatic usage in mind. For example, unlike other linear interpolation methods, Encord’s interpolation algorithm does not require a matching number of vertices between objects in set keyframes. Our algorithm also allows you to draw object vertices in arbitrary directions (e.g., clockwise, counterclockwise, and otherwise).
You can read more about interpolation here.
Encord’s Python SDK & APIs allow you to import model predictions programmatically. Importing model predictions helps to pre-annotate your data to save annotation costs.
The Label Editor has several DICOM specific settings.