Auto segmentation tracking (BETA)
Note
The auto-segmentation tracking feature is currently available in Beta. While in Beta, the feature might not be as stable as our fully released functionalities, and we encourage users to provide feedback to help us improve its performance and reliability.
Auto-segmentation tracking follows instance labels through a series of frames in a video, allowing you to create near pixel perfect annotations with little effort. Auto-segmentation tracking is designed to maintain continuity in object tracking, even if the objects momentarily vanish from view for several frames.
Since tracking algorithms propagate labels forward through frames, auto-segmentation tracking works if labels are made on lower number frame (for example, near the start of the video).
Ontologies
The following table shows the Ontology shapes for which auto-segmentation can be used. Shapes not supported by the auto-segmentation tracking use the standard object tracking algorithm instead.
Note
Auto-segmentation tracking is only available for videos.
Ontology shape | Auto-segmentation tracking |
---|---|
Bounding box | ✅ |
Rotatable bounding box | ✅ |
Polygon | ✅ |
Polyline | ❌ |
Primitive | ❌ |
Keypoint | ❌ |
Bitmask | ✅ |
Modalities
The following table shows the modalities that support auto-segmentation tracking.
Ontology shape | Auto-segmentation tracking |
---|---|
Images | ❌ |
Videos | ✅ |
Image Groups | ❌ |
Image Sequences | ✅ |
DICOM | ✅ |
Using auto-segmentation tracking
Note
Auto-segmentation tracking is computationally intensive and performance might be slower than other labeling actions.
Bounding box
-
Create at least one bounding box instance label on the frame you want to start tracking the instance from.
-
Right-click the shape.
-
Click Track object and select the Advanced (slow) option to run auto-segmentation tracking for 30 consecutive frames.
Tip
Use the keyboard shortcut Shift + T to run auto-segmentation tracking for 30 consecutive frames.
![](https://storage.googleapis.com/docs-media.encord.com/static/img/auto-seg-bb.png)
Rotatable bounding box
-
Create at least one rotatable bounding box instance label on the frame you want to start tracking the instance from.
-
Right-click the shape.
-
Click Track object and select the Advanced (slow) option to run auto-segmentation tracking for 30 consecutive frames.
Tip
Use the keyboard shortcut Shift + T to run auto-segmentation tracking for 30 consecutive frames.
![](https://storage.googleapis.com/docs-media.encord.com/static/img/auto-seg-rbb.png)
Bitmask
-
Create at least one bitmask instance label on the frame you want to start tracking the instance from.
-
Right-click the shape.
-
Click Track object to run auto-segmentation tracking for 30 consecutive frames.
Tip
Use the keyboard shortcut Shift + T to run auto-segmentation tracking for 30 consecutive frames.
![](https://storage.googleapis.com/docs-media.encord.com/static/img/auto-seg-bitmask.png)
Polygon
-
Create at least one bitmask instance label on the frame you want to start tracking the instance from.
-
Right-click the shape.
-
Click Track object to run auto-segmentation tracking for 30 consecutive frames.
Tip
Use the keyboard shortcut Shift + T to run auto-segmentation tracking for 30 consecutive frames.
![](https://storage.googleapis.com/docs-media.encord.com/static/img/auto-seg-polygon.png)
Tracking multiple objects
To track multiple objects of any shape:
-
Click one of the objects you want to track.
-
Hold Shift on your keyboard and select all the other objects you want to track.
-
Right-click on one of the selected objects and click Track X objects, where
X
is the number of objects selected.
Settings
The Object tracking section of the editor settings allows you to adjust the following.
Change tracking range
The range auto-segmentation tracking runs can be adjusted in the Object tracking section of the editor settings.
The default tracking range is set to 30 frames. This range includes the frame that auto-segmentation tracking starts on.
CRITICAL INFORMATION
Increasing the tracking range to more than 500 frames can slow down performance considerably. We recommend running auto-segmentation tracking in intervals of 500 frames at a time while this feature is in the beta phase. This feature's performance is continuously being improved, and these limits will be raised in the near future.
Change tracking algorithm
You can change the tracking algorithm in the Object tracking section of the editor settings.
- Select Advanced to run auto-segmentation tracking.
- Select Standard to run standard object tracking.
CRITICAL INFORMATION
The choice of setting determines which algorithm is run when using the Shift + T keyboard shortcut.
Updated 9 days ago