Skip to main content
Annotators have some choices when it comes to automatically labeling and tracking objects:
  • SAM 3: Automatically detect and track one or more objects.
  • SAM 2: Automatically detect objects and track one or more objects.
  • Interpolation: Tracking objects between a range of frames.
  • Agents: Automatically detect and track objects.

​
SAM 3

This feature is currently in Early Access. Its user experience is still under development and may change as improvements are introduced. Contact support to gain access to SAM 3.
SAM 3 allows you to label and track one or more objects in a data unit. You can also use prompts to SAM 3 to help with labeling.

​
Label one object

Labeling one object is the same for images and videos.
  1. Go to Annotate > Projects.
  2. Click a Project from the Project list.
  3. OPTIONAL: Filter the data units as required from the Queue.
  4. Click Start task, Initiate, or use the keyboard shortcut Command + Enter in the Queue. The Label Editor appears.
  5. Click the Toggle auto-annotation button for the object class from the Classes list.
    Use the keyboard shortcut for the object class then use the keyboard shortcut SHIFT + A.
    A dialog appears for SAM 3.
  6. Select Single object, if it is not selected.
  7. Click on an object in your image or video.
Use the following when needed:
  • Left click to INCLUDE an area.
  • Right click to EXCLUDE an area.
  1. Click Save.
Use the keyboard shortcut ENTER or middle click to save.
  1. Continue labeling as required.
  2. Click Submit to submit the task.

​
Track one object

You can track objects in videos or image sequences.
  1. Go to Annotate > Projects.
  2. Click a Project from the Project list.
  3. OPTIONAL: Filter the data units as required from the Queue.
  4. Click Start task, Initiate, or use the keyboard shortcut Command + Enter in the Queue. The Label Editor appears.
  5. Click the Toggle auto-annotation button for the object class from the Classes list. A dialog appears for SAM 3.
Use the keyboard shortcut for the object class then use the keyboard shortcut SHIFT + A.
  1. Select Single object, if it is not selected.
  2. Click on an object in your image or video.
Use the following when needed:
  • Left click to INCLUDE an area.
  • Right click to EXCLUDE an area.
  1. Specify the number of frames to track the object across.
    You can specify a global setting for number of frames to track in Editor Settings > Label Tracking > Number of frames to predict.
  2. Click Track backwards or Track forwards.
  3. Click Save.
Use the keyboard shortcut ENTER or middle click to save.
  1. Continue labeling as required.
  2. Click Submit when you are done labeling the task.
This task assumes you are already in the Label Editor.Method 1
  1. Hover over a label in the Label Editor workspace.
  2. Right click. A menu appears.
  3. Click one of the following:
    • Track forwards or use the keyboard shortcut SHIFT + T.
    • Track backwards or use the keyboard shortcut SHIFT + OPTION/ALT + T.
  4. Click Save.
Use the keyboard shortcut ENTER or middle click to save.
  1. Continue labeling/tracking objects as required.
  2. Click Submit to submit the task.
Method 2
  1. Hover over a label in the Labels list. A small groups of icons appear.
  2. Click the kebab icon. A menu appears.
  3. Click one of the following:
    • Track forwards from this frame or use the keyboard shortcut SHIFT + T.
    • Track backwards from this frame or use the keyboard shortcut SHIFT + OPTION/ALT + T.
  4. Click Save.
Use the keyboard shortcut ENTER or middle click to save.
  1. Continue labeling/tracking objects as required.
  2. Click Submit to submit the task.

​
Label multiple objects

You can track objects in videos, image groups, or image sequences.
  1. Go to Annotate > Projects.
  2. Click a Project from the Project list.
  3. OPTIONAL: Filter the data units as required from the Queue.
  4. Click Start task, Initiate, or use the keyboard shortcut Command + Enter in the Queue. The Label Editor appears.
  5. Click the Detect all button for the object class from the Classes list.
You can also use the keyboard shortcut for the class, then click SHIFT + A, and then click Detect all > Detect all objects.
You can specify a global setting for number of frames to track in Editor Settings > Label Tracking > Number of frames to predict.
  1. Click Save.
Use the keyboard shortcut ENTER or middle click to save.
  1. Continue labeling as required.
  2. Click Submit when you are done labeling the task.

​
Use SAM 3 prompt

  1. Go to Annotate > Projects.
  2. Click a Project from the Project list.
  3. OPTIONAL: Filter the data units as required from the Queue.
  4. Click Start task, Initiate, or use the keyboard shortcut Command + Enter in the Queue. The Label Editor appears.
  5. Click the Toggle auto-annotation button for the object class from the Classes list. A dialog appears for SAM 3.
Use the keyboard shortcut for the object class then use the keyboard shortcut SHIFT + A.
  1. Select Detect all.
  2. Provide a prompt to start labeling.
  3. Click on an object in your image or video.
Use the following when needed:
  • Left click to INCLUDE an area.
  • Right click to EXCLUDE an area.
  1. Specify the number of frames to track the object across.
    You can specify a global setting for number of frames to track in Editor Settings > Label Tracking > Number of frames to predict.
  2. Click Track backwards or Track forwards.
  3. Click Save.
Use the keyboard shortcut ENTER or middle click to save.
  1. Continue labeling as required.
  2. Click Submit when you are done labeling the task.

​
Track multiple objects

You can track objects in videos or image sequences. You can track multiple objects, of multiple classes, at the same time.
In this example, we’ll label multiple objects at once by class and then track all of them.
  1. Go to Annotate > Projects.
  2. Click a Project from the Project list.
  3. OPTIONAL: Filter the data units as required from the Queue.
  4. Click Start task, Initiate, or use the keyboard shortcut Command + Enter in the Queue. The Label Editor appears.
  5. Click the Toggle auto-annotation button for the object class from the Classes list. A dialog appears for SAM 3.
Use the keyboard shortcut for the object class then use the keyboard shortcut SHIFT + A.
  1. Select Detect all.
  2. Select the labels you want to track objects on by doing one of the following:
    • Hold down SHIFT and select the labels to track from the Label Editor workspace or from the Labels list
    • Left click and drag the area, in the Label Editor workspace, with the labels you want to track.
  3. Right click. A menu appears.
  4. Click one of the following:
    • Track forwards or use the keyboard shortcut SHIFT + T.
    • Track backwards or use the keyboard shortcut SHIFT + OPTION/ALT + T.
    You can specify a global setting for number of frames to track in Editor Settings > Label Tracking > Number of frames to predict.
  5. Click Save.
Use the keyboard shortcut ENTER or middle click to save.
  1. Continue labeling and tracking as required.
  2. Click Submit when you are done labeling the task.
This task assumes you are already in the Label Editor.Method 1
  1. Select the labels you want to track objects on:
    • Hold down SHIFT and select the labels to track in the Label Editor workspace.
    • Left click and drag the area, in the Label Editor workspace, with the labels you want to track.
  2. Right click. A menu appears.
  3. Click one of the following:
    • Track forwards or use the keyboard shortcut SHIFT + T.
    • Track backwards or use the keyboard shortcut SHIFT + OPTION/ALT + T.
  4. Click Save.
Use the keyboard shortcut ENTER or middle click to save.
  1. Continue labeling/tracking objects as required.
  2. Click Submit to submit the task.
Method 2
  1. Hold down SHIFT and select labels in the Labels list.
  2. Move your cursor to the Label Editor workspace.
  3. Right click. A menu appears.
  4. Click one of the following:
    • Track forwards or use the keyboard shortcut SHIFT + T.
    • Track backwards or use the keyboard shortcut SHIFT + OPTION/ALT + T.
  5. Click Save.
Use the keyboard shortcut ENTER or middle click to save.
  1. Continue labeling/tracking objects as required.
  2. Click Submit to submit the task.

​
SAM 2

​
Label one object

Labeling one object is the same for images and videos.
  1. Go to Annotate > Projects.
  2. Click a Project from the Project list.
  3. OPTIONAL: Filter the data units as required from the Queue.
  4. Click Start task, Initiate, or use the keyboard shortcut Command + Enter in the Queue. The Label Editor appears.
  5. Click the Toggle auto-annotation button for the object class from the Classes list.
    Use the keyboard shortcut for the object class then use the keyboard shortcut SHIFT + A.
    A dialog appears for SAM 2.
  6. Click on an object in your image or video.
Use the following when needed:
  • Left click to INCLUDE an area.
  • Right click to EXCLUDE an area.
  1. Click Save.
Use the keyboard shortcut ENTER or middle click to save.
  1. Continue labeling as required.
  2. Click Submit to submit the task.

​
Track one object

You can track objects in videos or image sequences.
  1. Go to Annotate > Projects.
  2. Click a Project from the Project list.
  3. OPTIONAL: Filter the data units as required from the Queue.
  4. Click Start task, Initiate, or use the keyboard shortcut Command + Enter in the Queue. The Label Editor appears.
  5. Click the Toggle auto-annotation button for the object class from the Classes list. A dialog appears.
Use the keyboard shortcut for the object class then use the keyboard shortcut SHIFT + A.
  1. Click on an object in your image or video.
Use the following when needed:
  • Left click to INCLUDE an area.
  • Right click to EXCLUDE an area.
  1. Specify the number of frames to track the object across.
    You can specify a global setting for number of frames to track in Editor Settings > Label Tracking > Number of frames to predict.
  2. Click Track backwards or Track forwards.
  3. Click Save.
Use the keyboard shortcut ENTER or middle click to save.
  1. Continue labeling as required.
  2. Click Submit when you are done labeling the task.
This task assumes you are already in the Label Editor.Method 1
  1. Hover over a label in the Label Editor workspace.
  2. Right click. A menu appears.
  3. Click one of the following:
    • Track forwards or use the keyboard shortcut SHIFT + T.
    • Track backwards or use the keyboard shortcut SHIFT + OPTION/ALT + T.
  4. Click Save.
Use the keyboard shortcut ENTER or middle click to save.
  1. Continue labeling/tracking objects as required.
  2. Click Submit to submit the task.
Method 2
  1. Hover over a label in the Labels list. A small groups of icons appear.
  2. Click the kebab icon. A menu appears.
  3. Click one of the following:
    • Track forwards from this frame or use the keyboard shortcut SHIFT + T.
    • Track backwards from this frame or use the keyboard shortcut SHIFT + OPTION/ALT + T.
  4. Click Save.
Use the keyboard shortcut ENTER or middle click to save.
  1. Continue labeling/tracking objects as required.
  2. Click Submit to submit the task.

​
Track multiple objects

You can track objects in videos or image sequences. You can track multiple objects, of multiple classes, at the same time.
In this example, we’ll label multiple objects, and then track all of them.
  1. Go to Annotate > Projects.
  2. Click a Project from the Project list.
  3. OPTIONAL: Filter the data units as required from the Queue.
  4. Click Start task, Initiate, or use the keyboard shortcut Command + Enter in the Queue. The Label Editor appears.
  5. Click the Toggle auto-annotation button for the object class from the Classes list. A dialog appears.
Use the keyboard shortcut for the object class then use the keyboard shortcut SHIFT + A.
  1. Create and edit some object labels.
  2. Select the labels you want to track objects on by doing one of the following:
    • Hold down SHIFT and select the labels to track from the Label Editor workspace or from the Labels list
    • Left click and drag the area, in the Label Editor workspace, with the labels you want to track.
  3. Right click. A menu appears.
  4. Click one of the following:
    • Track forwards or use the keyboard shortcut SHIFT + T.
    • Track backwards or use the keyboard shortcut SHIFT + OPTION/ALT + T.
You can specify a global setting for number of frames to track in Editor Settings > Label Tracking > Number of frames to predict.
  1. Click Save.
Use the keyboard shortcut ENTER or middle click to save.
  1. Continue labeling and tracking as required.
  2. Click Submit when you are done labeling the task.
This task assumes you are already in the Label Editor.Method 1
  1. Select the labels you want to track objects on:
  • Hold down SHIFT and select the labels to track in the Label Editor workspace.
  • Left click and drag the area, in the Label Editor workspace, with the labels you want to track.
  1. Right click. A menu appears.
  2. Click one of the following:
    • Track forwards or use the keyboard shortcut SHIFT + T.
    • Track backwards or use the keyboard shortcut SHIFT + OPTION/ALT + T.
  3. Click Save.
Use the keyboard shortcut ENTER or middle click to save.
  1. Continue labeling/tracking objects as required.
  2. Click Submit to submit the task.
Method 2
  1. Hold down SHIFT and select labels in the Labels list.
  2. Move your cursor to the Label Editor workspace.
  3. Right click. A menu appears.
  4. Click one of the following:
    • Track forwards or use the keyboard shortcut SHIFT + T.
    • Track backwards or use the keyboard shortcut SHIFT + OPTION/ALT + T.
  5. Click Save.
Use the keyboard shortcut ENTER or middle click to save.
  1. Continue labeling/tracking objects as required.
  2. Click Submit to submit the task.

​
Interpolation

Use interpolation to automatically create labels by estimating the location that labels should be created, between a start frame and an end frame.
  1. Go to Annotate > Projects.
  2. Click a Project from the Project list.
  3. OPTIONAL: Filter the data units as required from the Queue.
  4. Click Start task, Initiate, or use the keyboard shortcut Command + Enter in the Queue. The Label Editor appears.
  5. Create a label, for an object, on a start frame.
  6. Move forward in the data unit to the location of the end frame.
  7. Click Instantiate object in frame for the object from the Labels list.
  8. Do one of the following to interpolate/track between the start and end frames:
    • Right click the label and click Interpolate from the menu.
    • Click the kebab icon for the label in the Labels list and click Interpolate.
    • Use the shortcut SHIFT + I.
9 . Edit the created labels as needed.
  1. Continue creating and editing labels.
  2. Click Submit.

​
Re-Interpolate

Interpolation results can be improved by manually correcting some interpolated labels, and re-running the interpolation. Enable the Interpolation auto adjustments toggle in the Drawing settings section of the label editor settings. This ensures that only labels with a confidence score (α) less than 100% are overwritten in successive interpolations, while manual labels (α = 100%) are kept and serve as key frames.
This process can be repeated any number of times.
All labels created using interpolation are always assigned α = 99%.

​
Agents

Your Workspace/Project Admin communicates that they configured an Editor Agent to perform annotation on tasks in the Queue. You trigger the Editor Agent from inside the Label Editor.
  1. Go to Annotate > Projects.
  2. Click a Project from the Project list.
  3. OPTIONAL: Filter the data units as required from the Queue.
  4. Click Start task, Initiate, or use the keyboard shortcut Command + Enter in the Queue. The Label Editor appears.
  5. Move the cursor to the Label Editor workspace.
  6. Right click. A menu appears.
  7. Hover over Trigger agent. A menu appears.
  8. Click the Agent you want to trigger.
  9. Edit the labels as required.
  10. Click Submit.