Segment anything model (SAM)

Suggest Edits

Segment anything model (SAM) allows you to automatically create labels around distinct features in all supported file formats. Read more about auto segmentation advances in our explainer blog post here.

ℹ️

Note

SAM is available for ontologies that have the Polygon, Bounding box, or Bitmask annotation types.

See the video, or use the step-by-step tutorial below to learn how to use SAM effectively.

1. Start labeling using SAM

Creating a new instance label

Click the icon within the Polygon, or Bounding box class when viewing a task in the label editor to make the SAM pop-up appear.

👍

Tip

Use the Shift + A keyboard shortcut to toggle SAM mode.

Creating labels for existing instances

You can use SAM to create labels for an existing object instance.

  1. Navigate to the frame you want to add the instance.
  2. Click Instantiate object next to the instance, or press the instance's hotkey on your keyboard (W in the example below).
  1. Press Shift + A on your keyboard to enable SAM.

👍

Tip

SAM mode will remain active when switching to a different annotation class, and will only be deactivated by clicking the icon again.

2. Using SAM to segment a frame or image

Click the area on the image or frame you would like to segment - the pop-up will indicate that auto-segmentation is running, as shown below.

You can also click and drag your cursor to select the part of an image you would like to segment.

3. Including / Excluding areas from selection

Once the prediction has run, a part of the image or frame will be highlighted in blue.

  • You can add to the selected area by left-clicking the part of the image you'd like to add to the label. This is shown in the video below.
  • You can exclude parts of the selected area by right-clicking the part you'd like to exclude from the label.
  • If you've made a mistake and want to re-start, simply click Reset on the SAM pop-up to start again.

4. Confirming the label

Once the correct section is highlighted, click Apply Label on the SAM pop-up, or press Enter on your keyboard to confirm the selection.

A label appears outlining the area that was highlighted - the label shape depends on whether the selection was made using a bounding box, a bitmask, or a polygon.

ℹ️

Note

SAM works in conjunction with the 'Permanent drawing' setting which can increase the speed at which SAM labels are created in video frames.

Updated 19 days ago