> ## Documentation Index
> Fetch the complete documentation index at: https://docs.encord.com/llms.txt
> Use this file to discover all available pages before exploring further.
# SAM Segmentation & Tracking
Segment Anything Model (SAM) allows you to automatically create labels around distinct features in all supported file formats.
* SAM 2 is used for tracking.
* SAM 3 is used for labeling objects
***
## Segmentation
### Availability
| Ontology Shape | Supported? |
| ---------------------- | ---------- |
| Polygon | ✅ |
| Bounding box | ✅ |
| Bitmask | ✅ |
| Rotatable bounding box | ✅ |
| Keypoint | ❌ |
| Polyline | ❌ |
| Object Primitive | ❌ |
| Modality | Supported |
| -------------- | --------- |
| Single Image | ✅ |
| Image Group | ✅ |
| Image Sequence | ✅ |
| Video | ✅ |
| Audio | ❌ |
| Text / HTML | ❌ |
| Document | ❌ |
### Label New Instances
Use the Shift + A keyboard shortcut to toggle SAM mode. Enabling SAM mode automatically activates SAM whenever the class is selected, so there's no need to click the wand icon separately.
1. Click the wand icon on Ontology class you want to label with.
2. Click, or click-and-drag the cursor across the part of the image you want to label.
A popup appears.
* Left-click parts of the image to include them in the label.
* Right-click parts of the selection to exclude them from the label.
3. Click **Save** or press ENTER
### Label Existing Instances
Videos and image sequences can show a single object instance in multiple frames.
1. Navigate to the frame you want to create the label.
2. Click the **Toggle auto-annotation** button or use the SHIFT + A keyboard shortcut.
SAM mode remains active when switching to a different annotation class. Deactivate SAM by clicking the **Toggle auto-annotation** button, or using the SHIFT + A keyboard shortcut again.
3. Click, or click-and-drag the cursor across the part of the image you want to label.
A popup appears.
* Left-click parts of the image to include them in the label.
* Right-click parts of the selection to exclude them from the label.
4. Click **Save** or press ENTER
### Label All Objects
There are two ways to label all objects of a specific Ontology class.
**Method 1**:
Click the *Sparkles* symbol in the Class.
This action can be undone using CMD / CTRL + Z
**Method 2**:
1. Click the *Wand* icon in the Class.
2. In the pop-up, ensure the toggle is set to **All objects prediction**.
3. Type the name of the class you want to label.
4. Click **Detect all objects**.
5. Optionally, remove unwanted objects using the *Delete* button on an instance.
6. Click **Done**.
***
### Label One Or More Objects
1. Select the *Wand* icon in the Ontology class you want to label.
A pop-up appears.
2. In the SAM pop-up, ensure the toggle is set to **Single-object detection**.
3. Select the objects you want to label by either clicking a single object or clicking and dragging across the area of the frame containing multiple objects.
4. Optionally track the objects across frames.
5. Click **Save**.
***
### Panoptic Exclude Mode
When creating bitmask labels with single object detection, you can use panoptic exclude mode to automatically exclude existing bitmasks from new labels. This prevents overlapping bitmasks and ensures clean, non-overlapping annotations.
Panoptic exclude mode only works with bitmask annotations and is automatically applied when using SAM.
To use panoptic exclude mode:
1. Enable **SAM mode** by clicking the wand icon or pressing Shift + A
2. Select the **Bitmask** annotation class
3. In the bitmask settings panel, set **Panoptic mode** to **Exclude**
4. Use SAM single object detection to create new bitmask labels
When you create a new bitmask with SAM, it automatically excludes any existing bitmasks in the same area, ensuring your annotations don't overlap.
***
## Tracking
Encord supports tracking objects forwards and backwards across multiple frames. This allows you to create labels for an object in one frame and automatically generate labels for the same object in subsequent frames.
Video tracking is computationally intensive and performance might be slower than other labeling actions.
### Availability
The following table shows the Ontology shapes for which forwards and backwards tracking can be used.
| **Ontology shape** | **SAM Tracking** |
| ---------------------- | ---------------- |
| Bounding box | ✅ |
| Rotatable bounding box | ✅ |
| Polygon | ✅ |
| Polyline | ✅ |
| Object Primitive | ✅ |
| Keypoint | ✅ |
| Bitmask | ✅ |
The following table shows the modalities that support forwards and backwards tracking.
| **Modalities** | **SAM Tracking** |
| --------------- | ------------------ |
| Images | ❌ |
| Videos | ✅ |
| Image Groups | ✅ (Forwards only) |
| Image Sequences | ✅ (Forwards only) |
| DICOM | ✅ (Forwards only) |
| Audio | ❌ |
| Text / HTML | ❌ |
| Documents | ❌ |
### Track While Creating Labels (Bounding Boxes, Polygons, Bitmasks)
Before you begin, set your tracking range in the [Editor Settings](/platform-documentation/Annotate/annotate-label-editor/annotate-label-editor-settings-shortcuts#object-tracking). By default, object tracking is configured to run for 30 frames. This means objects are tracked forwards or backwards across 30 consecutive frames.
1. Click the *wand* icon next to the class you want to label, or press Shift + A to toggle SAM mode.
2. Click the instance you want to label. Adjust your label until the correct region is selected (if needed).
3. Specify the number of frames you want to track the instance across.
4. Click **Backwards** or **Forwards** to start the tracking algorithm.
You can also use keyboard shortcuts to initiate tracking:
* Tracking forwards: SHIFT + T
* Tracking backwards: SHIFT + T + OPTION
5. Click **Save** when tracking completes to save the generated labels.
### Track Existing Instances
**Method 1**:
The **Automated labeling** feature supports tracking forwards through data units. Specify frames after the existing frame to track.
1. Click **Automated labeling**.
2. Expand the **Tracking and interpolation** section.
3. Select the instances you want to track.
4. Make sure the **Tracking** method is selected.
5. Set the **tracking annotation interval**. This defines how many consecutive frames an instance can be missing before the tracking algorithm stops.
6. Specify the **frame range** you want to track the selected instances across.
7. Click **Run tracking** to begin the process.
**Method 2**:
Before you begin, set your default tracking range in the [Editor Settings](/platform-documentation/Annotate/annotate-label-editor/annotate-label-editor-settings-shortcuts#object-tracking). By default, object tracking is configured to run for 30 frames, meaning SAM tracking tracks the object forwards or backwards across 30 consecutive frames.
1. Right-click the instance label you want to start tracking.
2. Click **Track forwards** or **Track backwards** to run tracking for 30 consecutive frames.
You can also use keyboard shortcuts to initiate tracking:
* Tracking forwards: SHIFT + T
* Tracking backwards: SHIFT + T + OPTION
**Method 3 — Configurable editor object table**:
In the configurable editor, you can initiate tracking directly from the object table without using the sidebar or right-clicking on the canvas.
**Track forwards** and **Track backwards** only appear in the row menu for trackable shapes on frame-based media (videos, image sequences, image groups, and DICOM). These options are hidden in view-only mode.
1. Hover over the instance row in the object table. A **…** (more actions) button appears at the end of the row.
2. Click the **…** button to open the actions menu.
3. Click **Track forwards** or **Track backwards** to start tracking using your configured tracking range.
### Tracking Multiple Objects
To track multiple objects of any shape:
1. Click one of the objects you want to track.
2. Hold Shift on your keyboard and select all the other objects you want to track.
3. Right-click on one of the selected objects and click **Track forwards all** or **Track backwards all**.
You can also use keyboard shortcuts to initiate tracking on multiple object instances:
* Track forwards all: SHIFT + T
* Track backwards all: SHIFT + T + OPTION
***
### Settings
The *Object tracking* section of the [editor settings](/platform-documentation/Annotate/annotate-label-editor/annotate-label-editor-settings-shortcuts#object-tracking) allows you to adjust the following.
**Change Tracking Range**
The range that tracking runs can be adjusted in the *Object tracking* section of the [editor settings](/platform-documentation/Annotate/annotate-label-editor/annotate-label-editor-settings-shortcuts#object-tracking). The default tracking range is set to 30 frames. This range includes the frame that tracking starts on.
**Advanced Tracking**
This toggle lets you choose between a faster, simpler tracking algorithm or a slower, more advanced one with higher accuracy. The advanced algorithm is enabled by default.
