Scene Annotation - E2E

This tutorial shows the shortest production-style path for working with point cloud and multimodal Scenes in Encord: prepare sensor assets, register Scenes, create a Project, annotate, review, and reuse the Scene data.

Enable hardware acceleration in Chrome before annotating large point clouds or multimodal Scenes.

What You Need

A cloud storage integration that can access your point cloud and image assets.
A folder in Files & Folders for the registered Scenes.
A Dataset and Project for annotation work.
An Ontology with the Scene tools you need, such as cuboids, polylines, keypoints, or classifications.

1. Prepare Scene Assets

Put each Scene’s files in a stable cloud prefix. For example:

s3://my-bucket/scenes/scene-001/
  lidar/000000.pcd
  lidar/000001.pcd
  front/000000.jpg
  front/000001.jpg

Use the same frame index for files that should appear together. If your sensors are asynchronous, use sensor timestamps instead and keep the timestamps consistent across streams.

2. Register Scenes

Use the Python SDK to build and upload Scenes. The SDK is preferred over hand-authored JSON because it validates stream names, references, camera links, and frame-of-reference relationships before registration.

Build and Upload a Scene

from uuid import UUID

from encord.beta.scene import SceneBuilder, identity_pose, intrinsics_pinhole
from encord.storage import StorageFolder


def build_scene() -> SceneBuilder:
    scene = SceneBuilder()

    # Anchor the sensor streams to the moving platform.
    ego = scene.add_for_stream("ego")
    ego.add_pose(identity_pose(), timestamp=0)

    # Matching timestamps make the point cloud and image appear together.
    scene.add_pcd_stream("lidar", frame_of_reference=ego).add_pcd(
        uri="s3://my-bucket/scenes/scene-001/lidar/000000.pcd",
        timestamp=0,
    )

    # Inline camera parameters create a camera stream named "front/camera".
    scene.add_image_stream(
        "front",
        width=1920,
        height=1080,
        intrinsics=intrinsics_pinhole(fx=1000, fy=1000, ox=960, oy=540),
        timestamp=0,
        frame_of_reference=ego,
    ).add_image(
        uri="s3://my-bucket/scenes/scene-001/front/000000.jpg",
        timestamp=0,
    )

    return scene


def register_scene(storage_folder: StorageFolder, integration_id: str) -> UUID:
    scene = build_scene()

    # Pass the SceneBuilder itself. upload_scene validates and serializes it internally.
    return storage_folder.upload_scene(
        scene=scene,
        title="scene-001",
        integration_id=integration_id,
        client_metadata={"source": "sample"},
    )

For sequences, additional sensors, and distortion models, see Create and Register Scenes and Scene SDK Workflows.

3. Create the Annotation Project

Add the registered Scene items to a Dataset.
Create or reuse an Ontology for the labels you need.
Create a Project from the Dataset and Ontology.
Assign work through your Workflow as usual.

Use metadata such as route, split, capture_date, or scenario on Scene items if you need to filter, prioritize, or route tasks later.

4. Annotate Scenes

Open a task and use the Scene editor to inspect the point cloud, camera views, and synchronized frames. Recommended first pass:

Confirm the point cloud orientation and scale look correct.
Check that camera images align with the point cloud if calibration is present.
Use the built-in Scene navigation tutorial to choose mouse or trackpad controls.
Create the first few labels, move through frames, and verify object continuity.

For editor workflows, see Annotating Scenes.

5. Review and Reuse Scene Data

After annotation, spot-check a few completed tasks in review. Pay special attention to frame alignment and camera projection quality; most Scene issues are easiest to catch visually. Use SceneReader when you need to inspect, copy, or migrate registered Scene assets.

Read and Copy a Scene

from uuid import UUID

from encord.beta.scene import SceneReader
from encord.orm.storage import DataUploadItems, StorageItemType
from encord.storage import StorageFolder, StorageItem


def find_scene(storage_folder: StorageFolder, title: str) -> StorageItem:
    scene_items = storage_folder.list_items(
        search=title,
        item_types=[StorageItemType.SCENE],
        page_size=1,
    )
    scene_item = next(iter(scene_items), None)
    if scene_item is None:
        raise ValueError(f"Scene with title '{title}' not found.")
    return scene_item


def get_lidar_signed_url(scene_item: StorageItem, timestamp: float = 0) -> str:
    scene = SceneReader(scene_item).read()
    lidar = scene.get_stream("lidar", kind="point_cloud")
    return lidar.get_event(timestamp=timestamp).signed_url


def get_image_signed_urls(scene_item: StorageItem, timestamp: float = 0) -> list[tuple[str, str]]:
    scene = SceneReader(scene_item).read()
    return [(stream_id, event.signed_url) for stream_id, event in scene.get_images_at_timestamp(timestamp)]


def copy_scene_to_folder(
    scene_item: StorageItem,
    target_storage_folder: StorageFolder,
    *,
    integration_id: str,
    title: str,
    source_uri_prefix: str,
    target_uri_prefix: str,
) -> UUID:
    payload = SceneReader(scene_item).to_upload_payload(
        title=title,
        uri_mapper=lambda uri: uri.replace(source_uri_prefix, target_uri_prefix),
    )

    return target_storage_folder.add_private_data_to_folder_start(
        integration_id=integration_id,
        private_files=DataUploadItems(scenes=[payload]),
    )

Troubleshooting

Symptom	Check
Scene does not load	Confirm the integration can access every URI used in the Scene.
Camera view is offset	Check camera intrinsics, sensor pose, and world/camera conventions.
Streams advance out of sync	Check the timestamps assigned to each stream event.
Editor feels slow	Enable browser hardware acceleration and test with a smaller sample Scene.

Overview

LiDAR, Radar and Sensor Data

Time Series Data

Visual Data

Audio Data

Text, HTML, and PDFs

DICOM and NIFTI

Multimodal

Geospatial Data

Scene Annotation - E2E

What You Need

1. Prepare Scene Assets

2. Register Scenes

3. Create the Annotation Project

4. Annotate Scenes

5. Review and Reuse Scene Data

Troubleshooting

​What You Need

​1. Prepare Scene Assets

​2. Register Scenes

​3. Create the Annotation Project

​4. Annotate Scenes

​5. Review and Reuse Scene Data

​Troubleshooting

What You Need

1. Prepare Scene Assets

2. Register Scenes

3. Create the Annotation Project

4. Annotate Scenes

5. Review and Reuse Scene Data

Troubleshooting