Scene SDK Workflows

Upload an MCAP recording directly as a Scene, or use the Scene SDK (encord.beta.scene) to combine separate point cloud, image, camera, and frame-of-reference streams without writing raw Scene JSON. Use this guide for the workflow. Use the Scene SDK reference for method-level details.

SDK Version Requirements

The examples on this page need encord 0.1.205 or later. To update, run pip install --upgrade encord. The Scene features were added in these versions:

Feature	Added in
`SceneBuilder` and `StorageFolder.upload_scene`	0.1.196
`SceneReader`, including `SceneReader.to_upload_payload`	0.1.198
`StorageFolder.upload_scene_file`, Scene view settings, Scene layouts, and time series streams	0.1.202
Reading frame-of-reference streams with `kind="frame_of_reference"`	0.1.205
Image stream `rotation`	0.1.212

SceneRead is a deprecated alias of SceneReader. Use SceneReader in new code.

Upload an MCAP Recording

MCAP support is in Beta. Contact your Encord support representative to enable the feature for your Organization.

For an .mcap recording on your computer, use StorageFolder.upload_scene_file. You do not need a SceneBuilder or a cloud integration to upload a local file to Encord-managed storage. upload_scene_file also uploads other single-file Scenes: .bag (ROS 1), .db3 (ROS 2), .e57, .las, .laz, .pcd, and .ply files. It detects the format from the file extension. For an .mcap recording in your cloud storage, register it through a cloud integration with StorageFolder.add_private_data_to_folder_start.

# Import dependencies
from encord import EncordUserClient
from encord.storage import StorageFolder

# User input
SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"  # Replace with the file path to your SSH private key
FOLDER_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the unique ID for the folder in Files & Folders
MCAP_FILE_PATH = "/Users/chris-encord/recording.mcap"  # Replace with the file path to your MCAP recording
SCENE_TITLE = "Robot recording"  # Replace with the title for the Scene

# Create user client using ssh key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
    # For US platform users use "https://api.us.encord.com"
    domain="https://api.encord.com",
)

# Get the folder
folder: StorageFolder = user_client.get_storage_folder(FOLDER_ID)

# Upload the MCAP recording as a Scene
scene_uuid = folder.upload_scene_file(file_path=MCAP_FILE_PATH, title=SCENE_TITLE)
print(f"Uploaded Scene item {scene_uuid}")

# Import dependencies
from encord import EncordUserClient
from encord.orm.storage import DataUploadItems, DataUploadScene
from encord.storage import StorageFolder

# User input
SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"  # Replace with the file path to your SSH private key
FOLDER_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the unique ID for the folder in Files & Folders
INTEGRATION_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the unique ID for the cloud storage integration
MCAP_URI = "s3://my-bucket/recording.mcap"  # Replace with the URI of the MCAP recording in your cloud storage
SCENE_TITLE = "Robot recording"  # Replace with the title for the Scene

# Create user client using ssh key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
    # For US platform users use "https://api.us.encord.com"
    domain="https://api.encord.com",
)

# Get the folder
folder: StorageFolder = user_client.get_storage_folder(FOLDER_ID)

# Register the MCAP recording as a Scene
upload_job_id = folder.add_private_data_to_folder_start(
    integration_id=INTEGRATION_ID,
    private_files=DataUploadItems(
        scenes=[
            DataUploadScene(
                title=SCENE_TITLE,
                scene={"url": MCAP_URI, "format": "mcap"},
            ),
        ],
    ),
)

# Wait for the registration to finish
result = folder.add_private_data_to_folder_get_result(upload_job_id)
print(result)

Replace the values in the # User input section with your own. Add the returned Scene item to a Dataset before creating an annotation Project. See MCAP Support for supported streams, transforms, size guidance, and examples. Use StorageItem.update(scene_layout=...) to configure an MCAP layout, and follow MCAP Scene labels for continuous MCAP Scene 3D events, segmentation, and time-series labels.

Build and Upload a Composite Scene

Build the Scene with SceneBuilder, then register it into Files & Folders with StorageFolder.upload_scene.

Build and Upload a Scene

# Import dependencies
from uuid import UUID

from encord import EncordUserClient
from encord.beta.scene import SceneBuilder, identity_pose, intrinsics_pinhole
from encord.storage import StorageFolder

# User input
SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"  # Replace with the file path to your SSH private key
FOLDER_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the unique ID for the folder in Files & Folders
INTEGRATION_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the ID of a cloud integration that can read every URI in the Scene
SCENE_TITLE = "scene-001"  # Replace with the title for the Scene


def build_scene() -> SceneBuilder:
    scene = SceneBuilder()

    # Timestamps are whole-number frame indices that start at 0.
    # Events with the same timestamp appear together in the editor.

    # Anchor the sensor streams to the moving platform.
    ego = scene.add_for_stream("ego")
    ego.add_pose(identity_pose(), timestamp=0)

    scene.add_pcd_stream("lidar", frame_of_reference=ego).add_pcd(
        uri="s3://my-bucket/scenes/scene-001/lidar/000000.pcd",
        timestamp=0,
    )

    # Inline camera parameters create a camera stream named "front/camera".
    # The timestamp argument is required: it is the timestamp of that camera's parameters.
    scene.add_image_stream(
        "front",
        width=1920,
        height=1080,
        intrinsics=intrinsics_pinhole(fx=1000, fy=1000, ox=960, oy=540),
        timestamp=0,
        frame_of_reference=ego,
    ).add_image(
        uri="s3://my-bucket/scenes/scene-001/front/000000.jpg",
        timestamp=0,
    )

    return scene


def register_scene(storage_folder: StorageFolder, integration_id: str, title: str) -> UUID:
    # Pass the SceneBuilder itself. upload_scene validates and serializes it,
    # waits for registration to finish, and returns the new Scene item's UUID.
    return storage_folder.upload_scene(
        scene=build_scene(),
        title=title,
        integration_id=integration_id,
        client_metadata={"source": "sample"},
    )


# Create user client using ssh key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
    # For US platform users use "https://api.us.encord.com"
    domain="https://api.encord.com",
)

# Get the folder and register the Scene
folder: StorageFolder = user_client.get_storage_folder(FOLDER_ID)
scene_uuid = register_scene(folder, INTEGRATION_ID, SCENE_TITLE)
print(f"Registered Scene item {scene_uuid}")

upload_scene validates the Scene, waits for registration to finish, and returns the new Scene item’s UUID. Add that item to a Dataset before creating an annotation Project.

Add Multi-frame Sensor Streams

Create stream builders once, then append events. Use the same timestamp for events that should appear together in the editor.

Multi-frame Scene Streams

# Import dependencies
from encord import EncordUserClient
from encord.beta.scene import Direction, SceneBuilder, intrinsics_pinhole, translation_only
from encord.storage import StorageFolder

# User input
SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"  # Replace with the file path to your SSH private key
FOLDER_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the unique ID for the folder in Files & Folders
INTEGRATION_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the ID of a cloud integration that can read every URI in the Scene
FRAME_COUNT = 3  # Replace with the number of frames in your sequence


def build_multiframe_scene(frame_count: int) -> SceneBuilder:
    scene = SceneBuilder()
    # World and camera conventions must have the same handedness.
    scene.set_world_convention(x=Direction.FORWARD, y=Direction.LEFT, z=Direction.UP)
    scene.set_camera_convention(x=Direction.RIGHT, y=Direction.DOWN, z=Direction.FORWARD)

    # Create each stream builder once, then append one event per frame.
    ego = scene.add_for_stream("ego")
    lidar = scene.add_pcd_stream(
        "lidar",
        frame_of_reference=ego,
        pose=translation_only(x=0.0, y=0.0, z=1.8),  # LiDAR mounted 1.8 m above the ego origin
    )
    front = scene.add_image_stream(
        "front",
        width=1920,
        height=1080,
        intrinsics=intrinsics_pinhole(fx=1000, fy=1000, ox=960, oy=540),
        timestamp=0,
        frame_of_reference=ego,
        pose=translation_only(x=1.2, y=0.0, z=1.5),  # Camera 1.2 m forward and 1.5 m up
    )

    # Timestamps are whole-number frame indices that start at 0.
    # Give events the same timestamp to show them together in the editor.
    for frame in range(frame_count):
        ego.add_pose(translation_only(x=frame * 0.5, y=0.0, z=0.0), timestamp=frame)
        lidar.add_pcd(
            uri=f"s3://my-bucket/scenes/scene-001/lidar/{frame:06d}.pcd",
            timestamp=frame,
        )
        front.add_image(
            uri=f"s3://my-bucket/scenes/scene-001/front/{frame:06d}.jpg",
            timestamp=frame,
        )

    return scene


# Create user client using ssh key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
    # For US platform users use "https://api.us.encord.com"
    domain="https://api.encord.com",
)

# Register the Scene. upload_scene waits for registration and returns the Scene item's UUID.
folder: StorageFolder = user_client.get_storage_folder(FOLDER_ID)
scene_uuid = folder.upload_scene(
    scene=build_multiframe_scene(FRAME_COUNT),
    title="scene-001",
    integration_id=INTEGRATION_ID,
)
print(f"Registered Scene item {scene_uuid}")

Timestamps must be whole numbers that start at 0, such as the frame indices 0, 1, 2, .... Fractional timestamps such as 0.5 fail registration, and a Scene that starts at a large value, such as a Unix epoch timestamp, does not open in the label editor. If your sensors are sampled at different rates, convert their timestamps to integer frame indices on one shared timeline that starts at 0.

Define Camera Streams Separately

Passing width, height, intrinsics, and timestamp to add_image_stream creates a camera stream named <image stream name>/camera automatically. All four arguments are required in this mode. To control the camera stream yourself, for example to give it its own name, mount pose, or a new set of camera parameters when calibration changes, create it with SceneBuilder.add_camera_stream. Then link the image stream to it using the camera argument.

Separate Camera Stream

# Import dependencies
from encord import EncordUserClient
from encord.beta.scene import SceneBuilder, identity_pose, intrinsics_pinhole, translation_only
from encord.storage import StorageFolder

# User input
SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"  # Replace with the file path to your SSH private key
FOLDER_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the unique ID for the folder in Files & Folders
INTEGRATION_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the ID of a cloud integration that can read every URI in the Scene
FRAME_COUNT = 3  # Replace with the number of frames in your sequence


def build_scene_with_camera_stream(frame_count: int) -> SceneBuilder:
    scene = SceneBuilder()

    ego = scene.add_for_stream("ego")

    # Camera parameters stream, mounted 1.2 m forward and 1.5 m up from the ego frame of reference
    front_camera = scene.add_camera_stream(
        "front_camera",
        frame_of_reference=ego,
        pose=translation_only(x=1.2, y=0.0, z=1.5),
    )
    front_camera.add_camera_params(
        1920,
        1080,
        intrinsics_pinhole(fx=1000, fy=1000, ox=960, oy=540),
        timestamp=0,
    )
    # If the calibration changes, add another set of camera parameters at the frame where it changes.

    # Link the image stream to the camera stream by its name.
    # Set frame_of_reference and pose on the camera stream, not on the image stream.
    front = scene.add_image_stream("front", camera="front_camera")

    # Timestamps are whole-number frame indices that start at 0.
    for frame in range(frame_count):
        ego.add_pose(identity_pose(), timestamp=frame)
        front.add_image(
            uri=f"s3://my-bucket/scenes/scene-001/front/{frame:06d}.jpg",
            timestamp=frame,
        )

    return scene


# Create user client using ssh key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
    # For US platform users use "https://api.us.encord.com"
    domain="https://api.encord.com",
)

# Register the Scene. upload_scene waits for registration and returns the Scene item's UUID.
folder: StorageFolder = user_client.get_storage_folder(FOLDER_ID)
scene_uuid = folder.upload_scene(
    scene=build_scene_with_camera_stream(FRAME_COUNT),
    title="scene-001",
    integration_id=INTEGRATION_ID,
)
print(f"Registered Scene item {scene_uuid}")

Rotate Image Streams

If a camera is mounted sideways or upside down, set rotation on its image stream to display the images upright in the label editor. rotation is the clockwise rotation in degrees, and must be 0, 90, 180, or 270. The default is 0. You can set it with a separate camera stream, as in the example below, or with inline camera parameters. Image stream rotation requires encord 0.1.212 or later. Rotation only changes how the images are displayed. Keep the camera’s width, height, intrinsics, distortion, and pose in the original image coordinates, as the camera captured the images. Encord rotates the images and anything projected onto them together.

Rotate an Image Stream

# Import dependencies
from encord import EncordUserClient
from encord.beta.scene import SceneBuilder, identity_pose, intrinsics_pinhole, translation_only
from encord.storage import StorageFolder

# User input
SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"  # Replace with the file path to your SSH private key
FOLDER_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the unique ID for the folder in Files & Folders
INTEGRATION_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the ID of a cloud integration that can read every URI in the Scene
FRAME_COUNT = 3  # Replace with the number of frames in your sequence


def build_scene_with_rotated_camera(frame_count: int) -> SceneBuilder:
    scene = SceneBuilder()

    ego = scene.add_for_stream("ego")

    # A rear camera mounted upside down. Give its calibration in the original image coordinates,
    # exactly as the camera captured the images: width, height, and intrinsics are not rotated.
    rear_camera = scene.add_camera_stream(
        "rear_camera",
        frame_of_reference=ego,
        pose=translation_only(x=-1.0, y=0.0, z=1.5),
    )
    rear_camera.add_camera_params(
        1920,
        1080,
        intrinsics_pinhole(fx=1000, fy=1000, ox=960, oy=540),
        timestamp=0,
    )

    # Display the images rotated 180 degrees clockwise in the label editor.
    # rotation must be 0, 90, 180, or 270.
    rear = scene.add_image_stream("rear", camera="rear_camera", rotation=180)

    # Timestamps are whole-number frame indices that start at 0.
    for frame in range(frame_count):
        ego.add_pose(identity_pose(), timestamp=frame)
        rear.add_image(
            uri=f"s3://my-bucket/scenes/scene-001/rear/{frame:06d}.jpg",
            timestamp=frame,
        )

    return scene


# Create user client using ssh key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
    # For US platform users use "https://api.us.encord.com"
    domain="https://api.encord.com",
)

# Register the Scene. upload_scene waits for registration and returns the Scene item's UUID.
folder: StorageFolder = user_client.get_storage_folder(FOLDER_ID)
scene_uuid = folder.upload_scene(
    scene=build_scene_with_rotated_camera(FRAME_COUNT),
    title="scene-001",
    integration_id=INTEGRATION_ID,
)
print(f"Registered Scene item {scene_uuid}")

Image streams read with SceneReader have a rotation attribute, and SceneReader.to_upload_payload keeps the rotation when you copy a Scene.

Position Sensors with Poses

Every pose places a child frame inside its parent frame. The position is the child’s origin in parent coordinates, and the rotation turns the child’s axes into the parent’s axes, so a point p in the child frame is at R @ p + t in the parent frame. For a sensor, the pose is where it is mounted on its frame of reference and which way it points: its extrinsic calibration.

Link a point cloud or camera stream to a frame of reference with frame_of_reference=, and set a static mount pose with pose=. You can also call set_frame_of_reference() and set_pose() on the stream after you create it.
When a frame moves, such as an ego vehicle, add a pose event to its frame-of-reference stream for each frame.
Nest frames of reference with add_for_stream(name, parent_for_id=...). A frame of reference without a parent is attached to the Scene root, "root".
For an image stream linked to a camera stream with camera=, set the frame of reference and pose on the camera stream.

Build poses with these helpers from encord.beta.scene:

Helper	Rotation
`identity_pose()`	No rotation and no translation.
`translation_only(x, y, z)`	No rotation.
`rotation_only(rotation)`	A rotation such as `rotation_z(angle)`, with angles in radians. No translation.
`euler_pose(rx, ry, rz, x, y, z)`	Euler angles in radians, applied as extrinsic X-Y-Z rotations. Each angle must be between -2.1π and 2.1π.
`quaternion_pose(qx, qy, qz, qw, x, y, z)`	A unit quaternion, with the scalar component `qw` fourth.
`matrix_pose(rotation, x, y, z)`	A 3x3 rotation matrix as 9 values in column-major order.
`affine_transform(matrix)`	A 4x4 affine matrix as 16 values in column-major order. The bottom row must be `[0, 0, 0, 1]`.

Position Sensors with Poses

# Import dependencies
import math

from encord import EncordUserClient
from encord.beta.scene import (
    SceneBuilder,
    euler_pose,
    intrinsics_pinhole,
    matrix_pose,
    quaternion_pose,
    rotation_only,
    rotation_z,
    translation_only,
)
from encord.storage import StorageFolder

# User input
SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"  # Replace with the file path to your SSH private key
FOLDER_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the unique ID for the folder in Files & Folders
INTEGRATION_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the ID of a cloud integration that can read every URI in the Scene
FRAME_COUNT = 3  # Replace with the number of frames in your sequence


def build_scene_with_poses(frame_count: int) -> SceneBuilder:
    scene = SceneBuilder()

    # A pose places a child frame inside its parent frame. The position is the child's origin in
    # parent coordinates, and the rotation turns the child's axes into the parent's axes:
    # a point p in the child frame is at R @ p + t in the parent frame.

    # Moving platform, posed relative to the Scene root (the world frame).
    ego = scene.add_for_stream("ego")

    # Nested frame of reference: a sensor head that rotates relative to the ego frame.
    sensor_head = scene.add_for_stream("sensor_head", parent_for_id=ego)

    # Static LiDAR mount on the ego frame.
    # euler_pose(rx, ry, rz, x, y, z): angles in radians, applied as extrinsic X-Y-Z rotations.
    lidar = scene.add_pcd_stream(
        "lidar",
        frame_of_reference=ego,
        pose=euler_pose(0.0, 0.0, math.radians(1.5), x=0.9, y=0.0, z=1.8),
    )

    # Camera mounted on the rotating sensor head.
    # quaternion_pose(qx, qy, qz, qw, x, y, z): the quaternion must have unit length.
    # Some datasets, such as nuScenes, store quaternions as [w, x, y, z]. Reorder them first.
    scene.add_camera_stream(
        "front_camera",
        frame_of_reference=sensor_head,
        pose=quaternion_pose(0.0, 0.0, 0.0, 1.0, x=0.2, y=0.0, z=0.1),
    ).add_camera_params(
        1920,
        1080,
        intrinsics_pinhole(fx=1000, fy=1000, ox=960, oy=540),
        timestamp=0,
    )
    front = scene.add_image_stream("front", camera="front_camera")

    # You can also set the frame of reference and pose after creating a stream.
    # matrix_pose(rotation, x, y, z): a 3x3 rotation matrix as 9 values in column-major order.
    # This matrix rotates the rear LiDAR 180 degrees about the z-axis.
    rear_lidar = scene.add_pcd_stream("rear_lidar")
    rear_lidar.set_frame_of_reference(ego).set_pose(
        matrix_pose([-1.0, 0.0, 0.0, 0.0, -1.0, 0.0, 0.0, 0.0, 1.0], x=-1.0, y=0.0, z=1.6)
    )

    # Timestamps are whole-number frame indices that start at 0.
    for frame in range(frame_count):
        ego.add_pose(translation_only(x=frame * 0.5, y=0.0, z=0.0), timestamp=frame)
        sensor_head.add_pose(rotation_only(rotation_z(math.radians(frame * 10))), timestamp=frame)
        lidar.add_pcd(uri=f"s3://my-bucket/scenes/scene-001/lidar/{frame:06d}.pcd", timestamp=frame)
        rear_lidar.add_pcd(uri=f"s3://my-bucket/scenes/scene-001/rear_lidar/{frame:06d}.pcd", timestamp=frame)
        front.add_image(uri=f"s3://my-bucket/scenes/scene-001/front/{frame:06d}.jpg", timestamp=frame)

    return scene


# Create user client using ssh key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
    # For US platform users use "https://api.us.encord.com"
    domain="https://api.encord.com",
)

# Register the Scene. upload_scene waits for registration and returns the Scene item's UUID.
folder: StorageFolder = user_client.get_storage_folder(FOLDER_ID)
scene_uuid = folder.upload_scene(
    scene=build_scene_with_poses(FRAME_COUNT),
    title="scene-001",
    integration_id=INTEGRATION_ID,
)
print(f"Registered Scene item {scene_uuid}")

Coordinate Conventions

Set coordinate conventions in the SDK when your data uses known world and camera axes. For example, driving datasets often use a world convention such as forward, left, up and camera axes such as right, down, forward. Use SceneBuilder.set_world_convention and SceneBuilder.set_camera_convention with Direction values. The world and camera conventions must have matching handedness.

Add Time Series and Configure a Layout

Add a CSV time series stream with SceneBuilder.add_time_series_stream(name, uri=...). A time series stream is a single file and has no events. See Time Series Column Structure for the CSV format. Set SceneBuilder.layout to a SceneLayout to choose the tiles the label editor shows and how they are arranged:

tiles maps tile IDs to Scene3DViewerTile, SceneImageTile, or SceneTimeSeriesTile objects. Image and time series tiles reference streams by stream name.
layout arranges the tiles with nested SceneTileLayout splits. SceneTileLayoutDirection.ROW places the two panes side by side, and SceneTileLayoutDirection.COLUMN stacks them. split_percentage sets the first pane’s share of the space, from 5 to 95.
timeline lists the time series tile IDs to show in the timeline.

Every tile must be used in layout or timeline, and a layout can contain at most one 3D viewer tile. To change the layout of a registered Scene, call StorageItem.update(scene_layout=...). Pass scene_layout=None to remove the layout. For MCAP recordings, see Configure an MCAP layout.

Time Series Stream and Scene Layout

# Import dependencies
from encord import EncordUserClient
from encord.beta.scene import (
    Scene3DViewerTile,
    SceneBuilder,
    SceneImageTile,
    SceneLayout,
    SceneTileLayout,
    SceneTileLayoutDirection,
    SceneTimeSeriesTile,
    identity_pose,
    intrinsics_pinhole,
)
from encord.storage import StorageFolder

# User input
SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"  # Replace with the file path to your SSH private key
FOLDER_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the unique ID for the folder in Files & Folders
INTEGRATION_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the ID of a cloud integration that can read every URI in the Scene
FRAME_COUNT = 3  # Replace with the number of frames in your sequence


def build_scene_with_telemetry(frame_count: int) -> SceneBuilder:
    scene = SceneBuilder()

    ego = scene.add_for_stream("ego")
    lidar = scene.add_pcd_stream("lidar", frame_of_reference=ego)
    front = scene.add_image_stream(
        "front",
        width=1920,
        height=1080,
        intrinsics=intrinsics_pinhole(fx=1000, fy=1000, ox=960, oy=540),
        timestamp=0,
        frame_of_reference=ego,
    )

    # Timestamps are whole-number frame indices that start at 0.
    for frame in range(frame_count):
        ego.add_pose(identity_pose(), timestamp=frame)
        lidar.add_pcd(uri=f"s3://my-bucket/scenes/scene-001/lidar/{frame:06d}.pcd", timestamp=frame)
        front.add_image(uri=f"s3://my-bucket/scenes/scene-001/front/{frame:06d}.jpg", timestamp=frame)

    # A time series stream is a single CSV file. It has no events: its timestamp column and
    # channels come from the CSV itself.
    scene.add_time_series_stream("telemetry", uri="s3://my-bucket/scenes/scene-001/telemetry.csv")

    # Tiles reference streams by stream name. Every tile must appear in the layout or the timeline,
    # and a layout can have at most one 3D viewer tile.
    scene.layout = SceneLayout(
        tiles={
            "3d": Scene3DViewerTile(has_side_view=False, show_camera_switcher=True),
            "front": SceneImageTile(stream_name="front"),
            "telemetry": SceneTimeSeriesTile(stream_name="telemetry"),
        },
        # ROW splits the space side by side: the 3D view gets 60%, the front camera the rest.
        layout=SceneTileLayout(
            direction=SceneTileLayoutDirection.ROW,
            first="3d",
            second="front",
            split_percentage=60,
        ),
        # Show the telemetry chart in the timeline below the views.
        timeline=["telemetry"],
    )

    return scene


# Create user client using ssh key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
    # For US platform users use "https://api.us.encord.com"
    domain="https://api.encord.com",
)

# Register the Scene. upload_scene validates that every tile references a stream of the right type.
folder: StorageFolder = user_client.get_storage_folder(FOLDER_ID)
scene_uuid = folder.upload_scene(
    scene=build_scene_with_telemetry(FRAME_COUNT),
    title="scene-001",
    integration_id=INTEGRATION_ID,
)
print(f"Registered Scene item {scene_uuid}")

# To change the layout of a registered Scene later:
# user_client.get_storage_item(scene_uuid).update(scene_layout=new_layout)
# Pass scene_layout=None to remove the layout.

Scene View Settings

SceneViewSettings holds per-Scene rendering settings that the label editor applies when the Scene is opened from the Editor or Explorer view. Unset fields fall through to the annotator’s own preferences.

point_cloud_colouring — one of SceneHeightColouring, SceneProvidedColouring, SceneSolidColouring, SceneImageColouring, or SceneSensorColouring. SceneHeightColouring takes color_height_bounds, the minimum and maximum heights used for the color gradient.
point_radius — radius used to render point cloud points, from 0 to 100.
render_image_outside_base_range — Render the full image captured by the camera. Useful for wide-angle and fisheye lenses.
radius_indicators — a list of SceneRadiusIndicator, each with the frame_of_reference_id at its center, a radius in scene units, and a hex color.
radius_filter_enabled — whether point clouds are filtered to the configured radius indicators.

Assign the settings to SceneBuilder.settings before upload, patch a registered Scene with StorageItem.update(scene_view_settings=...), or read them from SceneReader.view_settings. Pass scene_view_settings=None to restore the defaults:

Configure Scene View Settings

from uuid import UUID

from encord import EncordUserClient
from encord.beta.scene import (
    SceneBuilder,
    SceneHeightColouring,
    SceneProvidedColouring,
    SceneRadiusIndicator,
    SceneReader,
    SceneViewSettings,
    identity_pose,
)
from encord.orm.dataset import LongPollingStatus
from encord.orm.storage import DataUploadItems
from encord.storage import StorageFolder, StorageItem


# --- Configuration ---
SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"  # Replace with the file path to your SSH private key
FOLDER_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the unique ID for the folder in Files & Folders
TARGET_FOLDER_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the unique ID for the folder to copy the Scene into
INTEGRATION_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the ID of a cloud integration that can read every URI in the Scene


def register_scene_with_view_settings(
    storage_folder: StorageFolder,
    integration_id: str,
) -> UUID:
    """Create and upload a Scene with initial view settings."""

    scene = SceneBuilder()

    ego = scene.add_for_stream("ego")
    ego.add_pose(identity_pose(), timestamp=0)

    scene.add_pcd_stream(
        "lidar",
        frame_of_reference=ego,
    ).add_pcd(
        uri="s3://my-bucket/scenes/scene-001/lidar/000000.pcd",
        timestamp=0,
    )

    # Every radius indicator must name "root" (the world frame) or a frame-of-reference stream in this Scene.
    scene.settings = SceneViewSettings(
        point_cloud_colouring=SceneHeightColouring(
            color_height_bounds=(-1.5, 4.0),
        ),
        point_radius=2.0,
        radius_filter_enabled=True,
        radius_indicators=[
            SceneRadiusIndicator(
                frame_of_reference_id="ego",
                radius=3,
                color="#FF8A00",
            ),
            SceneRadiusIndicator(
                frame_of_reference_id="root",
                radius=4,
                color="#3366FF80",
            ),
        ],
    )

    return storage_folder.upload_scene(
        scene=scene,
        title="scene-001",
        integration_id=integration_id,
    )


def update_scene_view_settings(scene_item: StorageItem) -> None:
    """Update the persisted view settings of an existing Scene."""

    reader = SceneReader(scene_item)

    current_settings = reader.view_settings or SceneViewSettings()

    current_settings.point_radius = 5.0

    # Pass scene_view_settings=None to restore the defaults. Omit it to leave the settings unchanged.
    scene_item.update(
        scene_view_settings=current_settings,
    )


def copy_scene_with_new_view_settings(
    scene_item: StorageItem,
    target_storage_folder: StorageFolder,
    *,
    integration_id: str,
    title: str,
) -> UUID:
    """Copy a Scene with different view settings and return the new Scene item's UUID."""

    reader = SceneReader(scene_item)

    new_settings = reader.view_settings or SceneViewSettings()

    new_settings.point_cloud_colouring = SceneProvidedColouring(
        display_point_intensity=False,
        srgb_colors=True,
    )

    reader.view_settings = new_settings

    # to_upload_payload() includes the Scene's view settings.
    upload_payload = reader.to_upload_payload(title=title)

    upload_job_id = target_storage_folder.add_private_data_to_folder_start(
        integration_id=integration_id,
        private_files=DataUploadItems(
            scenes=[upload_payload],
        ),
    )

    # add_private_data_to_folder_start() returns the upload job's ID, not the Scene's.
    # Wait for the job to finish, then return the new Scene item's UUID.
    result = target_storage_folder.add_private_data_to_folder_get_result(upload_job_id)
    if result.status != LongPollingStatus.DONE:
        raise RuntimeError(f"Scene copy did not finish ({result.status}): {result.errors}")
    return result.items_with_names[0].item_uuid


# --- Connect to Encord ---

user_client: EncordUserClient = (
    EncordUserClient.create_with_ssh_private_key(
        ssh_private_key_path=SSH_PATH,
        # For US platform users, use:
        # domain="https://api.us.encord.com",
        domain="https://api.encord.com",
    )
)

folder = user_client.get_storage_folder(FOLDER_ID)
target_folder = user_client.get_storage_folder(TARGET_FOLDER_ID)


# --- Create a Scene with view settings ---

scene_uuid = register_scene_with_view_settings(
    folder,
    INTEGRATION_ID,
)

scene_item = user_client.get_storage_item(scene_uuid)


# --- Read the Scene's view settings ---

reader = SceneReader(scene_item)
print(f"Initial view settings: {reader.view_settings}")


# --- Update the stored Scene's view settings ---

update_scene_view_settings(scene_item)

# A SceneReader caches the Scene the first time it reads it. Create a new one to see the update.
reader = SceneReader(scene_item)
print(f"Updated view settings: {reader.view_settings}")


# --- Copy the Scene with different view settings ---

copy_uuid = copy_scene_with_new_view_settings(
    scene_item,
    target_folder,
    integration_id=INTEGRATION_ID,
    title="scene-001-copy",
)

print(f"Copied Scene item {copy_uuid}")

frame_of_reference_id must be "root" for the world frame, or the name of a frame-of-reference stream in the same Scene.

Read and Copy Registered Scenes

SceneReader fetches a registered Scene and returns signed URLs for its constituent files. It can also convert a registered Scene back into a DataUploadScene for migration or copying. SceneReader.read() supports composite Scenes only. It raises a ValueError for single-file Scenes, such as MCAP recordings. SceneReader.to_upload_payload() works for both composite and single-file Scenes.

Read and Copy a Scene

# Import dependencies
from uuid import UUID

from encord import EncordUserClient
from encord.beta.scene import SceneReader
from encord.orm.dataset import LongPollingStatus
from encord.orm.storage import DataUploadItems, StorageItemType
from encord.storage import StorageFolder, StorageItem

# User input
SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"  # Replace with the file path to your SSH private key
FOLDER_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the unique ID for the folder that contains the Scene
TARGET_FOLDER_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the unique ID for the folder to copy the Scene into
INTEGRATION_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the ID of a cloud integration that can read the copied URIs
SCENE_TITLE = "scene-001"  # Replace with the exact title of the Scene
SOURCE_URI_PREFIX = "s3://my-bucket/"  # Replace with the URI prefix of the original files
TARGET_URI_PREFIX = "s3://my-other-bucket/"  # Replace with the URI prefix of the copied files


def find_scene(storage_folder: StorageFolder, title: str) -> StorageItem:
    # search= matches partial titles, so check for an exact match.
    for item in storage_folder.list_items(search=title, item_types=[StorageItemType.SCENE]):
        if item.name == title:
            return item
    raise ValueError(f"Scene with title '{title}' not found.")


def get_lidar_signed_url(scene_item: StorageItem, timestamp: int = 0) -> str:
    # read() supports composite Scenes only. It raises ValueError for single-file Scenes, such as MCAP recordings.
    scene = SceneReader(scene_item).read()
    lidar = scene.get_stream("lidar", kind="point_cloud")
    # get_event() matches the timestamp exactly and raises KeyError when no event has it.
    # Use find_event() to get None instead.
    return lidar.get_event(timestamp=timestamp).signed_url


def get_image_signed_urls(scene_item: StorageItem, timestamp: int = 0) -> list[tuple[str, str]]:
    scene = SceneReader(scene_item).read()
    # Returns (stream name, signed URL) pairs. Image streams with no event at this timestamp are skipped.
    return [(stream_id, event.signed_url) for stream_id, event in scene.get_images_at_timestamp(timestamp)]


def copy_scene_to_folder(
    scene_item: StorageItem,
    target_storage_folder: StorageFolder,
    *,
    integration_id: str,
    title: str,
    source_uri_prefix: str,
    target_uri_prefix: str,
) -> UUID:
    def move_uri(uri: str) -> str:
        if uri.startswith(source_uri_prefix):
            return target_uri_prefix + uri[len(source_uri_prefix) :]
        return uri

    # to_upload_payload() works for composite and single-file Scenes, and keeps view settings and layout.
    # uri_mapper also accepts a dict of {old URI: new URI}. URIs missing from the dict are kept unchanged.
    payload = SceneReader(scene_item).to_upload_payload(title=title, uri_mapper=move_uri)

    upload_job_id = target_storage_folder.add_private_data_to_folder_start(
        integration_id=integration_id,
        private_files=DataUploadItems(scenes=[payload]),
    )

    # add_private_data_to_folder_start() returns the upload job's ID, not the Scene's.
    # Wait for the job to finish, then return the new Scene item's UUID.
    result = target_storage_folder.add_private_data_to_folder_get_result(upload_job_id)
    if result.status != LongPollingStatus.DONE:
        raise RuntimeError(f"Scene copy did not finish ({result.status}): {result.errors}")
    return result.items_with_names[0].item_uuid


# Create user client using ssh key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
    # For US platform users use "https://api.us.encord.com"
    domain="https://api.encord.com",
)

folder: StorageFolder = user_client.get_storage_folder(FOLDER_ID)
target_folder: StorageFolder = user_client.get_storage_folder(TARGET_FOLDER_ID)

scene_item = find_scene(folder, SCENE_TITLE)
print(f"LiDAR at frame 0: {get_lidar_signed_url(scene_item)}")
for stream_id, url in get_image_signed_urls(scene_item):
    print(f"{stream_id} at frame 0: {url}")

copy_uuid = copy_scene_to_folder(
    scene_item,
    target_folder,
    integration_id=INTEGRATION_ID,
    title=f"{SCENE_TITLE}-copy",
    source_uri_prefix=SOURCE_URI_PREFIX,
    target_uri_prefix=TARGET_URI_PREFIX,
)
print(f"Copied Scene item {copy_uuid}")

Use find_stream and find_event when a missing stream or timestamp is acceptable: they return None. Use get_stream and get_event when missing data should fail fast: they raise KeyError. find_event and get_event only match an event whose timestamp is exactly the one you pass.

SceneReader fetches the Scene the first time you read it and caches it. Create a new SceneReader to see changes made after that.
A time series stream has no events. find_stream(name, kind="time_series") returns its CSV url and signed_url.
to_upload_payload(uri_mapper=...) takes a function or a dict of {old URI: new URI}. URIs that are not in the dict are kept unchanged. The payload keeps the Scene’s view settings and layout.
add_private_data_to_folder_start returns an upload job ID. Pass it to add_private_data_to_folder_get_result to wait for the copy and get the new Scene item’s UUID.

Read Frame of Reference Streams

A Scene’s coordinate frames are exposed as frame-of-reference streams. Read them with get_stream(..., kind="frame_of_reference") (or find_stream) to inspect each frame’s pose over time. Every point-cloud and image stream also reports the frame it is expressed in through its frame_of_reference_id.

Read a Frame of Reference Stream

# Import dependencies
from encord import EncordUserClient
from encord.beta.scene import FrameOfReferenceEvent, SceneReader
from encord.storage import StorageItem

# User input
SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"  # Replace with the file path to your SSH private key
SCENE_ITEM_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the unique ID for a composite Scene item


def print_ego_trajectory(scene_item: StorageItem) -> None:
    """Read the poses of the 'ego' frame-of-reference stream over time."""

    # Requires SDK v0.1.205 or later. read() supports composite Scenes only.
    scene = SceneReader(scene_item).read()

    # Use find_stream() when a missing stream is acceptable (returns None),
    # or get_stream() when it should fail fast (raises KeyError).
    ego = scene.find_stream("ego", kind="frame_of_reference")
    if ego is None:
        print("No 'ego' frame-of-reference stream in this Scene.")
        return

    print(f"Stream '{ego.stream_id}' has {ego.num_events} pose(s).")

    for event in ego.events:
        # Each event is a FrameOfReferenceEvent: a pose relative to its parent frame.
        print(f"  t={event.timestamp}")
        print(f"    frame_id:        {event.frame_id}")
        print(f"    parent_frame_id: {event.parent_frame_id}")  # None when no parent is declared
        print(f"    position:        {event.position}")  # (x, y, z) translation to the parent frame
        print(f"    rotation:        {event.rotation}")  # 3x3 column-major matrix, as a 9-tuple


def get_pose_at(scene_item: StorageItem, stream_id: str, timestamp: int) -> FrameOfReferenceEvent:
    """Return the pose of a named frame-of-reference stream at an exact timestamp."""

    scene = SceneReader(scene_item).read()

    # get_stream() and get_event() raise KeyError if the stream or timestamp is missing.
    stream = scene.get_stream(stream_id, kind="frame_of_reference")
    return stream.get_event(timestamp=timestamp)


def print_stream_frames(scene_item: StorageItem) -> None:
    """Print the frame of reference each point cloud and image stream is expressed in."""

    scene = SceneReader(scene_item).read()

    # frame_of_reference_id is None when the stream has no frame of reference.
    for lidar in scene.point_cloud_streams:
        print(f"Point cloud '{lidar.stream_id}' is expressed in frame: {lidar.frame_of_reference_id}")
    for image in scene.image_streams:
        print(f"Image '{image.stream_id}' camera frame: {image.frame_of_reference_id}")


# Create user client using ssh key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
    # For US platform users use "https://api.us.encord.com"
    domain="https://api.encord.com",
)

scene_item: StorageItem = user_client.get_storage_item(SCENE_ITEM_ID)
print_ego_trajectory(scene_item)
print(get_pose_at(scene_item, "ego", timestamp=0))
print_stream_frames(scene_item)

Each FrameOfReferenceEvent gives the frame’s position (translation to its parent frame) and rotation (a 3x3 column-major matrix as a 9-tuple) at a timestamp, along with its frame_id and parent_frame_id.

Camera Distortion Models

The supported distortion models are defined in encord.beta.scene.intrinsics.

SDK API	Model type	Distortion coefficients	Use when
`intrinsics_pinhole(...)`	`pinhole`	None	Use for calibrated pinhole cameras with no distortion.
`intrinsics_radial(...)`	`radial`	`k1`, `k2`, `k3`	Use when calibration only supplies radial coefficients.
`intrinsics_plumb_bob(...)`	`plumb_bob`	`k1`, `k2`, `k3`, `t1`, `t2`	Use for Brown-Conrady calibration with radial and tangential coefficients.
`intrinsics_fisheye(...)`	`fisheye`	`k1`, `k2`, `k3`, `k4`	Use for fisheye lenses calibrated with four coefficients.
`intrinsics_rational_polynomial(...)`	`rational_polynomial`	`k1`, `k2`, `k3`, `k4`, `k5`, `k6`, `t1`, `t2`	Use for OpenCV rational-polynomial calibration.
`intrinsics_simple(..., model="division")`	`division`	`k`	Use for one-parameter division-model calibration.
`intrinsics_simple(..., model="ucm")`	`ucm`	`xi`, `k1`, `k2`, `k3`	Use for unified camera model calibration.
`intrinsics_cylindrical(...)`	`cylindrical`	None	Use for cylindrical camera projections.

Camera Distortion Models

# Import dependencies
from encord.beta.scene import (
    SceneBuilder,
    SimpleIntrinsics,
    intrinsics_cylindrical,
    intrinsics_fisheye,
    intrinsics_pinhole,
    intrinsics_plumb_bob,
    intrinsics_radial,
    intrinsics_rational_polynomial,
    intrinsics_simple,
)


def build_distortion_model_examples() -> dict[str, SimpleIntrinsics]:
    # fx, fy: focal lengths in pixels. ox, oy: principal point in pixels.
    # Distortion coefficients are keyword-only. Replace the values with your calibration.
    return {
        "pinhole": intrinsics_pinhole(fx=1000, fy=1000, ox=960, oy=540),
        "radial": intrinsics_radial(fx=1000, fy=1000, ox=960, oy=540, k1=0.1, k2=-0.01, k3=0.001),
        "plumb_bob": intrinsics_plumb_bob(
            fx=1000,
            fy=1000,
            ox=960,
            oy=540,
            k1=0.1,
            k2=-0.01,
            k3=0.001,
            t1=0.0001,
            t2=-0.0001,
        ),
        "fisheye": intrinsics_fisheye(
            fx=1000,
            fy=1000,
            ox=960,
            oy=540,
            k1=0.1,
            k2=-0.01,
            k3=0.001,
            k4=-0.0001,
        ),
        "rational_polynomial": intrinsics_rational_polynomial(
            fx=1000,
            fy=1000,
            ox=960,
            oy=540,
            k1=0.1,
            k2=-0.01,
            k3=0.001,
            k4=-0.0001,
            k5=0.00001,
            k6=-0.000001,
            t1=0.0001,
            t2=-0.0001,
        ),
        # Models without a dedicated constructor use intrinsics_simple with the model name.
        # Check the spelling: the SDK drops an unrecognized model name without raising an error.
        "division": intrinsics_simple(fx=1000, fy=1000, ox=960, oy=540, model="division", k=0.01),
        "ucm": intrinsics_simple(fx=1000, fy=1000, ox=960, oy=540, model="ucm", xi=0.5, k1=0.1, k2=0, k3=0),
        "cylindrical": intrinsics_cylindrical(fx=1000, fy=1000, ox=960, oy=540),
    }


# Use any of these as the intrinsics of an image stream or a camera stream.
scene = SceneBuilder()
scene.add_image_stream(
    "front",
    width=1920,
    height=1080,
    intrinsics=build_distortion_model_examples()["fisheye"],
    timestamp=0,
).add_image(uri="s3://my-bucket/scenes/scene-001/front/000000.jpg", timestamp=0)

For most workflows, use intrinsics_pinhole or a dedicated distortion constructor. Use intrinsics_advanced only when you already have full calibration matrices and need to supply them directly.

Validation Checklist

Before handing Scenes to annotators:

Confirm every point cloud and image URI is accessible by the integration used for registration.
Keep stream names stable and descriptive, such as lidar, front, rear, or ego.
Use whole-number timestamps that start at 0. Fractional timestamps fail registration, and epoch timestamps stop the Scene opening in the editor.
Confirm every SceneRadiusIndicator names "root" or a frame-of-reference stream the Scene defines. Upload fails otherwise.
Check that quaternions are in (qx, qy, qz, qw) order and Euler angles are in radians.
Confirm every layout tile references an existing stream and is used in layout or timeline.
Open at least one registered Scene in the editor to verify orientation, calibration, and frame navigation.

Create and Register Scenes

Learn the Scene data model and registration choices.

Get Started

General

Data

Ontologies

Projects

Labels

DICOM

Scene SDK Workflows

SDK Version Requirements

Upload an MCAP Recording

Build and Upload a Composite Scene

Add Multi-frame Sensor Streams

Define Camera Streams Separately

Rotate Image Streams

Position Sensors with Poses

Coordinate Conventions

Add Time Series and Configure a Layout

Scene View Settings

Read and Copy Registered Scenes

Read Frame of Reference Streams

Camera Distortion Models

Validation Checklist

Create and Register Scenes

​SDK Version Requirements

​Upload an MCAP Recording

​Build and Upload a Composite Scene

​Add Multi-frame Sensor Streams

​Define Camera Streams Separately

​Rotate Image Streams

​Position Sensors with Poses

​Coordinate Conventions

​Add Time Series and Configure a Layout

​Scene View Settings

​Read and Copy Registered Scenes

​Read Frame of Reference Streams

​Camera Distortion Models

​Validation Checklist

Create and Register Scenes

SDK Version Requirements

Upload an MCAP Recording

Build and Upload a Composite Scene

Add Multi-frame Sensor Streams

Define Camera Streams Separately

Rotate Image Streams

Position Sensors with Poses

Coordinate Conventions

Add Time Series and Configure a Layout

Scene View Settings

Read and Copy Registered Scenes

Read Frame of Reference Streams

Camera Distortion Models

Validation Checklist