Create and Register Scenes

A Scene is the Encord data unit that contains 3D sensor data. It can contain point clouds, such as LiDAR and radar point clouds, camera images, camera calibration, and vehicle or robot poses. Annotators open Scenes as tasks for 3D labeling and review. Scenes can be created in Encord by either:

Referencing a single .mcap file
Creating a Composite Scene to reference existing files in your cloud bucket.

For the supported point cloud and container formats, see Supported data formats.

Upload an `.mcap` file directly

If your sensor data is already in an .mcap recording, you can upload the file directly. Upload the file in Encord Files & Folders, or use the SDK to upload a local file or register a file from cloud storage:

# Import dependencies
from encord import EncordUserClient
from encord.storage import StorageFolder

# User input
SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"  # Replace with the file path to your SSH private key
FOLDER_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the unique ID for the folder in Files & Folders
MCAP_FILE_PATH = "/Users/chris-encord/recording.mcap"  # Replace with the file path to your MCAP recording
SCENE_TITLE = "Robot recording"  # Replace with the title for the Scene

# Create user client using ssh key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
    # For US platform users use "https://api.us.encord.com"
    domain="https://api.encord.com",
)

# Get the folder
folder: StorageFolder = user_client.get_storage_folder(FOLDER_ID)

# Upload the MCAP recording as a Scene
scene_uuid = folder.upload_scene_file(file_path=MCAP_FILE_PATH, title=SCENE_TITLE)
print(f"Uploaded Scene item {scene_uuid}")

# Import dependencies
from encord import EncordUserClient
from encord.orm.storage import DataUploadItems, DataUploadScene
from encord.storage import StorageFolder

# User input
SSH_PATH = "/Users/chris-encord/ssh-private-key.txt"  # Replace with the file path to your SSH private key
FOLDER_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the unique ID for the folder in Files & Folders
INTEGRATION_ID = "00000000-0000-0000-0000-000000000000"  # Replace with the unique ID for the cloud storage integration
MCAP_URI = "s3://my-bucket/recording.mcap"  # Replace with the URI of the MCAP recording in your cloud storage
SCENE_TITLE = "Robot recording"  # Replace with the title for the Scene

# Create user client using ssh key
user_client: EncordUserClient = EncordUserClient.create_with_ssh_private_key(
    ssh_private_key_path=SSH_PATH,
    # For US platform users use "https://api.us.encord.com"
    domain="https://api.encord.com",
)

# Get the folder
folder: StorageFolder = user_client.get_storage_folder(FOLDER_ID)

# Register the MCAP recording as a Scene
upload_job_id = folder.add_private_data_to_folder_start(
    integration_id=INTEGRATION_ID,
    private_files=DataUploadItems(
        scenes=[
            DataUploadScene(
                title=SCENE_TITLE,
                scene={"url": MCAP_URI, "format": "mcap"},
            ),
        ],
    ),
)

# Wait for the registration to finish
result = folder.add_private_data_to_folder_get_result(upload_job_id)
print(result)

See MCAP Support for supported schemas, size guidance, custom layouts, and label import/export.

Register a Composite Scene with the `SceneBuilder`

Use the SceneBuilder when your sensor streams live in separate files that you want to combine into a composite Scene. See Scene SDK workflow for the full documentation on the SceneBuilder

Create or load a StorageFolder and cloud integration ID with the Python SDK, then use the SceneBuilder to build and upload the Scene. For method-level details, see the SceneBuilder reference.

Build and Upload a Scene

from uuid import UUID

from encord.beta.scene import SceneBuilder, identity_pose, intrinsics_pinhole
from encord.storage import StorageFolder


def build_scene() -> SceneBuilder:
    scene = SceneBuilder()

    # Anchor the sensor streams to the moving platform.
    ego = scene.add_for_stream("ego")
    ego.add_pose(identity_pose(), timestamp=0)

    # Matching timestamps make the point cloud and image appear together.
    scene.add_pcd_stream("lidar", frame_of_reference=ego).add_pcd(
        uri="s3://my-bucket/scenes/scene-001/lidar/000000.pcd",
        timestamp=0,
    )

    # Inline camera parameters create a camera stream named "front/camera".
    scene.add_image_stream(
        "front",
        width=1920,
        height=1080,
        intrinsics=intrinsics_pinhole(fx=1000, fy=1000, ox=960, oy=540),
        timestamp=0,
        frame_of_reference=ego,
    ).add_image(
        uri="s3://my-bucket/scenes/scene-001/front/000000.jpg",
        timestamp=0,
    )

    return scene


def register_scene(storage_folder: StorageFolder, integration_id: str) -> UUID:
    scene = build_scene()

    # Pass the SceneBuilder itself. upload_scene validates and serializes it internally.
    return storage_folder.upload_scene(
        scene=scene,
        title="scene-001",
        integration_id=integration_id,
        client_metadata={"source": "sample"},
    )

After registration, add the Scene item to a Dataset and create an annotation Project as you would for other data types.

Overview

LiDAR, Radar and Sensor Data

Time Series Data

Visual Data

Audio Data

Text, HTML, and PDFs

DICOM and NIFTI

Multimodal

Geospatial Data

Create and Register Scenes

Upload an `.mcap` file directly

Register a Composite Scene with the `SceneBuilder`

​Upload an .mcap file directly

​Register a Composite Scene with the SceneBuilder

Upload an `.mcap` file directly

Register a Composite Scene with the `SceneBuilder`