Skip to main content
PCD Projects use “Scenes”. Scenes are the data units that your Taskers, and possibly your Agents, work with to create and review your labels. A Scene can be one of the following:
  • A PCD file (.pcd, .ply, .las, .laz, and so on)
  • A group of videos and PCD files bound together as a coherent group.
Encord supports the following for multimodal Scenes:
  • Videos up to 1GB in size
  • Up to 9 videos in a Scene
  • Up to 100 frames in each video
  • Up to 10 million cloud data points per frame
The Encord platform supports many more Point Cloud Data file formats. If you do not see a format you want supported, contact us at [email protected].

​
Scene Formats

Encord supports various ways of importing/registering Scenes. All examples use the InputScene format as the root structure.

​
URL to PCD File

A Scene consisting of a single PCD file, in cloud storage: https://example.com/left_001.pcd This format automatically validates that the URL has a supported file extension.

​
Stream of PCD Files without Timestamps

This format consists of multiple PCD files organized into a stream structure. For multiple PCD files without timing information, Encord assigns implicit timestamps of 1, 2, 3. At every time point T, Encord displays the last PCD available at or before T.
Stream without timestamps
{
  "left_lidar": {
    "type": "point_cloud",
    "events": [
      {
        "uri": "https://example.com/left_001.pcd"
      },
      {
        "uri": "https://example.com/left_002.pcd"
      }
    ]
  }
}

​
Stream of PCD Files with Timestamps

The main difference between datasets that have timestamps and that do not is that Encord treats items in “frames” or sees time as continuous. If temporal information is missing from the PCD stream you can add the timestamp:
Stream with timestamps
{
  "left_lidar": {
    "type": "point_cloud",
    "events": [
      {
        "uri": "https://example.com/left_001.pcd",
        "timestamp": 1634567890.123
      },
      {
        "uri": "https://example.com/left_002.pcd",
        "timestamp": 1634567890.223
      }
    ]
  }
}
A number of timestamp formats are supported (see Timestamp Formats).

​
Multiple PCD Streams (Left and Right Sensors)

Two synchronized PCD streams from different sensors:
Multiple streams
{
  "left_lidar": {
    "type": "point_cloud",
    "events": [
      {
        "uri": "https://example.com/left_001.pcd",
        "timestamp": 1634567890.123
      },
      {
        "uri": "https://example.com/left_002.pcd",
        "timestamp": 1634567890.223
      }
    ]
  },
  "right_lidar": {
    "type": "point_cloud",
    "events": [
      {
        "uri": "https://example.com/right_001.pcd",
        "timestamp": 1634567890.125
      },
      {
        "uri": "https://example.com/right_002.pcd",
        "timestamp": 1634567890.225
      }
    ]
  }
}

​
Add Frame of Reference for Ego Vehicle

Add a frame of reference stream to represent an ego vehicle’s pose over time:
Frame of reference
{
  "ego_vehicle": {
    "type": "frame_of_reference",
    "id": "ego_vehicle",
    "parentForId": null,
    "events": [
      {
        "timestamp": 1634567890.123,
        "pose": {
          "position": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0
          },
          "rotation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0,
            "w": 1.0
          }
        }
      },
      {
        "timestamp": 1634567890.223,
        "pose": {
          "position": {
            "x": 1.5,
            "y": 0.2,
            "z": 0.0
          },
          "rotation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.1,
            "w": 0.995
          }
        }
      }
    ]
  },
  "left_lidar": {
    "type": "point_cloud",
    "frameOfReference": "ego_vehicle",
    "events": [
      {
        "uri": "https://example.com/left_001.pcd",
        "timestamp": 1634567890.123
      },
      {
        "uri": "https://example.com/left_002.pcd",
        "timestamp": 1634567890.223
      }
    ]
  },
  "right_lidar": {
    "type": "point_cloud",
    "frameOfReference": "ego_vehicle",
    "events": [
      {
        "uri": "https://example.com/right_001.pcd",
        "timestamp": 1634567890.125
      },
      {
        "uri": "https://example.com/right_002.pcd",
        "timestamp": 1634567890.225
      }
    ]
  }
}

​
Sensor-Specific Frames of Reference with Calibration

Create individual frames of reference for each sensor, with calibration relative to the ego vehicle:
Sensor-Specific Frames of Reference with Calibration
{
  "ego_vehicle": {
    "type": "frame_of_reference",
    "id": "ego_vehicle",
    "parentForId": null,
    "events": [
      {
        "timestamp": 1634567890.123,
        "pose": {
          "position": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0
          },
          "rotation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0,
            "w": 1.0
          }
        }
      },
      {
        "timestamp": 1634567890.223,
        "pose": {
          "position": {
            "x": 1.5,
            "y": 0.2,
            "z": 0.0
          },
          "rotation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.1,
            "w": 0.995
          }
        }
      }
    ]
  },
  "left_lidar_frame": {
    "type": "frame_of_reference",
    "id": "left_lidar_frame",
    "parentForId": "ego_vehicle",
    "events": [
      {
        "timestamp": 1634567890.123,
        "pose": {
          "position": {
            "x": 0.0,
            "y": 0.5,
            "z": 1.8
          },
          "rotation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0,
            "w": 1.0
          }
        }
      }
    ]
  },
  "right_lidar_frame": {
    "type": "frame_of_reference",
    "id": "right_lidar_frame",
    "parentForId": "ego_vehicle",
    "events": [
      {
        "timestamp": 1634567890.123,
        "pose": {
          "position": {
            "x": 0.0,
            "y": -0.5,
            "z": 1.8
          },
          "rotation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0,
            "w": 1.0
          }
        }
      }
    ]
  },
  "left_lidar": {
    "type": "point_cloud",
    "frameOfReference": "left_lidar_frame",
    "events": [
      {
        "uri": "https://example.com/left_001.pcd",
        "timestamp": 1634567890.123
      },
      {
        "uri": "https://example.com/left_002.pcd",
        "timestamp": 1634567890.223
      }
    ]
  },
  "right_lidar": {
    "type": "point_cloud",
    "frameOfReference": "right_lidar_frame",
    "events": [
      {
        "uri": "https://example.com/right_001.pcd",
        "timestamp": 1634567890.125
      },
      {
        "uri": "https://example.com/right_002.pcd",
        "timestamp": 1634567890.225
      }
    ]
  }
}

​
Image Stream without Camera Parameters

Add an image stream using an existing camera reference:
Image stream without camera parameters
{
  "ego_vehicle": {
    "type": "frame_of_reference",
    "id": "ego_vehicle",
    "parentForId": null,
    "events": [
      {
        "timestamp": 1634567890.123,
        "pose": {
          "position": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0
          },
          "rotation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0,
            "w": 1.0
          }
        }
      },
      {
        "timestamp": 1634567890.223,
        "pose": {
          "position": {
            "x": 1.5,
            "y": 0.2,
            "z": 0.0
          },
          "rotation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.1,
            "w": 0.995
          }
        }
      }
    ]
  },
  "front_camera": {
    "type": "camera_parameters",
    "events": [
      {
        "timestamp": 1634567890.123,
        "widthPx": 1920,
        "heightPx": 1080,
        "intrinsics": {
          "type": "simple",
          "fx": 1000.0,
          "fy": 1000.0,
          "ox": 960.0,
          "oy": 540.0
        }
      }
    ]
  },
  "front_camera_images": {
    "type": "image",
    "camera": "front_camera",
    "events": [
      {
        "uri": "https://example.com/image_001.jpg",
        "timestamp": 1634567890.123
      },
      {
        "uri": "https://example.com/image_002.jpg",
        "timestamp": 1634567890.223
      }
    ]
  }
}

​
Image Stream with Camera Calibration

Add full camera calibration parameters with the image stream:
Image stream with camera calibration
{
  "ego_vehicle": {
    "type": "frame_of_reference",
    "id": "ego_vehicle",
    "parentForId": null,
    "events": [
      {
        "timestamp": 1634567890.123,
        "pose": {
          "position": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0
          },
          "rotation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0,
            "w": 1.0
          }
        }
      },
      {
        "timestamp": 1634567890.223,
        "pose": {
          "position": {
            "x": 1.5,
            "y": 0.2,
            "z": 0.0
          },
          "rotation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.1,
            "w": 0.995
          }
        }
      }
    ]
  },
  "front_camera": {
    "type": "camera_parameters",
    "events": [
      {
        "timestamp": 1634567890.123,
        "widthPx": 1920,
        "heightPx": 1080,
        "intrinsics": {
          "type": "simple",
          "fx": 1000.0,
          "fy": 1000.0,
          "ox": 960.0,
          "oy": 540.0,
          "model": {
            "type": "pinhole"
          }
        },
        "extrinsics": {
          "position": {
            "x": 0.0,
            "y": 0.0,
            "z": 2.0
          },
          "rotation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0,
            "w": 1.0
          }
        }
      }
    ]
  },
  "front_camera_images": {
    "type": "image",
    "camera": "front_camera",
    "events": [
      {
        "uri": "https://example.com/image_001.jpg",
        "timestamp": 1634567890.123
      },
      {
        "uri": "https://example.com/image_002.jpg",
        "timestamp": 1634567890.223
      }
    ]
  }
}

​
Format Details

​
Pose Representations

Poses can be specified in multiple formats. Named Position + Quaternion: 
{
  "position": { "x": 1.0, "y": 2.0, "z": 3.0 },
  "rotation": { "x": 0.0, "y": 0.0, "z": 0.0, "w": 1.0 }
}
Named Position + Euler Angles: 
{
  "position": { "x": 1.0, "y": 2.0, "z": 3.0 },
  "rotation": { "x": 0.1, "y": 0.2, "z": 0.3 }
}
4x4 Affine Transform Matrix (column-major): 
[
  1, 0, 0, 0,
  0, 1, 0, 0,
  0, 0, 1, 0,
  1, 2, 3, 1
]

​
Timestamp Formats

Encord supports multiple timestamp formats:
  • Unix timestamp (float): 1634567890.123
  • Unix timestamp (int): 1634567890
  • ISO datetime string: “2021-10-18T10:31:30.123Z”
  • Time-only string: “10:31:30.123”

​
Scene Configuration

You can specify coordinate system conventions:
Scene Configuration
{
  "worldConvention": {
    "x": "right",
    "y": "forward",
    "z": "up"
  },
  "cameraConvention": {
    "x": "right",
    "y": "down",
    "z": "forward"
  },
  "content": {...} // Your scene content here
}

​
PCD Concepts

​
Sensor Data Streams

Streams comprise a sequence of messages coming out of a sensor (LiDAR, camera, accelerometer) at discrete moments in time.

​
Stream Rendering and Data Access

Encord renders the latest available data per stream, where at any given time point T, the most recent data available at or before T from that stream is displayed. For streams without explicit timestamps, Encord assigns implicit sequential timestamps (1, 2, 3, etc.), allowing for consistent temporal ordering while maintaining flexibility in data ingestion.

​
Frame of Reference Hierarchies

​
Hierarchical Coordinate System Organization

Frame of reference hierarchies establish spatial relationships between different coordinate systems in a tree structure, with a root frame at the top. A typical hierarchy includes:
  • World Frame: The global reference frame, often representing a fixed point in the environment
  • Ego Vehicle Frame: The coordinate system of the primary platform (car, robot, etc.)
  • Sensor Frames: Individual coordinate systems for each sensor, positioned relative to the vehicle frame

​
Static vs Dynamic Transformations

Frame relationships can be either static (fixed relative position/orientation) or dynamic (changing over time). Static transforms are used for rigidly mounted sensors, while dynamic transforms represent moving parts or the motion of the entire system through the world. Dynamic transforms are represented as messages in a stream.

​
Coordinate System Conventions

Different domains use different coordinate system conventions. The format allows specification of both world and camera coordinate conventions:
  • World Convention: Typically right-handed systems where axes x,y,z represent directions like “right,” “forward,” and “up”
  • Camera Convention: Often follows computer vision conventions where axes might represent “right,” “down,” and “forward”

​
Camera Calibration and Image Distortion

​
Intrinsic Camera Parameters

To project 3D world points onto the 2D image plane, we need camera calibration that involves determining both intrinsic and extrinsic parameters. Intrinsic parameters are specific to the camera hardware and include:
  • Focal Length (fx, fy): The distance between the camera lens and the image sensor, typically measured in pixels
  • Principal Point (cx, cy): The coordinates of the image center where the optical axis intersects the image plane
  • Skew Coefficient: Accounts for non-rectangular pixels (rarely used in modern cameras)

​
Extrinsic Camera Parameters

Extrinsic parameters define the camera’s position and orientation in 3D space relative to the world coordinate system:
  • Rotation Matrix (R): Describes the camera’s orientation using a 3x3 rotation matrix
  • Translation Vector (t): Specifies the camera’s position in world coordinates

​
Lens Distortion Correction

Camera lenses can introduce distortions that cause straight lines to appear curved in images. The format supports distortion correction through distortion coefficients that model:
  • Radial Distortion: Caused by light rays bending more near the lens edges, creating barrel or pincushion effects
  • Tangential Distortion: Results from lens misalignment with the image sensor

​
Data Format Architecture

The format supports two primary data organization approaches: Message-Based JSON: Every stream and message within a stream is in a big JSON scene file. The data of point clouds or images themselves are URIs to external files. Container-Based Storage: Single files (MCAP, ROS bag, DB3) contain multiple sensor streams and their messages and include the point cloud and image data