Ego-Exo4D Data Format and Loader

Ego-Exo4D includes Project Aria's SLAM (Trajectory and Semi-Dense Point Cloud), and Eye Gaze MPS outputs. For information about these, go to MPS data formats. This dataset also contains 3D calibration for the multiple static external cameras (GoPros).

Go to the Aria Ego-Exo4D Examples Notebook for code snippets to load and use this data.

3D Static camera calibration

Data Format

Column	Type	Description
cam_uid	string	Unique identifier of camera
graph_uid	string	Unique identifier of the world coordinate frame
{tx,ty,tz,qx,qy,qz,qw}_world_cam	float	Pose of the camera coordinate frame in world frame T_world_cam, translation (tx, ty, tz) in meters and rotation quaternion (qx, qy, qz, qw)
image_width	int	Image size in pixels
image_height	int	Image size in pixels
intrinsics_type	string	Camera intrinsics calibration type. Currently support types: KANNALABRANDTK3: KB3 model
intrinsics_{0-7}	float	Camera intrinsics parameters
start_frame_idx	int	Used to determine if start frame number of the video is stationary, and if stationary camera pose and intrinsic calibration results can be applied. start_frame_idx and end_frame_idx will both be -1 if the stationary pose and intrinsic calibration can be applied to the whole video
end_frame_idx	int	Used to determine if the end frame number of the video is stationary and if stationary camera pose and intrinsic calibration results can be applied. start_frame_idx and end_frame_idx will both be -1 if the stationary pose and intrinsic calibration can be applied to the whole video

Load 3D Static Calibration data

Data loaders for MPS outputs are provided as part of Project Aria Tools (projectaria_tools/main/core/mps). As part of this, the loaders put the outputs into data structures that are easier for other tools to consume.

Ego-Exo4d data is unique in that it has recordings from static cameras as well as moving Project Aria glasses. The following commands enable you to load poses and intrinsic calibration of set stationary cameras (GoPros) alongside Project Aria data.

Python

import projectaria_tools.core.mps as mps

static_cameras_path = "/path/to/mps/output/trajectory/static_cam_calibs.csv"
static_cameras = mps.read_static_camera_calibrations(static_cameras_path)

C++

#include <StaticCameraCalibrationReader.h>
using namespace projectaria::tools::mps;

std::string staticCamerasPath = "/path/to/mps/output/trajectory/static_cam_calibs.csv";
StaticCameraCalibrations staticCameras = readStaticCameraCalibrations(staticCamerasPath);