Skip to main content

Project Aria Tools Python Visualization

Overview​

This page introduces our core Python visualization tools, developed with Rerun, that are part of Project Aria Tools.

  • Aria Sensor Viewer: 3D visualization of Aria sensors
  • MPS Viewer: renders MPS metadata (point cloud, device trajectory and wearer eye gaze)

Requirements​

Run Aria Sensor Viewer​

viewer_aria_sensors displays the relative position and orientation of all most of Project Aria glasses sensors (cameras, IMUs, microphones, magnetometer & barometer) in a common reference.

viewer_aria_sensors --vrs $MPS_SAMPLE_PATH/sample.vrs

Aria Sensors Viewer

tip
  • Selecting the different sensors in the Blueprint left column will help you quickly identify where a given sensor is located

Run MPS Viewer​

viewer_mps displays an interactive visualization of the Aria VRS RGB frames along with MPS data (Closed loop trajectory, Global point cloud, Wearer eye gaze). As you are playing or moving along the timeline, you can see the position of the camera and the wearer eye gaze direction at the timestamp of your choice.


viewer_mps --vrs $MPS_SAMPLE_PATH/sample.vrs

or to specify each MPS file

viewer_mps --vrs $MPS_SAMPLE_PATH/sample.vrs \
--trajectory $MPS_SAMPLE_PATH/trajectory/closed_loop_trajectory.csv \
--points $MPS_SAMPLE_PATH/trajectory/global_points.csv.gz \
--eyegaze $MPS_SAMPLE_PATH/eye_gaze/personalized_eye_gaze.csv \
--hands $MPS_SAMPLE_PATH/hand_tracking/wrist_and_palm_poses.csv

MPS Visualizer

tip
  • Switching between device_time and timestamp timeline allows you to retrieve a specific timestamp for the VRS sequence or MPS annotation

Visualization of Multi-SLAM data​

The MPS Viewer can also be used to visualize 3D data derived from multiple Project Aria device recordings, if the MPS data has been generated using Multi-SLAM or is part of datasets that contain this type of data, such as Aria Everyday Activities (AEA).

AEA Example​

The example below uses Trajectory and Semi-Dense Point Cloud data from AEA. Go to AEA Dataset Download for how to download this data.

viewer_mps --trajectory `find -P ~/Documents/projectaria_tools_aea_data/loc1*/*/*/closed_loop_trajectory.csv -print` --points `find -P Documents/projectaria_tools_aea_data/loc1*/*/*/semidense_points.csv.gz -print`

Visualization of aligned AEA 3D data

MPS CLI Example​

The following visualization uses Multi-Slam data generated using MPS CLI sample data. The MPS CLI is part of the Aria Research Kit.

viewer_mps --trajectory `find -P  ~/documents/multi_slam_output/*/*/closed_loop_trajectory.csv -print` --points `find -P ~/documents/multi_slam_output/*/*/semidense_points.csv.gz -print`

Visualization of aligned 3D data

An introduction to Rerun​

Rerun is an open source SDK and engine for visualizing and interacting with multi modal data streams. It's usable from Python, Rust and C++. Rerun consists in a log API and a visualizer.

The main GUI sections of the Rerun visualizer are:

  • BluePrint
    • A: User defined Scene Graph (Entities and hierarchy you define)
    • B: Visual view of the Scene Graph (User customizable)
  • Timeline
    • C: Interactive navigation and inspection of log events on multiple timeline (log, frame, or device time)
  • Visibility and property control
    • D: Fine grained control and inspection of Entities/Components

Rerun Visualizer Interface