Skip to main content

Project Aria Tools C++ Visualization

Overview​

This page introduces our C++ visualization tools available in Project Aria Tools. We've provided example datasets to test these tools.

Requirements​

Step 0 : Check system requirements and download codebase​

Go to the Download Codebase page to:

  • Check your system is supported
  • Download projectaria_tools codebase from the GitHub

Step 1 : Download the sample dataset​

Step 2 : Build and install visualizers​

The visualizers need the C++ version of Project Aria Tools to run.

Run Aria Viewer​

Aria Viewer enable you to to visualize Aria device recorded VRS files. It shows all sensor data including:

  • Camera images
  • IMU
  • Audio (visualization of waveform, sound is not available)
cd $HOME/Documents/projectaria_sandbox/build

./tools/visualization/aria_viewer --vrs $MPS_SAMPLE_PATH/sample.vrs

Aria Viewer Screenshot

Run MPS 3D Scene Viewer​

The MPS 3D Scene Viewer renders a static scene using location MPS output.

Through this tool you can create visualizations using:

  • Closed loop trajectories
  • Global point cloud
  • Static camera poses
  • Open loop trajectories
    • Because open loop is in odometry frame of reference, it shouldn’t be visualized with closed loop trajectories, global points or static camera poses

This tutorial generates a visualization containing:

  • Closed loop trajectories
  • Global point cloud
cd $HOME/Documents/projectaria_sandbox/build

./tools/mps_visualization/mps_3d_scene_viewer \
    --closed-loop-traj $MPS_SAMPLE_PATH/trajectory/closed_loop_trajectory.csv \
    --global-point-cloud $MPS_SAMPLE_PATH/trajectory/global_points.csv.gz

Screenshot of 3D Scene Viewer UI

info

Because the sample dataset doesn't have static cameras you won't be able to interact with the static camera settings

Run MPS 3D Replay Viewer​

The MPS 3D Replay Viewer renders static scene and dynamic elements at each frame: Aria's pose + 2D/3D observations rays + eye gaze data.

Through this tool you can create visualizations using:

  • Closed loop trajectories
  • Semi-Dense Point Cloud
    • Global point cloud
    • Point observations
  • Static camera poses

This tutorial generates a visualization containing:

  • Static elements
    • Closed loop trajectories
    • Global point cloud
  • Dynamic elements
    • One closed loop trajectory for replay
    • 2D/3D point observations rays
    • Generalized and Personalized Eye Gaze vectors
cd $HOME/Documents/projectaria_sandbox/build

./tools/mps_visualization/mps_3d_replay_viewer \
  --vrs $MPS_SAMPLE_PATH/sample.vrs  \
  --replay-trajectory $MPS_SAMPLE_PATH/trajectory/closed_loop_trajectory.csv \
  --closed-loop-traj $MPS_SAMPLE_PATH/trajectory/closed_loop_trajectory.csv \
  --global-point-cloud $MPS_SAMPLE_PATH/trajectory/global_points.csv.gz \
  --point-obs $MPS_SAMPLE_PATH/trajectory/semidense_observations.csv.gz \
  --generalized-eye-gaze $MPS_SAMPLE_PATH/eye_gaze/generalized_eye_gaze.csv \
  --calibrated-eye-gaze $MPS_SAMPLE_PATH/eye_gaze/calibrated_eye_gaze.csv \
  --hands $MPS_SAMPLE_PATH/hand_tracking/wrist_and_palm_poses.csv

Screenshot of 3D Replay Viewer UI

MPS Eye Gaze visualizer​

The MPS Eye Gaze visualizer renders the computed eye gaze and vrs data side by side. The visualizer contains:

  • Eye Tracking camera stream
  • RGB, Mono Scene (SLAM) left and right camera streams
    • A red dot shows the projection of the eye gaze onto the image
    • The projection is computed using a fixed depth of 1m
  • 2D graph plot of the gaze yaw and pitch angles in radians
  • 2D radar plot of the eye gaze yaw and pitch angles

Run visualizer and visualize both generalized and optional calibrated eye gaze​

cd $HOME/Documents/projectaria_sandbox/build

./tools/mps_visualization/mps_eyegaze_viewer --vrs $MPS_SAMPLE_PATH/sample.vrs \
    --generalized-eye-gaze $MPS_SAMPLE_PATH/eye_gaze/general_eye_gaze.csv \
    --calibrated-eye-gaze $MPS_SAMPLE_PATH/eye_gaze/personalized_eye_gaze.csv

Screenshot of MPS Eye Gaze Viewer

Troubleshooting​

Check the Troubleshooting Guide if you encounter issues using this tutorial.