Skip to main content

Aria Digital Twin Dataset

Overview​

Project Aria Tools provides Python and C++ APIs to access the Aria Digital Twin (ADT) dataset (paper).

About the data​

ADT provides raw and synthesized sensor data from Project Aria glasses, combined with groundtruth data generated using a motion capture system including depth images, device trajectories, object trajectories and bounding boxes, and human tracking. We also provide processed sensor data from our Machine Perception Services. Go to ADT Data Format to see a full list of the data we provide.

The ADT dataset contains 222 sequences recording single and dual-person activities. The data was recorded in two spaces: an apartment and a single room office. There are 74 single-instance dynamic objects shared between the two spaces.

Go to the Getting Started Tutorial to explore the sample dataset (available on Google colab, no download necessary) or the Dataset Download page to get started.

The sample dataset is a single-user dataset with body pose in the Apartment. It is a pretty representative example that should give you an idea of the dataset.

Apartment scene​

170 sequences were recorded in the apartment scene. The apartment comprised of a living room, kitchen, dining room and bedroom and contained 281 unique stationary objects.

Given some objects have multiple instances that may differ slightly, the apartment has 324 stationary object instances in total.

Office scene​

52 sequences were recorded in the office scene, a single room with minimal office furniture. The office room contained 15 unique stationary objects and 20 stationary object instances.

Activities​

In the office scene, users examined objects. For the apartment scene we designed five single-person activities and three dual-person activities.

The single-person activities were:

  • Room decoration
  • Meal preparation
  • Work
  • Object examination
  • Room cleaning

The dual-person activities included:

  • Partying
  • Room cleaning
  • Dining table cleaning

Every activity has 10 to 50 sequences and the activity names are embedded into the sequence_names.

Other statistics:​

  • Number of multi-person sequences: 77
  • Number of sequences with no skeletons: 110
  • Number of sequences with 1 skeleton: 92
  • Number of sequences with 2 skeleton: 20
info

We provide a mix of datasets where users may or may not be wearing an Aria and/or a bodysuit. Please refer to the skeleton_aria_association.json to see the case for each specific sub-sequence.

Documentation​

The ADT section of the wiki covers:

  • Getting Started
    • A quickstart tutorial available as a Google colab project or install Project Aria Tools python package to run locally, run the ADT notebook to access and visualize ground-truth data.
  • Dataset Download
    • A walkthrough of using adt_benchmark_dataset_downloader to download the published ADT dataset.
  • Data Format
    • How ADT data is organized and stored
  • Data Loader
    • APIs to load ADT data with handy code snippets
  • Visualizers
    • Compile and run our visualizer using an example that accesses ADT data in C++.
  • Advanced tutorials
    • A guide to learn how device synchronization works in ADT and run through a Jupyter notebook.
  • ADT Challenges
    • Learn more about the ADT Grand Challenge