Skip to main content

AEA Data Format

The Aria Everyday Activities dataset contains multiple activity sequences for one to two Project Aria glasses users. We created recordings using scripts to represent all day activities with always on sensing.

Each Aria glasses recording is stored as its own sequence with all data related to that recording self contained within that sequence folder. An example sequence folder would look like this:

loc1_script1_seq1_rec1
   ├── recording.vrs
   ├── metadata.json
   ├── speech.csv
   ├──  MPS
       ├── eye_gaze
           ├── general_eye_gaze.csv
           ├── summary.json
       ├── slam
           ├── closed_loop_trajectory.csv
           ├── open_loop_trajectory.csv
           ├── online_calibration.csv
           ├── semidense_observations.csv.gz
           ├── semidense_points.csv.gz
           ├── summary.json

SLAM output​

The SLAM outputs were created in a shared coordinate frame. The Multi-SLAM data format page contains more information about this output. Please note the file structure is slightly different (vrs_to_multi_slam.json is not necessary) compared to MPS CLI Multi-SLAM requests, as the shared coordinate frame is organized by location.

Speech to Text annotation​

Speech to Text annotation provides text strings generated by Automatic Speech Recognition (ASR) with timestamps and confidence rating. The ASR annotation used an in-house proprietary system. Similar results can be acquired via open-source ASR solutions.

Table 2: speech.csv Structure

startTime_nsendTime_nswrittenconfidence
5404055040I’m0.25608
7292073920looking0.84339

Timestamps Mapping Data​

Project Aria glasses and multi-view devices operating in proximity to each other (<100m) can leverage SMPTE timecode to receive a synchronized time clock with sub-millisecond accuracy.

The mapping between device time and timecode clock for each sequence is stored in the VRS file as a Time Domain Mapping Class. Go to Timestamps in Aria VRS Files for more information about how Aria sensor data is timestamped.

To translate the local timestamp of an arbitrary piece of data to the timecode time domain, you can interpolate between device timestamps in the time domain mapping data. An implementation of this mechanism is already provided in VrsDataProvider. To synchronize data from a secondary device, you can query that second VRS with this timecode time. This functionality is also already implemented in the VrsDataProvider class.