Skip to main content

Data Loader

Data loading is broken down into two main loaders: AriaDigitalTwinDataPathsProvider, AriaDigitalTwinDataProvider. In this section, we will give a high level introduction to the two data loader types.

As with most of projectaria_tools, we provide a C++ implementation, with pybindings so that users can use our libraries in C++ or Python without loosing functionality.

AriaDigitalTwinDataPathsProvider​

The main goal of this loader is to give the user an easy way to load all filepaths associated with an ADT sequence and the sequence metadata given an input to the sequence folder root path. The AriaDigitalTwinDataPathsProvider also allows the user to select specific annotations to load (e.g., with or without skeleton). Once a class is instantiated, this can be used manually to query for specific files without needing to know filenames, or the data paths class can be extracted and fed directly to the AriaDigitalTwinDataProvider described below.

from projectaria_tools.projects.adt import AriaDigitalTwinDataPathsProvider

# define the sequence path you want to load
sequence_path = "PATH/TO/An_ADT_sequence"

# create path provider
paths_provider = AriaDigitalTwinDataPathsProvider(sequence_path)

# load the set of ground truth data files without skeleton occlusion
data_paths_without_skeleton_occlusion = paths_provider.get_datapaths(False)

# load the set of ground truth data files with skeleton occlusion
data_paths_with_skeleton_occlusion = paths_provider.get_datapaths(True)

AriaDigitalTwinDataProvider​

This is the core data loader that takes an instance of the AriaDigitalTwinDataPaths class (generated by the AriaDigitalTwinDataPathsProvider) and provides you will query functions to access all ADT data. The following shows an example snippet to load ground truth data with the AriaDigitalTwinDataProvider:

from projectaria_tools.projects.adt import AriaDigitalTwinDataPathsProvider, AriaDigitalTwinDataProvider

# define the sequence path you want to load
sequence_path = "PATH/TO/An_ADT_sequence"

# create path provider
paths_provider = AriaDigitalTwinDataPathsProvider(sequence_path)

# load the set of ground truth data files with skeleton occlusion
data_paths_with_skeleton_occlusion = paths_provider.get_datapaths(True)

# create data provider
gt_provider = AriaDigitalTwinDataProvider(data_paths_with_skeleton_occlusion)

Skip Data loading​

All data loaders are designed to allow the user to skip the loading of specific data types. You can do this by setting the path to an empty string in your AriaDigitalTwinDataPaths instance prior to constructing the AriaDigitalTwinDataProvider.

Check Data Existence​

Since we allow users to skip specific data type loading as explained above, we also provide functions in in AriaDigitalTwinDataProvider to check if data exists by calling their appropriate functions before calling the corresponding getter functions. E.g. hasObject3dBoundingboxes()

Ground Truth Data Getter Functions​

For a full example of the Python getters, please refer to the notebook in the Getting Started. For a full example of the C++ getters, please refer to the visualizer example.

Getting Instance Information​

In ADT, we define an instance to be either a human or an object. The attributes of an instance is defined in class InstanceInfo in AriaDigitalTwinDataTypes. We use instanceType to differentiate a human and an object.

Time Query Options​

You may have also noticed the timeQueryOptions parameter in the above getter functions. Same as dataprovider, all getter functions for timestamped data allow you to specify how to query the timestamps. The options are defined in TimeTypes

Accessing Timestamped Data​

All timestamped data query APIs return a templated DataWithDt class. For example, BoundingBox2dDataWithDt defined in AriaDigitalTwinDataTypes as:

using BoundingBox2dDataWithDt = DataWithDt<TypeBoundingBox2dMap>;

The goal of wrapping all data in a DataWithDt class is to ensure all returned timestamped data has two fields: isValid, and dtNs. Where isValid defines whether or not the returned data is valid, since all timestamp queries may be invalid times, and dtNs to ensure the user always knows the time difference between the returned data and the query time.

Interpolation Function​

We provide interpolation functions for 6DoF Aria poses and Object 3d bounding boxes called "getInterpolatedAria3dPoseAtTimestampNs" and "getInterpolatedObject3dBoundingBoxesAtTimestampNs" in AriaDigitalTwinDataProvider

Time Synchronization Between Subsequences​

The Multiperson Synchronization advanced tutorial shows how to synchronize subsequences in an ADT sequence.

Further resources:

Skeleton Data​

Separate from the 2D skeleton data, we also have skeleton frames as measured by Optitrack. This data can be accessed directly from the AriaDigitalTwinDataProvider, or using the AriaDigitalTwinSkeletonProvider which can be extracted from AriaDigitalTwinDataProvider. Motive, the software that runs the Optitrack system, generates two types of skeleton data:

  1. Skeleton Markers: a set of 3D marker positions of all visible markers that are attached to the bodysuit. If markers are occluded, they are set to [0,0,0]. We provide a helper function to get the labels: getMarkerLabels() in AriaDigitalTwinSkeletonProvider. For more information see motive’s Biomech57 template

  2. Skeleton Joints: a set of estimated 3D joint positions. We provide a list of these joint positions for each timestamp, as well as the joint labels getJointConnections(), and connections getJointLabels() in in AriaDigitalTwinSkeletonProvider

Note that both the markers and the joints are provided in the ADT Scene frame to be consistent with all other ground truth data.