Skip to main content

Welcome to the Open Science Initiative

The Open Science Initiative (OSI) is Meta's commitment to advancing egocentric AI research through open collaboration. We provide researchers with access to cutting-edge tools, datasets, and machine learning models built on data from Aria glasses. OSI enables you to analyze, visualize, and build upon rich multimodal egocentric data without needing your own Aria device.

What is OSI?​

OSI makes egocentric AI research accessible by open-sourcing three key components:

📊 Public Datasets​

High-quality, multimodal egocentric datasets collected with Aria glasses, featuring:

  • Raw sensor streams from multiple synchronized cameras, IMUs, audio, and more
  • On-device machine perception outputs (VIO, eye gaze, hand tracking)
  • Offline machine perception annotations from cloud-based processing
  • Rich semantic annotations from state-of-the-art perception algorithms

Our datasets capture real-world scenarios with time-synchronized multi-participant recordings, enabling research in computer vision, multimodal learning, robotics, and contextual AI.

🛠️ projectaria-tools​

A comprehensive Python/C++ library for working with Aria data:

  • Load and visualize VRS (Video Recording Storage) files
  • Access device calibration and sensor data
  • Work with machine perception outputs (VIO, eye gaze, hand tracking)
  • Process and analyze multimodal sensor streams
  • Export data to standard formats

🤖 Open Models​

Pre-trained machine learning models and algorithms for egocentric perception:

  • Coming soon...

What You Can Do with OSI​

📥 Access Research Data​

Download and explore publicly available egocentric datasets:

  • Multi-sensor time-synchronized recordings
  • Ground truth annotations from advanced perception algorithms
  • Pre-processed machine perception outputs
  • Calibration data for all sensors

🔬 Analyze Egocentric Data​

Use Project Aria Tools to:

  • Load and visualize VRS recordings in Python
  • Access raw sensor data (RGB, CV cameras, IMU, audio, etc.)
  • Work with device calibration and coordinate transformations
  • Query time-synchronized sensor streams efficiently
  • Visualize 3D trajectories and point clouds with Rerun

🧪 Build and Train Models​

Leverage open datasets and tools to:

  • Train computer vision models on egocentric data
  • Develop multimodal perception algorithms
  • Research human attention and gaze patterns
  • Study hand-object interactions
  • Explore social dynamics in multi-participant scenarios

📊 Benchmark Your Algorithms​

Compare your methods against:

  • Baseline perception algorithms
  • Pre-trained open models
  • Published research results
  • State-of-the-art egocentric AI systems

Getting Started​

Choose your starting point based on your research goals:

For Dataset Users​

Begin with the Aria Gen 2 Pilot Dataset to explore available data. Then follow the Download Guide to access the dataset and start your research.

For Tool Users​

Install Project Aria Tools to work with Aria data. The library provides Python APIs and tutorials for loading, visualizing, and processing VRS files.

For Algorithm Developers​

Explore the Open Models section to access pre-trained models and algorithms. Use them as baselines or building blocks for your own research.

For First-Time Users​

Start with the Python Tutorials to learn the basics:

  1. VrsDataProvider Basics - Load and access Aria data
  2. Device Calibration - Work with sensor calibration
  3. Queued Sensor Data - Efficiently stream multi-sensor data
  4. Eye Tracking & Hand Tracking - Access on-device perception outputs
  5. MPS Data Loading - Visualize machine perception results

Research Impact​

OSI enables research in:

  • Egocentric Vision: First-person computer vision and scene understanding
  • Multimodal AI: Fusion of vision, audio, IMU, and physiological signals
  • Human-AI Interaction: Gaze-based interfaces and attention modeling
  • Robotics: Manipulation learning from human demonstrations
  • Social Computing: Multi-participant interaction analysis
  • Contextual AI: Understanding user context and intent
  • Augmented Reality: Spatial computing and scene reconstruction

Community & Support​

Join a growing community of researchers working with Aria data:

  • Access open-source code and tutorials
  • Contribute to Project Aria Tools development
  • Share your research and findings
  • Collaborate on new datasets and models

Citation​

If you use OSI datasets or tools in your research, please cite the relevant papers and acknowledge the Project Aria platform.

Ready to explore? Choose a starting point above to begin your journey with the Open Science Initiative.