Ego-Exo4D Dataset
Ego-Exo4D is a foundational dataset for research on video learning and multimodal perception. The open dataset was released in November 2023 and in March 2024 was updated to include more data, more annotations, and more modalities. Ego-Exo4D comes from years of collaboration between Meta’s FAIR (Fundamental Artificial Intelligence Research), Meta’s Project Aria, and 15 university partners. Ego-Exo4D is unique because of its simultaneous capture of:
- First-person “egocentric” views, from a participant’s wearable camera
- Multiple “exocentric” views, from cameras surrounding the participant
The two perspectives are complementary. While the egocentric perspective reveals what the participant sees and hears, the exocentric views reveal the surrounding scene and the context. Together, these two perspectives give AI models a new window into complex human skill.
More than 800 participants from 13 cities worldwide performed these activities in 131 different natural scene contexts, yielding long-form captures from 1 to 42 minutes each and 1,422 hours of video combined. The dataset contains skilled activities covering both physical (Soccer, Basketball, Dance, Bouldering, Music) and procedural (Cooking, Bike Repair, Health) tasks.
Ego-Exo4D has its own independent website and documentation. Project Aria Tools provides documentation and tooling for working with Aria data that can be helpful when working with Ego-Exo4D data.
Resources:
Ego-Exo4D resources on Project Aria Tools
There are a range of resources available for working with Aria data using Project Aria Tools, and Load Static Calibration Data was specifically created to support working with Ego-Exo4D data.