Project Aria In-Session Eye Gaze Calibration

Overview

This page provides an overview of what In-Session Eye Gaze Calibration does, and how to collect In-Session Eye Gaze Calibration with Project Aria glasses.

Not required to request Eye Gaze MPS

In-Session Eye Gaze Calibration provides an additional personalized Eye Gaze output, but is not required for requesting Eye Gaze MPS.

What In-Session Eye Gaze Calibration does

In-Session Eye Gaze Calibration enables researchers to improve the eye gaze estimations in Eye Gaze MPS outputs, enabling researchers to more accurately determine where wearers are looking during the recordings.

When you request Eye Gaze Machine Perception Services (MPS) and the file has an in-session Eye Gaze Calibration as part of the VRS file, you will receive two outputs:

• general_eye_gaze.csv - based on the standard eye gaze configuration
• personalized_eye_gaze.csv - personalized eye gaze data based on the calibration data collected in the recording

Every person is unique in terms of how they move their eyes and look at objects, so the personalized_eye_gaze estimation is expected to be more accurate for the individual.

Further resources:

• Eye Gaze Data Formats
• How to Request MPS
• Recording Profiles
• Machine Perception Services Tutorial - includes sample output from a recording where good in-session Eye Gaze Calibration data was collected

note

Eye gaze calibration is not the same as Aria device calibration. For information about Aria device calibration, go to Project Aria Device Calibration.

HW & SW Requirements

Android

Hardware

• ARCore Depth API (https://developers.google.com/ar/devices) support

Software

• Google Play Services for AR (https://play.google.com/store/apps/details?id=com.google.ar.core)
  • For phones that have ARCore Depth API capabilities, it's normally installed by default
  • You'll see the error message "Please check that Google Play Services for AR is installed" if you don't have ARCore Depth API installed.

iOS

Hardware

• TrueDepth camera (iPhone X or later)

Software

• iOS 14 or later

How to Collect In-Session Eye Gaze Calibration

In-Session Eye Gaze Calibration can only be initiated via the Mobile Companion App. Follow these steps for any recording where you may wish to generate Calibrated Eye Gaze MPS.

1. In the Mobile Companion app, create a new recording using a profile that includes ET and RGB cameras (such as Profile 15 or 25)
2. Once your recording has started, close the recording window
   • Select X on the top left of the screen
3. Go to Device Settings
   • Tap the Paired Glasses info card on the Dashboard
4. Select Eye Tracking Calibration
5. Confirm that you’d like to run it during the current recording session
6. Follow the prompts to calibrate your glasses

To view Device Settings

Tap the paired glasses info card.

Multiple Users Within the One Recording

It’s possible for multiple users to do an in-session calibration within the one recording.

When a new user gets the glasses, the first thing they should do is the in-session Eye Tracking Calibration.

• See Eye Gaze Data Format for how multiple users are tracked.

Eye Gaze Calibration tips

Things to avoid

❌ Do not wear a face covering during eye calibration.

❌ Choose an area with ample and even lighting; do not face a bright light, window or reflective surface.

❌ Do not set your phone screen brightness too high compared to your surroundings.

❌ Do not fully extend your arm(s) during eye calibration. Your elbows should be bent so that the phone is roughly 1 ft (30 cm) away from your face.

Things to do

✅ The phone should be held straight in front of your face, so that you shouldn't look up or down to see the screen. Hold the phone plumb (90 degrees) vertically to the ground.

✅ The "Leveler" stage appears if the position of your phone isn't within specifications for the calibration process. Adjust the phone in front of you and its distance by bending your elbow until the smaller, black circle turns into a green disk with a check mark.

✅ Once the "Leveler" stage is successfully completed, do your best to keep your phone in exactly the same position throughout the full eye calibration process. If your phone is moved to a position no longer suited to calibrate your device, the app will return to the "Leveler" stage.

✅ During eye calibration stages 1 to 10, move your nose towards the direction indicated by the arrow. If you're only following the direction with your gaze without moving your head, the calibration stage will time out and fail. However, please make sure to keep your eyes fixed on the number within the dot the whole time.