Building Blocks for Digital Agents Research

New to agents? See our Intro to UI Agents. We take you through the installation and running your first agent step-by-step.

Why OpenApps? Evaluate and train multimodal agents to use apps like humans do (by clicking, typing, and scrolling):

✅ Unlimited data (for evaluating and training UI-agents): Configurable state and design to generate thousands of versions of each app

✅ Lightweight: runs on a single CPU (and Python-based); no Docker or OS emulators needed

✅ Ground truth rewards: task rewards are based on the underlying state and all app logic is transparent in Python

Install

Install the conda alternative uv and clone the repo:

   git clone https://github.com/facebookresearch/OpenApps.git

Install dependencies:

   uv sync

For other installation options and online shop setup see Installation.

Run OpenApps

uv run launch.py 

For an overview, checkout our video tutorial.

App variations

Each app can be modified with variables available in config/apps. You can override any of these via command line:

uv run launch.py 'apps.todo.init_todos=[["Call Mom", false]]'

OpenApps also comes with pre-defined variations that can affect the content and appearance of apps.

Appearance

export APPEARANCE=challenging_font

export APPEARANCE=dark_theme

export APPEARANCE=default

Launch specific apps with selected appearance:

uv run launch.py apps/start_page/appearance=$APPEARANCE

Or specific apps with: apps/calendar/appearance=$APPEARANCE.

Content

export CONTENT=german

export CONTENT=long_descriptions

uv run launch.py apps/pop_ups=adversarial_descriptions

uv run launch.py apps/start_page/content=$CONTENT

Or specific apps with: apps/calendar/content=$CONTENT.

You can see the specific variables for each defined in the individual apps. For example, config/apps/maps/appearance/dark_theme.yaml.

Launch Agent

For agents to directly interact with apps, install: playwright install chromium.

Launch an agent to perform a task of adding a meeting with Dennis to the calendar:

uv run launch_agent.py agent=dummy task_name=add_meeting_with_dennis

# export OPENAI_API_KEY=""
uv run launch_agent.py agent=GPT-5-1 task_name=add_meeting_with_dennis

You can specify the agent of your choice with the agent= argument. For example agent=dummy is a simple agent that clicks randomly on any buttons, great for exploration!

Learn more about launching with OpenAI, Claude, VLLM models, or specialized models such as UI-Tars in agents guide and available tasks in our task guide.

To see the agent solving the task live:

uv run launch_agent.py browsergym_env_args.headless=False

Logs

By default, information about the number of steps an agent took, task success, etc. will be shown in the terminal:

...
Experiment results
exp_dir: /Users/m...
n_steps: 10
cum_reward: 0.0
stats.cum_agent_elapsed: 0.0017838478088378906
stats.max_agent_elapsed: 0.0002570152282714844
...

All logs are stored log_outputs will contain information about each run

You can also enable logging to weights and biases by logging into your account and setting the flag: use_wandb=True.

Launch Agent(s) Across Multiple Tasks

launch thousands of app variations to study agent behaviors in parallel

coming soon!

Testing

Run all tests via:

uv run -m pytest tests/

Attribution

Our apps are built on top of several excellent frameworks:

• FastHTML framework and examples which allowed us to build fully functional apps in Python, the language most familiar to AI researchers.
• Browser Gym and AgentLab:
• Spacy: for natural language processing
• Open Street Maps: for our Maps apps.
• (and for the optional webshop) we rely on WebShop developed by Princeton University

Some icons are have been designed using resources from Flaticon.com

Our work is licensed under CC-BY-NC, please refer to the LICENSE file in the top level directory. Copyright © Meta Platforms, Inc. See the Terms of Use and Privacy Policy for this project.