Building Blocks for Digital Agents Research

New to agents? See our Intro to UI Agents.

Install

In a new uv environment:

uv pip install git+https://github.com/facebookresearch/openapps.git

For other installation options and online shop setup see Installation.

Run OpenApps

uv run launch.py 

App variations

Each app can be modified with variables available in config/apps. You can override any of these via command line:

uv run launch.py app.todo.title='Super Todo'

OpenApps also comes with pre-defined variations that can affect the content and appearance of apps.

Appearance

export APPEARANCE=challenging_font

export APPEARANCE=dark_theme

export APPEARANCE=default

Launch specific apps with selected appearance:

uv run launch.py apps/start_page/appearance=$APPEARANCE

Or specific apps with: apps/calendar/appearance=$APPEARANCE.

Content

export CONTENT=german

export CONTENT=long_descriptions

uv run launch.py apps/pop_ups=adversarial_descriptions

uv run launch.py apps/start_page/content=$CONTENT

Or specific apps with: apps/calendar/content=$CONTENT.

You can see the specific variables for each defined in the individual apps. For example, config/apps/maps/appearance/dark_theme.yaml.

Launch Agent

For agents to directly interact with apps, install: playwright install chromium.

Launch an agent to perform a task of adding a meeting with Dennis to the calendar:

uv run launch_agent.py agent=dummy task_name=add_meeting_with_dennis

# export OPENAI_API_KEY=""
uv run launch_agent.py agent=GPT-4o task_name=add_meeting_with_dennis

You can specify the agent of your choice with the agent= argument. For example agent=dummy is a simple agent that clicks randomly on any buttons, great for exploration!

Learn more about launching with OpenAI, Claude, VLLM models, or specialized models such as UI-Tars in agents guide and available tasks in our task guide.

To see the agent solving the task live:

uv run launch_agent.py browsergym_env_args.headless=False

Logs

By default, information about the number of steps an agent took, task success, etc. will be shown in the terminal:

...
Experiment results
exp_dir: /Users/m...
n_steps: 10
cum_reward: 0.0
stats.cum_agent_elapsed: 0.0017838478088378906
stats.max_agent_elapsed: 0.0002570152282714844
...

All logs are stored log_outputs will contain information about each run

You can also enable logging to weights and biases by logging into your account and setting the flag: use_wandb=True.

Launch Agent(s) Across Multiple Tasks

launch thousands of app variations to study agent behaviors in parallel

coming soon!

Testing

Run all tests via:

uv run -m pytest tests/

Attribution

Our apps are built on top of several excellent frameworks:

• FastHTML framework and examples which allowed us to build fully functional apps in Python, the language most familiar to AI researchers.
• Browser Gym and AgentLab:
• Spacy: for natural language processing
• Open Street Maps: for our Maps apps.
• (and for the optional webshop) we rely on WebShop developed by Princeton University

Some icons are have been designed using resources from Flaticon.com

Our work is licensed under CC-BY-NC, please refer to the LICENSE file in the top level directory. Copyright © Meta Platforms, Inc. See the Terms of Use and Privacy Policy for this project.