Scenario JSON Format

The Agents Research Environments scenarios can be exported and imported using a structured JSON format defined by Pydantic models. All Gaia2 scenarios are available in this standardized JSON format.

Overview

The scenario format is built around the ExportedTrace structure, which contains all the information needed to define, execute, and validate a scenario. This includes metadata, applications, events, and validation criteria.

Core Structure

ExportedTrace

The root structure for all Meta Agents Research Environments scenarios:

class ExportedTraceBase(BaseModel):
    world_logs: list[str] = []
    apps: list[ExportedApp] = []
    events: list[ExportedEvent | ExportedOracleEvent] = []
    completed_events: list[ExportedCompletedEvent] = []
    version: str
    context: str | None = None
    augmentation: dict | None = None

class ExportedTrace(ExportedTraceBase):
   metadata: ExportedTraceMetadata

Fields:

  • metadata: Comprehensive metadata about the scenario

  • world_logs: Agent and environment log from the simulation

  • apps: List of applications available in the scenario

  • events: Scheduled events

  • completed_events: Events that have been executed

  • version: Scenario format version

  • context: Optional additional context information

  • augmentation: Optional additional data for augmentation

Metadata Components

The metadata section contains three main categories of information:

Definition Metadata

Core scenario definition:

class ExportedTraceDefinitionMetadata(BaseModel):
   scenario_id: str
   seed: int | None = None
   duration: float | None = None
   time_increment_in_seconds: int | None = None
   start_time: float | None = None
   run_number: int | None = None  # Run number for multiple runs of the same scenario
   hints: list[ExportedHint] = []
   config: str | None = None
   has_a2a_augmentation: bool = False
   has_tool_augmentation: bool = False
   has_env_events_augmentation: bool = False
   has_exception: bool = False
   exception_type: str | None = None
   exception_message: str | None = None
   tags: list[str] | None = None
   hf_metadata: ExportedHugging FaceMetadata | None = None

Fields:

  • scenario_id: Unique identifier for the scenario

  • seed: Random seed for reproducibility

  • duration: Maximum scenario runtime in seconds

  • time_increment_in_seconds: Time increment used to run the scenario

  • start_time: Scenario start timestamp (Unix timestamp)

  • hints: List of hints to guide agent behavior

  • tags: Capability tags for categorization (e.g., “execution”, “search”)

Simulation Metadata

Information about the execution environment:

class ExportedTraceSimulationMetadata(BaseModel):
    agent_id: str
    model_id: str

Fields:

  • agent_id: Identifier of the agent used for execution

  • model_id: Model identifier for tracking and reproducibility

Annotation Metadata

Human validation and quality assurance:

class ExportedTraceAnnotationMetadata(BaseModel):
   annotation_id: str | None = None
   annotator: str | None = None
   validation_decision: str | None = None
   comment: str | None = None
   date: float = 0.0

Fields:

  • validation_decision: Human validation status (“valid”, “invalid”, “needs_review”)

  • comment: Additional notes or feedback from human reviewers

  • annotator: Person who validated the scenario

Applications

Scenarios include multiple applications that agents can interact with:

ExportedApp Structure

class ExportedApp(BaseModel):
    name: str
    class_name: str
    app_state: dict[str, Any]

Fields:

  • name: Human-readable application name

  • class_name: Python class implementing the application

  • app_state: Initial state data for the application

Common Application Types

Email Applications

  • Sending and receiving emails

  • Email filtering and organization tools

Messaging Systems

  • Real-time communication platforms

  • Chat and collaboration tools

File Systems

  • File browsers and managers

  • Document storage and retrieval systems

  • Search and file organization tools

Events and Actions

Events define what happens during scenario execution:

Event Structure

class ExportedEvent(BaseModel):
    class_name: str
    event_type: str
    event_time: float | None
    event_id: str
    dependencies: list[str]
    event_relative_time: float | None
    action: ExportedAction | None = None

Fields:

  • class_name: Event type class (e.g., “Event”, “ConditionCheckEvent”)

  • event_type: Category of event (“AGENT”, “ENV”, “USER”)

  • event_time: Timestamp when the event occurs

  • event_id: Unique event identifier

  • dependencies: IDs of prerequisite events that must complete first

  • event_relative_time: Relative time to parent events

  • action: The action to be executed when the event fires

Event Categories

AGENT Events

Actions initiated by the AI agent during execution

ENV Events

Environmental changes scheduled by the scenario

USER Events

Simulated user interactions and inputs

Action Structure

class ExportedAction(BaseModel):
    action_id: str
    app: str | None = None
    function: str | None = None
    operation_type: str | None = None
    args: list[ExportedActionArg] | None = None

Fields:

  • action_id: Unique action identifier

  • app: Target application name

  • function: Tool name

  • operation_type: Type of operation (“READ”, “WRITE”, “SEARCH”, etc.)

  • args: List of arguments to pass to the tool

Action Arguments

class ExportedActionArg(BaseModel):
    name: str
    value: str | None = None
    value_type: str | None = None

Fields:

  • name: Parameter name

  • value: Parameter value (can be any JSON-serializable type)

  • value_type: Type hint for the parameter

Hints and Guidance

Hints provide guidance to agents during execution:

Hint Structure

class ExportedHint(BaseModel):
    hint_type: str
    content: str
    associated_event_id: str | None

Fields:

  • hint_type: Category of hint (“task”, “context”, “warning”, “tip”)

  • content: The actual hint text

  • associated_event_id: Optional reference to a related event

Hint Types

Task Hints

Direct guidance about the main objective

Environment Hints

Contextual information about the environment

Oracle Events

Oracle events define expected agent behaviors for validation and testing:

Oracle Event Structure

class ExportedOracleEvent(ExportedEvent):
    event_time_comparator: str

Fields:

  • event_time_comparator: Comparator for event time validation (“LESS_THAN”, “GREATER_THAN”, “EQUAL”)

Complete Example

Here’s a complete example of a scenario JSON structure:

{
  "metadata": {
    "scenario_id": "email_apology_001",
    "seed": 12345,
    "duration": 300.0,
    "start_time": 1640995200,
    "hints": [
      {
        "hint_type": "task",
        "content": "Check your email for client complaints",
        "associated_event_id": null
      }
    ],
    "tags": ["email", "communication", "customer-service"]
  },
  "world_logs": [],
  "apps": [
    {
      "name": "EmailApp",
      "class_name": "are.simulation.apps.email.EmailApp",
      "app_state": {
        "inbox": [
          {
            "from": "client@company.com",
            "subject": "Project Delay Concerns",
            "body": "We're concerned about the project timeline...",
            "timestamp": 1640995100
          }
        ]
      }
    }
  ],
  "events": [
    {
      "class_name": "ScheduledEvent",
      "event_type": "ENV",
      "event_time": 1640995260,
      "event_id": "reminder_001",
      "dependencies": [],
      "action": {
        "action_id": "notify_001",
        "app": "NotificationApp",
        "function": "show_reminder",
        "operation_type": "NOTIFY",
        "args": [
          {
            "name": "message",
            "value": "Don't forget to respond to the client",
            "type": "str"
          }
        ]
      }
    }
  ],
  "completed_events": [],
  "version": "2.0",
  "context": "Customer service scenario focusing on professional communication"
}

Schema Validation

The JSON format is validated using Pydantic models, which provide:

Type Safety

Ensures all fields have correct types

Data Validation

Validates field values and constraints

Serialization

Consistent conversion between Python objects and JSON

Error Reporting

Clear error messages for invalid data

Working with the Format

Programmatic Access

Use the JsonScenarioImporter to load and work with scenarios from JSON files:

from are.simulation.data_handler.importer import JsonScenarioImporter

# Initialize the importer
importer = JsonScenarioImporter()

# Load scenario from JSON file
with open('scenario.json', 'r') as f:
    json_content = f.read()

# Import the scenario
scenario, completed_events, world_logs = importer.import_from_json(
    json_str=json_content,
    load_completed_events=True
)

# Access scenario properties
print(f"Scenario ID: {scenario.scenario_id}")
print(f"Duration: {scenario.duration} seconds")
print(f"Number of hints: {len(scenario.hints)}")
print(f"Tags: {scenario.tags}")
print(f"Number of completed events: {len(completed_events)}")

Filtering Applications

Control which applications are loaded when importing scenarios:

from are.simulation.data_handler.importer import JsonScenarioImporter

importer = JsonScenarioImporter()

# Skip specific applications
scenario, _, _ = importer.import_from_json(
    json_str=json_content,
    apps_to_skip=["EmailClientApp", "SlackApp"]
)

# Or keep only specific applications
scenario, _, _ = importer.import_from_json(
    json_str=json_content,
    apps_to_keep=["FileSystemApp", "BrowserApp"]
)

Benchmark Scenarios

Import scenarios specifically for benchmarking with additional metadata handling:

from are.simulation.data_handler.importer import JsonScenarioImporter

importer = JsonScenarioImporter()

# Import as benchmark scenario
benchmark_scenario, completed_events, world_logs = importer.import_from_json_to_benchmark(
    json_str=json_content,
    load_completed_events=True
)

# Benchmark scenarios include additional Hugging Face metadata handling
if hasattr(benchmark_scenario, 'hf_metadata'):
    print(f"Hugging Face dataset: {benchmark_scenario.hf_metadata.dataset}")

Running Scenarios

Run scenarios from local JSON files:

# Directory of scenarios
uvx --from meta-agents-research-environments are-benchmark -d /path/to/scenarios/ -a default
# JSONL file with multiple scenarios
are-benchmark -d /path/to/scenarios.jsonl -a default

Run scenarios from Hugging Face:

# Basic usage
uvx --from meta-agents-research-environments are-benchmark --hf dataset_name --hf-split split_name
# With specific parameters
uvx --from meta-agents-research-environments are-benchmark --hf meta-agents-research-environments/gaia2 \
         --hf-split validation \
         --limit 5 \
         --agent default \
         --model your-model \
         --model_provider your-provider

Scenario Export

Use the JsonScenarioExporter to export scenarios and environments to JSON:

from are.simulation.data_handler.exporter import JsonScenarioExporter
from are.simulation.environment import Environment

# Initialize the exporter
exporter = JsonScenarioExporter()

# Export scenario and environment to JSON string
json_output = exporter.export_to_json(
    env=environment,
    scenario=scenario,
    scenario_id="my_scenario_001",
)

# Save to file
with open('exported_scenario.json', 'w') as f:
    f.write(json_output)

Export to File

Directly export scenarios to files with automatic naming:

from are.simulation.data_handler.exporter import JsonScenarioExporter

exporter = JsonScenarioExporter()

# Export directly to file
success, file_path = exporter.export_to_json_file(
    env=environment,
    scenario=scenario,
    output_dir="/path/to/output",
    export_apps=True  # Include app states in export
)

if success:
    print(f"Scenario exported to: {file_path}")
else:
    print("Export failed")

Controlling App Export

Control whether application states are included in exports:

from are.simulation.data_handler.exporter import JsonScenarioExporter

exporter = JsonScenarioExporter()

# Export without apps (useful when apps are stored separately)
json_output = exporter.export_to_json(
    env=environment,
    scenario=scenario,
    scenario_id="lightweight_scenario",
    export_apps=False  # Exclude app states to reduce file size
)

# Export with custom app states
custom_app_states = {
    "EmailApp": {
        "class_name": "are.simulation.apps.email.EmailApp",
        "serialized_state": '{"inbox": [], "sent": []}'
    }
}

json_output = exporter.export_to_json(
    env=environment,
    scenario=scenario,
    scenario_id="custom_app_scenario",
    apps_state=custom_app_states
)

Working with World Logs

Include agent logs and world state information in exports:

from are.simulation.data_handler.exporter import JsonScenarioExporter
from are.simulation.agents.are_simulation_agent import BaseAgentLog

exporter = JsonScenarioExporter()

# Create world logs
world_logs = [
    BaseAgentLog.from_dict({
        "timestamp": 1640995200,
        "level": "INFO",
        "message": "Agent started scenario execution"
    })
]

# Export with world logs
json_output = exporter.export_to_json(
    env=environment,
    scenario=scenario,
    scenario_id="scenario_with_logs",
    world_logs=world_logs,
)

Next Steps

For hands-on scenario creation, continue to Working with Scenarios for detailed development guidance.

For benchmarking with Meta Agents Research Environments, refer to Benchmarking with Meta Agents Research Environments

Core Scenario JSON Format Classes

ExportedTrace Classes

class are.simulation.data_handler.models.ExportedTraceBase(**data)[source]

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

world_logs: list[str]
apps: list[ExportedApp]
events: list[ExportedEvent | ExportedOracleEvent]
completed_events: list[ExportedCompletedEvent]
version: str
context: str | None
augmentation: dict | None
model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class are.simulation.data_handler.models.ExportedTrace(**data)[source]

Bases: ExportedTraceBase

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

metadata: ExportedTraceMetadata
model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class are.simulation.data_handler.models.ExportedTraceMetadata(**data)[source]

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

definition: ExportedTraceDefinitionMetadata
simulation: ExportedTraceSimulationMetadata | None
annotation: ExportedTraceAnnotationMetadata | None
execution: ExportedExecutionMetadata | None
runner_config: ScenarioRunnerConfig | None
model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class are.simulation.data_handler.models.ExportedTraceDefinitionMetadata(**data)[source]

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

scenario_id: str
seed: int | None
duration: float | None
time_increment_in_seconds: int | None
start_time: float | None
run_number: int | None
hints: list[ExportedHint]
config: str | None
has_a2a_augmentation: bool
has_tool_augmentation: bool
has_env_events_augmentation: bool
has_exception: bool
exception_type: str | None
exception_message: str | None
tags: list[str] | None
hf_metadata: ExportedHuggingFaceMetadata | None
model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class are.simulation.data_handler.models.ExportedApp(**data)[source]

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

name: str
class_name: str | None
app_state: dict[str, Any] | None
model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class are.simulation.data_handler.models.ExportedEvent(**data)[source]

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

class_name: str
event_type: str
event_time: float | None
event_id: str
dependencies: list[str]
event_relative_time: float | None
action: ExportedAction | None
model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class are.simulation.data_handler.models.ExportedCompletedEvent(**data)[source]

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

class_name: str
event_type: str
event_time: float
event_id: str
dependencies: list[str]
event_relative_time: float | None
action: ExportedAction | None
metadata: ExportedEventMetadata | None
model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class are.simulation.data_handler.models.ExportedOracleEvent(**data)[source]

Bases: ExportedEvent

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

event_time_comparator: str | None
model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class are.simulation.data_handler.models.ExportedAction(**data)[source]

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

action_id: str
app: str | None
function: str | None
operation_type: str | None
args: list[ExportedActionArg] | None
model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class are.simulation.data_handler.models.ExportedActionArg(**data)[source]

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

name: str
value: str | None
value_type: str | None
model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class are.simulation.data_handler.models.ExportedHint(**data)[source]

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

hint_type: str
content: str
associated_event_id: str
model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

Exporter class

class are.simulation.data_handler.exporter.JsonScenarioExporter[source]

Bases: object

JSON Scenario Exporter.

static convert_app(app_name, app_class, app_state)[source]
Return type:

ExportedApp

static process_state(state)[source]
Return type:

dict[str, Any] | None

static convert_event(event)[source]
Return type:

ExportedEvent | ExportedOracleEvent

static convert_completed_event(event)[source]
Return type:

ExportedCompletedEvent

static convert_event_metadata(metadata)[source]
Return type:

ExportedEventMetadata

static convert_action(action)[source]
Return type:

ExportedAction

static convert_action_args(name, value)[source]
Return type:

ExportedActionArg

static convert_oracle_action(action_desc)[source]
Return type:

ExportedAction

export_to_json_file(env, scenario, model_id=None, agent_id=None, validation_decision=None, validation_rationale=None, run_duration=None, output_dir=None, export_apps=True, trace_dump_format='hf', scenario_exception=None, runner_config=None)[source]

Export trace data from the environment to a JSON file. :type env: Environment :param env: Environment :type scenario: Scenario :param scenario: Scenario :type model_id: Optional[str] :param model_id: Model ID :type agent_id: Optional[str] :param agent_id: Agent ID :type validation_decision: Optional[str] :param validation_decision: Validation decision :type validation_rationale: Optional[str] :param validation_rationale: Validation rationale :type run_duration: Optional[float] :param run_duration: End-to-end run duration :type output_dir: Optional[str] :param output_dir: Output directory :type export_apps: bool :param export_apps: Whether to export apps in the trace :param config: ScenarioRunnerConfig or MultiScenarioRunnerConfig for filename generation and trace storage :rtype: tuple[bool, str | None] :return: Tuple containing success status and file path if successful

Return type:

tuple[bool, str | None]

export_to_json(env, scenario, scenario_id, runner_config=None, model_id=None, agent_id=None, validation_decision=None, annotation_id=None, annotator_name=None, context=None, comment=None, indent=None, apps_state=None, world_logs=None, export_apps=True, scenario_exception=None, **kwargs)[source]

Export trace data from the environment to a JSON string.

Parameters:

  • env (Environment) – Environment

  • scenario (Scenario) – Scenario

  • scenario_id (str) – Scenario ID

  • model_id (Optional[str]) – Model ID

  • agent_id (Optional[str]) – Agent ID

  • validation_decision (Optional[str]) – Validation decision

  • annotation_id (Optional[str]) – Annotation ID

  • annotator_name (Optional[str]) – Annotator name

  • context (Optional[str]) – Context for bug report

  • comment (Optional[str]) – Annotation comment

  • indent (Optional[int]) – Indentation level for the JSON output

  • apps_state (Optional[dict[str, Any]]) – State of apps to persist in trace. If set to None, the initial scenario app state is used. If this value is set, we assume the exported trace should be a Red Team replay trace.

  • world_logs (Optional[list[BaseAgentLog]]) – List of world logs

  • export_apps (bool) – Whether to export apps in the trace

  • kwargs (Any) – Additional parameters for internal use

Return type:

str

Returns:

JSON string representation of the trace data

export_to_json_lite(env, scenario, scenario_id, model_id=None, agent_id=None, validation_decision=None, validation_rationale=None, run_duration=None)[source]

Importer class

class are.simulation.data_handler.importer.JsonScenarioImporter[source]

Bases: object

JSON Scenario Importer.

SUPPORTED_VERSIONS = ['are_simulation_v1']
static map_action(action_data, app_name_to_class)[source]
static map_event_metadata(metadata_data)[source]
static map_event(input_json, app_name_to_class)[source]
import_from_json(json_str, apps_to_skip=None, apps_to_keep=None, load_completed_events=True)[source]

Imports a scenario and associated data from a JSON string or bytes.

Parameters:

  • json_str (str | bytes) – The JSON data representing the scenario.

  • apps_to_skip (Optional[list[str]]) – A list of app names to skip during import. If None, no apps are skipped.

  • apps_to_keep (Optional[list[str]]) – A list of app names to keep during import. If None, all apps are kept unless skipped.

  • load_completed_events (bool) – Whether to load completed events from the JSON. Defaults to True.

Return type:

tuple[ScenarioImportedFromJson, list[CompletedEvent], list[BaseAgentLog]]

Returns:

A tuple containing: - The imported scenario object. - A list of completed events. - A list of base agent logs.

import_from_json_to_benchmark(json_str, apps_to_skip=None, apps_to_keep=None, load_completed_events=True)[source]
Return type:

tuple[BenchmarkScenarioImportedFromJson, list[CompletedEvent], list[BaseAgentLog]]

JSON Scenario classes

class are.simulation.scenarios.scenario_imported_from_json.scenario.ScenarioImportedFromJson(_initialized=False, is_benchmark_ready=False, events=<factory>, apps=None, tags=(), scenario_id='scenario_imported_from_json', seed=0, nb_turns=None, run_number=None, config=None, has_a2a_augmentation=False, status=ScenarioStatus.Draft, comment=None, annotation_id=None, hints=None, additional_system_prompt=None, start_time=1758655074.158146, duration=1000, queue_based_loop=False, time_increment_in_seconds=1, working_dir='', _initial_apps=None, tool_augmentation_config=None, env_events_config=None, gui_config=None, augmentation_data=<factory>, serialized_events=None, serialized_apps=None, apps_to_skip=None, apps_to_keep=None, execution_metadata=None, has_tool_augmentation=False, has_env_events_augmentation=False, has_exception=False, exception_type=None, exception_message=None)[source]

Bases: Scenario

This is a special class used for importing scenarios from JSON files.

scenario_id: str = 'scenario_imported_from_json'
tags: tuple[CapabilityTag, ...] = ()
start_time: float | None = 1758655074.158146
duration: float | None = 1000
serialized_events: Any = None
serialized_apps: Any = None
apps_to_skip: list[str] | None = None
apps_to_keep: list[str] | None = None
hints: Any = None
status: ScenarioStatus = 'Draft'
execution_metadata: ExecutionMetadata | None = None
config: str | None = None
has_a2a_augmentation: bool = False
has_tool_augmentation: bool = False
has_env_events_augmentation: bool = False
has_exception: bool = False
exception_type: str | None = None
exception_message: str | None = None
build_events_flow()[source]

Core logic of the scenario, this is where the scenario is built. Where events are scheduled, event triggers are defined, as well as any element of the task. By default, this function is empty, and should be overridden by the scenario if any extra logic is needed.

Return type:

None

init_and_populate_apps(*args, **kwargs)[source]

Initialize the apps that will be used in the Scenario.

Return type:

None

class are.simulation.scenarios.scenario_imported_from_json.benchmark_scenario.BenchmarkScenarioImportedFromJson(_initialized=False, is_benchmark_ready=False, events=<factory>, apps=None, tags=(), scenario_id='scenario_imported_from_json', seed=0, nb_turns=None, run_number=None, config=None, has_a2a_augmentation=False, status=ScenarioStatus.Draft, comment=None, annotation_id=None, hints=None, additional_system_prompt=None, start_time=1758655074.158146, duration=1000, queue_based_loop=False, time_increment_in_seconds=1, working_dir='', _initial_apps=None, tool_augmentation_config=None, env_events_config=None, gui_config=None, augmentation_data=<factory>, serialized_events=None, serialized_apps=None, apps_to_skip=None, apps_to_keep=None, execution_metadata=None, has_tool_augmentation=False, has_env_events_augmentation=False, has_exception=False, exception_type=None, exception_message=None, event_id_to_turn_idx=None, oracle_run_event_log=None, _turns_initialized=False, hf_metadata=None)[source]

Bases: ScenarioImportedFromJson

This is a special class used for importing scenarios from JSON files.

nb_turns: int | None = None
event_id_to_turn_idx: dict[str, int] | None = None
oracle_run_event_log: list[CompletedEvent] | None = None
hf_metadata: ExportedHuggingFaceMetadata | None = None
build_turn_trigger(trigger_condition, is_end_of_turn_event=<function is_send_message_to_user>)[source]

Modify the events to trigger the turns with trigger condition

build_validation_fn(validation_fn, offline_validation=False)[source]

Build a validation function for the scenario

initialize_turns(trigger_condition=None, validation_fn=None, is_end_of_turn_event=<function is_send_message_to_user>, offline_validation=False)[source]

Initialize the turns.