Scenarios

A Scenario is the fundamental unit of state in the Mutant ecosystem. It defines the context, actors, and specific objectives of a simulated test case.

Overview

When evaluating an LLM or an AI Agent, you need a deterministic starting point—a seed. A Scenario serves as this seed. It encapsulates the "who," "where," and "what," providing the necessary scaffolding for the LLM to understand the situation before any behavioral mutations are applied.

Minimal Example

Here is a dead-simple example of creating a Scenario:

from mutant.core.scenario import Scenario

scenario = Scenario(
    title="Refund Request",
    description="The user wants a refund for a broken laptop.",
)

Architecture

A Scenario is a Pydantic model that strictly enforces structure. It contains:

Parameter	Type	Required	Description
`title`	`str`	Yes	Short identifier (e.g., "Refund Request").
`description`	`str`	Yes	A deeper explanation of the scenario. This is what gets mutated.
`domain`	`str`	No	Optional domain constraint (e.g. "finance", "healthcare").
`context`	`dict`	No	Structured background details (organization, rules).
`tags`	`list[str]`	No	Labels for filtering during reporting.

Advanced Example: Guiding Variation

If you want the mutation engine to generate highly diverse and context-aware variations, you should provide structured dictionaries into the context parameter and set a domain. This grounds the engine in reality and prevents generic outputs.

from mutant.core.scenario import Scenario

scenario = Scenario(
    title="Flight Cancellation",
    description="A user's flight was cancelled due to weather and they are seeking alternatives.",
    domain="travel",
    context={
        "user_tier": "premium",
        "current_location": "Airport Terminal 3",
        "policy": "Weather cancellations only allow rebooking for next 24 hours."
    },
    tags=["customer-support", "high-priority"]
)

Why this matters for variation: When you pass this advanced scenario into the augment engine, the LLM reads the context and domain. If you apply a dimension like emotion.angry, the LLM won't just generate generic anger; it will generate anger specific to a premium user stuck in Terminal 3 facing a 24-hour rebooking policy.

By defining rich scenarios, you drastically improve the quality and realism of the generated datasets!

How It Works in the Pipeline

Ingestion: The MutationEngine receives the seed Scenario.
Analysis: During the Analyze Behavior stage, Mutant evaluates the Scenario's core behavioral traits and contextual rules.
Planning: The engine plans how to realistically alter the Scenario based on requested Dimensions, ensuring it respects the provided context.
Execution: The Scenario is mutated into a MutationCase, containing both the original parameters and the newly injected behaviors.

4. Execution Example

from mutant.core.scenario import Scenario
from mutant.core.engine import augment
from mutant.providers.ollama import OllamaProvider

provider = OllamaProvider(model="llama3.1")

scenario = Scenario(
    title="Refund Request",
    description="The user wants a refund for a broken laptop."
)

# Pass the scenario into the augment engine
dataset = await augment(
    dataset=[scenario],
    provider=provider,
    mutations_per_case=5,
    dimensions=["emotion.angry"]
)

# Save the generated mutations
dataset.save("mutations.json")
print("✅ Dataset generated and saved successfully!")

$ python run_scenario.py
[Mutant] Augmenting 1 scenarios...
[Mutant] Using Provider: llama3.1
[Mutant] Applying dimensions: ['emotion.angry']
[Mutant] Generation complete in 4.2s.
✅ Dataset generated and saved successfully!