Examples

Here are some robust, copy-pasteable examples for common engineering workflows using Mutant.

---

1. Batch Dataset Augmentation

This is the most common use case for Data Engineers. You take an array of baseline scenarios (perhaps loaded from a CSV or JSON file) and blast them into hundreds of behavioral variations.

OpenAIOllama (Local)

from mutant.core.scenario import Scenario
from mutant.core.engine import augment
from mutant.providers.openai import OpenAIProvider

# 1. Initialize your LLM Provider
provider = OpenAIProvider(model="gpt-4o")

# 2. Define your base scenarios
scenarios = [
    Scenario(
        title="Refund Request",
        description="A user wants to return a broken product they bought 3 weeks ago.",
    ),
    Scenario(
        title="Account Lockout",
        description="User is locked out of their account because they forgot their password.",
    )
]

# 3. Run the augmentation pipeline concurrently
dataset = await augment(
    dataset=scenarios,
    provider=provider,
    mutations_per_case=5, # Generates 5 mutations per scenario
    dimensions=[
        "emotion.frustrated", 
        "language.broken_english", 
        "reasoning.multi_step"
    ]
)

# 4. View results
print(f"Total scenarios generated: {len(dataset.cases)}")

for case in dataset.cases:
    print(f"Original: {case.original}")
    print(f"Dimension Applied: {case.dimension_name} ({case.severity.value})")
    print(f"Mutated: {case.mutated}")
    print("-" * 40)

from mutant.core.scenario import Scenario
from mutant.core.engine import augment
from mutant.providers.ollama import OllamaProvider

# 1. Initialize local Ollama Provider
provider = OllamaProvider(model="llama3.1")

# 2. Define your base scenarios
scenarios = [
    Scenario(
        title="Refund Request",
        description="A user wants to return a broken product they bought 3 weeks ago.",
    ),
    Scenario(
        title="Account Lockout",
        description="User is locked out of their account because they forgot their password.",
    )
]

# 3. Run the augmentation pipeline concurrently
dataset = await augment(
    dataset=scenarios,
    provider=provider,
    mutations_per_case=5, # Generates 5 mutations per scenario
    dimensions=[
        "emotion.frustrated", 
        "language.broken_english", 
        "reasoning.multi_step"
    ]
)

# 4. View results
print(f"Total scenarios generated: {len(dataset.cases)}")

for case in dataset.cases:
    print(f"Original: {case.original}")
    print(f"Dimension Applied: {case.dimension_name} ({case.severity.value})")
    print(f"Mutated: {case.mutated}")
    print("-" * 40)

Example Output:

---

2. Generating HTML Coverage Reports

After generating a dataset, it is crucial to analyze its behavioral diversity. Mutant provides a built-in interactive HTML reporter.

from mutant.reports.html import HtmlReport
from mutant.core.scenario import Scenario
from mutant.core.mutation import MutationCase, MutationSeverity, MutationCategory

# Assuming you already have a Scenario and a list of generated MutationCases
# from the augment() pipeline...

report = HtmlReport()

# Save a self-contained HTML file to disk
report.save(
    scenario=scenario, 
    cases=dataset.cases, 
    path="my_evaluation_dashboard.html"
)

print("Dashboard generated successfully! Open my_evaluation_dashboard.html in your browser.")

---

3. Strict Quality Control

If you are generating datasets for rigorous red-teaming or benchmark creation, you can enforce strict quality control and semantic deduplication.

from mutant.core.engine import augment
from mutant.core.config import MutationConfig

# We pass a highly constrained config
strict_config = MutationConfig(
    count=20,
    quality_review=True,       # LLM explicitly reviews its own output
    quality_batch_size=5,      # Batches reviews to save tokens
    deduplicate=True,          # Removes semantically identical mutations
    temperature=0.4            # Lower temperature for deterministic outputs
)

dataset = await augment(
    dataset=[scenario],
    provider=provider,
    config=strict_config
)