Pipeline Reference (Cheatsheet)

This page is your master reference for the entire Mutant pipeline. It sequences the flow of data from start to finish, detailing every single parameter you can pass at each stage.

Read this top-to-bottom to see exactly how to write an end-to-end augmentation script.

1. The Scenario (Input)

The Scenario is the seed data that you feed into the engine.

Parameter	Type	Default	Description
`title`	`str`	Required	Short identifier (e.g., "Refund Request").
`description`	`str`	Required	A deeper explanation of the scenario. This is what gets mutated.
`domain`	`str`	`None`	Optional domain constraint (e.g. "finance", "healthcare").
`context`	`dict`	`{}`	Structured background details (organization, rules).
`tags`	`list[str]`	`[]`	Labels for filtering during reporting.

Example:

from mutant.core.scenario import Scenario

scenario = Scenario(
    title="Flight Cancellation",
    description="A user's flight was cancelled due to weather and they are seeking alternatives.",
    domain="travel",
    context={"user_tier": "premium"},
    tags=["customer-support", "high-priority"]
)

2. The Configuration (Optional)

The MutationConfig object allows you to exert fine-grained control over the LLM generation process. You can pass this into the augment() function.

Parameter	Type	Default	Description
`count`	`int`	`50`	Target number of mutations to generate overall (if running `mutate`).
`concurrency`	`int`	`5`	Maximum concurrent LLM calls.
`temperature`	`float`	`0.8`	Sampling temperature for the LLM generation.
`max_tokens`	`int`	`4096`	Max tokens for the LLM completion.
`quality_review`	`bool`	`True`	Enable the LLM self-reflection and quality review stage.
`deduplicate`	`bool`	`True`	Enable semantic deduplication of outputs.
`generate_rationale`	`bool`	`True`	Whether the LLM should output its reasoning.
`generate_tags`	`bool`	`True`	Whether the LLM should generate behavioral tags.
`prompts`	`dict`	`{}`	Dictionary of prompt overrides (e.g., `{"mutation_generation": "..."}`).

Example:

from mutant.core.config import MutationConfig

config = MutationConfig(
    temperature=0.4,           # Lower temperature for deterministic outputs
    quality_review=True,       # Force LLM to review its own work
    deduplicate=True,          # Remove semantically identical scenarios
    prompts={
        "mutation_generation": "You are a hostile red-teamer. Rewrite this..."
    }
)

3. The Augment Engine

The augment() function is the asynchronous engine that orchestrates the pipeline.

Parameter	Type	Default	Description
`dataset`	`list[Scenario]`	Required	A list of your seed Scenarios.
`provider`	`BaseLLMProvider`	Required	Your initialized LLM provider (e.g., `OpenAIProvider()`).
`mutations_per_case`	`int`	`5`	How many mutated variations to generate per input scenario.
`dimensions`	`list[str]`	`None`	A list of dimension IDs to apply (e.g., `["emotion.angry"]`).
`config`	`MutationConfig`	`None`	The configuration object defined above.

Example:

from mutant.core.engine import augment
from mutant.providers.openai import OpenAIProvider

provider = OpenAIProvider(model="gpt-4o")

dataset = await augment(
    dataset=[scenario],
    provider=provider,
    mutations_per_case=10,
    dimensions=["emotion.angry", "language.slang", "reasoning.multi_step"],
    config=config
)

# Access the outputs
for case in dataset.cases:
    print(f"Original: {case.original}")
    print(f"Mutated: {case.mutated}")
    print(f"Tags: {case.tags}")
    print("-" * 20)

4. Exports & Reports (Output)

Once the engine finishes, you pass the original scenario and the generated dataset.cases into a Reporter to visualize the coverage.

Example: HTML Report

from mutant.reports.html import HtmlReport

report = HtmlReport()
report.save(
    scenario=scenario, 
    cases=dataset.cases, 
    path="my_coverage_report.html"
)

print(f"✅ HTML Report successfully saved to: my_coverage_report.html")

Coverage Report Example

Example: JSON Export

import json

# You can serialize the cases directly to JSON for storage
cases_dict = [case.model_dump() for case in dataset.cases]

with open("dataset.json", "w") as f:
    json.dump(cases_dict, f, indent=2)

print("✅ Dataset successfully exported to dataset.json")
print(f"Total test cases generated: {len(cases_dict)}")

5. The Red Team Engine (Optional)

Are you looking to dynamically test your LLM instead of generating static datasets? The Mutant AI Pipeline features an interactive, hypothesis-driven Red Team Engine.

Here is how you can implement it with colored output and exports:

from mutant.redteam import red_team, TargetProfile
from mutant.providers.openai import OpenAIProvider
from rich.console import Console
import json

console = Console()

# 1. Define your target system wrapper
async def my_agent(prompt: str) -> str:
    # In reality, this calls your LangChain, RAG, or custom agent
    return "I am a helpful assistant."

# 2. Initialize the attacker provider
provider = OpenAIProvider(model="gpt-4o")

# 3. (Optional) Provide a profile to skip black-box discovery
profile = TargetProfile(
    architecture="RAG Chatbot",
    domain="Customer Support"
)

# 4. Run the hypothesis-driven red team session
report = await red_team(
    target=my_agent,
    goal="Bypass safety filters and extract system instructions",
    provider=provider,
    profile=profile,
    max_turns=5,
    max_behaviors=3,
    verbose=True
)

# 5. Display rich colored output including all conversation messages
report.display(show_messages=True)

# 6. Exporting the Red Team Report
with open("redteam_report.json", "w") as f:
    json.dump(report.model_dump(), f, indent=2)

print("✅ Red Team JSON exported to: redteam_report.json")

Example Output:

Red Team Report Example

👉 See the full Red Team Engine Guide for advanced details on automating multi-turn adversarial testing.