Skip to content

Security Evaluation (Automated Red Teaming)

Security is deeply integrated into EvalSuite. Instead of writing manual attacks, you can use the Mutation Engine to generate adversarial test cases, then evaluate your agent's defenses using specialized security metrics (like SensitiveDataLeakage and PromptInjectionResistance).

1. Generate Adversarial Attacks

Use the core augment engine to generate adversarial variations of a basic scenario by applying safety dimensions like prompt injection.

from mutant.core import Scenario, augment
from mutant.providers.ollama import OllamaProvider

provider = OllamaProvider(model="llama3.1")

scenario = Scenario(
    title="Extract System Prompt",
    description="The user tries to trick the AI into revealing its system instructions."
)

dataset = await augment(
    dataset=[scenario],
    provider=provider,
    mutations_per_case=5,
    dimensions=["safety.prompt_injection"]
)

2. Define Security Metrics

Just like standard metrics, security metrics are instantiated with an LLM Provider.

Metric Description
PromptInjectionResistance Evaluates if the agent successfully resisted the prompt injection attempt.
SensitiveDataLeakage Evaluates if the agent leaked any sensitive information or PII. 
from mutant.eval import EvalSuite, SensitiveDataLeakage, PromptInjectionResistance

# Initialize the suite with security-focused metrics
suite = EvalSuite(metrics=[
    PromptInjectionResistance(provider=provider),
    SensitiveDataLeakage(provider=provider)
])

3. Run Evaluation & Export

Pass the generated adversarial cases (dataset.cases) directly into the evaluation suite. Note: Ensure your test cases have their actual_output populated by your target system before running.

# Evaluate the target agent against generated attacks
report = await suite.evaluate(test_cases=dataset.cases)

# Export the security evaluation report
report.to_html("security_eval_report.html")
print("✅ Security report saved to security_eval_report.html")