Single AI Agent Runtime Security Testing Standards
2025-07WDTA’s new standard is part of its broader AI STR (Safety, Trust, Responsibility) certification suite. It provides a systematic approach for testing, validating, and certifying AI agent behavior.
Key features include:
-
Two-layer testing framework:
The standard combines Agent System Security Testing and Lifecycle Security Testing — evaluating not just the architecture (interface, model, tools, memory, and RAG systems), but also every phase from development to deployment and maintenance. -
Comprehensive threat modeling:
It identifies and classifies risks unique to agentic AI, such as prompt injection, memory poisoning, RAG data leakage, model stealing, and backdoor attacks — going far beyond traditional software vulnerabilities. -
Real-world attack simulations:
Includes adversarial test scenarios like jailbreaking, knowledge base poisoning, malicious tool invocation, and unauthorized memory access, ensuring agents are evaluated under realistic, high-stakes conditions. -
Quantifiable risk evaluation:
Defines clear assessment criteria and metrics (e.g., failure rates under attack, response compliance, leakage probability) to measure how securely an agent behaves in dynamic, unpredictable environments. -
Focus on single-agent autonomy:
This is the first known standard globally to focus on single-agent systems — where one AI agent acts independently using memory, tools, and models — a configuration increasingly adopted across industries.