Build an evaluation set from real workflow variation.
Include routine cases, ambiguous inputs, missing data, conflicting instructions, tool failures and scenarios that should require human escalation. The evaluation should make important failure modes visible before broad release.
Measure several layers separately.
- Task quality: Did the workflow reach the correct business result?
- Tool selection: Did the agent choose only allowed and appropriate actions?
- Argument quality: Were tool inputs correct and complete?
- Policy behavior: Did the system refuse or escalate when required?
- Operational performance: What were latency, cost and exception rates?
- Human acceptance: Can the responsible user understand, review and operate the system?
Use acceptance criteria before expansion.
A single “accuracy” number rarely captures an agent system. Define thresholds that reflect the consequences of the workflow, and keep individual failure categories visible so improvements do not hide regressions elsewhere.
Related service: Managed AI & Optimization.
References and further reading
For broader testing, evaluation, verification and validation principles, see the NIST AI Resource Center and the NIST Generative AI Profile.