Evaluation for AI Red Teamings
Measuring risk in AI red teaming means turning attack results into numbers and judgements: attack success rate, severity, and coverage across risk categories. Graders, human review and fixed test sets let teams compare versions and decide whether a system is safe enough to release.
Stage: Testing Practice · Part of the AI Red Teaming roadmap · Last reviewed 19 November 2026
What to learn in Evaluation
- Calculate attack success rate across repeated runs of the same prompt
- Define a severity scale tied to real impact
- Use a classifier or judge model to grade harmful output, and check it manually
- Track coverage across risk categories
- Compare results before and after a mitigation
Evaluation interview questions
What is attack success rate?
It is the share of attack attempts that achieve the harmful goal, such as the model revealing restricted content. Because outputs vary, run each attack multiple times and report the rate with the number of trials.
How do you rate the severity of an AI finding?
Consider the impact if exploited, such as data exposure or unauthorised actions, how easy the attack is and who can attempt it. A trivial attack that triggers a payment is more severe than a rare jailbreak that yields mild text.
Where Evaluation fits in the AI Red Teaming roadmap
Evaluation belongs to the Testing Practice stage (advanced level), which takes about 15 hours. Tools and automation.
Other units in this stage
Back to the full AI Red Teaming roadmap