AI Red Teaming Roadmap 2026
Test AI systems adversarially: threat models, prompt injection, jailbreaks, data and model attacks, agents and reporting.
4 stages · 8 units · Advanced · about 60 hours · Last reviewed 19 November 2026
What is a ai red teaming?
AI red teaming is the practice of attacking AI systems on purpose, with permission, to find weaknesses before real adversaries do. Testers probe language models, retrieval pipelines and agents for prompt injection, jailbreaks, data leakage and tool abuse, then document findings and help teams fix them.
Step-by-step ai red teaming roadmap
- 1
Foundations
Intermediate · about 15 hours
Threat landscape.
- Concepts
AI security concepts describe what makes AI systems attackable and how to reason about it. Threat modelling maps assets, entry points, attackers and impact for a model-backed system, so testing focuses on realistic risks such as untrusted input reaching a model that holds powerful tools.
- Rules of Engagement
Rules of engagement are the written agreement that defines what a red team may test, how, when and with what data. They set scope, authorisation, off-limits systems, reporting channels and ethical limits, protecting both the tester and the organisation from harm or legal trouble.
- Concepts
- 2
Attacks
Advanced · about 20 hours
Techniques.
- Prompt Attacks
Prompt attacks are inputs crafted to make a language model ignore its instructions or misuse its tools. They include direct and indirect prompt injection, jailbreaks that bypass safety training, and attempts to abuse model output or tool calls to reach other systems.
- Data and Model
Data and model risks are attacks aimed at what feeds and defines an AI system: poisoned training or retrieval data, leakage of private information, model extraction and weaknesses in RAG and agent pipelines. Red teamers test how data flows and what permissions the system holds.
- Prompt Attacks
- 3
Testing Practice
Advanced · about 15 hours
Tools and automation.
- Tooling
Red teaming tooling covers the open-source scanners and frameworks used to probe models, plus the manual methods that find what tools miss. Automation runs large sets of attack prompts and scores replies, while manual testing explores creative, multi-turn and context-specific attacks.
- Evaluation
Measuring risk in AI red teaming means turning attack results into numbers and judgements: attack success rate, severity, and coverage across risk categories. Graders, human review and fixed test sets let teams compare versions and decide whether a system is safe enough to release.
- Tooling
- 4
Defence and Reporting
Advanced
Close the loop.
- Defences
Defences are the mitigations that reduce AI risk after weaknesses are found. They layer input and output filtering, hardened prompts, least-privilege tools, isolation of untrusted content, monitoring and human approval, since no single control stops every attack on a language model.
- Reporting
AI red team reporting is the write-up that turns test activity into action: clear findings with reproduction steps, evidence, severity and recommended fixes. It also feeds governance, helping organisations track AI risks, assign owners and show due diligence to leadership and auditors.
- Defences
Frequently asked questions
Ready to practise for interviews?
Practise ai red teaming questions with an AI mock interviewer and get feedback on your answers.