Data and Model for AI Red Teamings
Data and model risks are attacks aimed at what feeds and defines an AI system: poisoned training or retrieval data, leakage of private information, model extraction and weaknesses in RAG and agent pipelines. Red teamers test how data flows and what permissions the system holds.
Stage: Attacks · Part of the AI Red Teaming roadmap · Last reviewed 19 November 2026
What to learn in Data and Model
- Explain how poisoned documents in a RAG index can steer answers
- Probe a model for memorised or leaked training data
- Test whether retrieval respects per-user access controls
- Assess an agent for excessive permissions across its tools
- Describe model extraction and what limits it
Data and Model interview questions
What is data poisoning?
Data poisoning means inserting malicious or misleading samples into training, fine-tuning or retrieval data so the system behaves wrongly later. In RAG it can be as simple as publishing a document that contains hidden instructions.
How do you test an agent for excessive agency?
List every tool and permission the agent holds, then try to make it use them outside its intended purpose through injected instructions. Findings include actions that need no confirmation, broad credentials and tools with write access.
Where Data and Model fits in the AI Red Teaming roadmap
Data and Model belongs to the Attacks stage (advanced level), which takes about 20 hours. Techniques.
Other units in this stage
Back to the full AI Red Teaming roadmap