Skip to main content

Prompt Attacks for AI Red Teamings

Prompt attacks are inputs crafted to make a language model ignore its instructions or misuse its tools. They include direct and indirect prompt injection, jailbreaks that bypass safety training, and attempts to abuse model output or tool calls to reach other systems.

Stage: Attacks · Part of the AI Red Teaming roadmap · Last reviewed 19 November 2026

What to learn in Prompt Attacks

Prompt Attacks interview questions

What is the difference between a jailbreak and a prompt injection?

A jailbreak tries to get the model to break its own safety rules, usually by the user in the chat. A prompt injection plants instructions that override the application's intent, often through content the model reads from elsewhere.

Why is insecure output handling dangerous?

If an app passes model output straight into a browser, shell or database, an attacker can steer the model to emit script or commands that then run. Treat model output as untrusted input and encode or validate it.

Where Prompt Attacks fits in the AI Red Teaming roadmap

Prompt Attacks belongs to the Attacks stage (advanced level), which takes about 20 hours. Techniques.

Back to the full AI Red Teaming roadmap