Risks
Prompt injection in AI agents
Prompt injection is input that changes what a language model does in ways its builders did not intend. OWASP lists it as LLM01:2026, unchanged from 2025. In an AI agent the damage comes from tools: the injected text picks the next call and its arguments. We test it by planting instructions and watching which tool calls fire.
- OWASP LLM01:2026
- AML.T0051
- AML.T0051.000
Same ID in 2025: LLM01:2025
What is prompt injection?
OWASP defines a prompt injection vulnerability as one that occurs when user prompts alter the model's behavior or output in unintended ways, and ranks it first in the 2025 Top 10 for LLM Applications. The entry splits it into direct injection, where the attacker types the payload into the conversation, and indirect injection, where the payload rides in on content the model reads. MITRE ATLAS tracks the same behavior as technique AML.T0051, LLM Prompt Injection, with Direct (AML.T0051.000) and Indirect (AML.T0051.001) sub-techniques.
The OWASP entry is blunt about prevention: because models are stochastic, it is unclear whether a fool-proof defense exists. That changes the question a buyer should ask. Not "can someone inject a prompt?" but "when they do, what can the agent actually do next?"
How does prompt injection show up in agent actions?
A chatbot that gets injected says something wrong. An agent that gets injected does something wrong. The difference is the tool layer, and that is where we spend the engagement. Typical landing points we look for:
- Tools and functions. The injected text chooses which function runs and fills its arguments, for example a recipient address or a record ID.
- MCP servers. A connected server exposes more operations than the task needs, and the injection reaches one of them.
- Money. Refund, credit, discount or payout calls that take an amount the model controls.
- Data. Reads, writes and deletes on records the current user should not touch.
- Outbound channels. Email, chat, webhooks and rendered links that carry data out.
- Tenants. Calls that cross from one customer's workspace into another's.
Direct injection matters most when the attacker is a logged-in user of your product. A support customer, a free-tier signup or a disgruntled teammate can all type into the agent, so the question is whether their words can widen what the agent will do on their behalf.
Illustrative trace
Illustrative example, not a real customer finding.
What do we test for prompt injection?
We map every payload to one of the four classes in an Agent Action Pentest, so the report reads as consequences, not clever strings:
| VulnZap class | What we try |
|---|---|
| Prompt injection | Override the task through the chat box, uploaded files and fields the agent summarizes. |
| Privilege escalation | Talk the agent into admin-only tools or another user's scope. |
| Data exfiltration | Get records, secrets or the system prompt into a reply, link, email or tool argument. |
| Unsafe actions | Trigger deletes, refunds or sends that skip the confirmation a person would apply. |
Each finding in the written report carries a severity and the exact steps to reproduce it, including the input we used and the tool call it produced.
How do you mitigate prompt injection?
OWASP LLM01:2025 lists seven measures that reduce impact rather than eliminate the flaw: constrain model behavior, define and validate expected output formats, implement input and output filtering, enforce privilege control and least privilege access, require human approval for high-risk actions, segregate and identify external content, and conduct adversarial testing and attack simulations.
For agents, the two that move the needle fastest are least privilege and human approval. If the refund tool cannot exceed the order total in code, an injected "refund 2500" fails no matter how persuasive it is. We check whether those limits exist in your code or only in the system prompt.
Which frameworks and integrations are affected?
Any stack where model output becomes a function call. See how this plays out in OpenAI Agents SDK apps, where guardrails and handoffs change who sees the input, and in LangChain and LangGraph agents, where tools are plain Python functions. For money movement, read agents with Stripe access. The indirect variant has its own page: indirect prompt injection.
Agent Action Pentest: $2,500 fixed. Written report within 72 hours of access, or a full refund.
See the offer or email yaz@plaw.io
Questions
Is prompt injection the same as a jailbreak?
No. MITRE ATLAS lists LLM Jailbreak separately as AML.T0054. A jailbreak tries to get past safety behavior; prompt injection hijacks the task. In an agent, injection is the bigger worry because the hijacked task can include tool calls.
Can a system prompt stop prompt injection?
Not reliably. OWASP says it is unclear whether fool-proof prevention exists and recommends controls outside the model, such as least privilege and human approval for high-risk actions.
What do you need from us to test it?
A staging environment or test account where the agent runs with realistic tools, plus a short description of each tool. We agree scope with you at intake.
Related pages
- 02Indirect prompt injection in AI agentsHidden instructions in emails, pages, files or tool results steer an agent that reads them.
- 03Excessive agency in AI agentsThe agent holds more tools, permissions or autonomy than the task needs.
- OWASP Top 10 for LLM Applications 2026, mapped to agent testsAll ten 2026 entries mapped to what an agent pentest covers, with a 2025 to 2026 crosswalk.
- AI red teaming vs an agent action pentestBroad AI red teaming compared with a fixed-scope test of agent actions.
- Security testing for OpenAI Agents SDK appsFunction tools, handoffs, guardrails and MCP servers in the OpenAI Agents SDK.
- Security testing for AI agents with Stripe accessRefunds, payouts and restricted API keys for agents that touch Stripe.
- Security testing for AI agents in SlackChannel history, message posting and link unfurling for agents installed in Slack.
- Security testing for AI agents with GitHub accessToken type, repository permissions and write paths for agents that touch GitHub.
- Security testing for AI agents with Gmail accessRead, send, modify and filter scopes for agents that work in a Gmail mailbox.
Sources
- OWASP GenAI LLM Top 10 2026 (published 2026-08-03)
- OWASP LLM01:2025 Prompt Injection
- OWASP Top 10 for LLM Applications 2025
- MITRE ATLAS AML.T0051 LLM Prompt Injection
- MITRE ATLAS AML.T0051.000 LLM Prompt Injection: Direct
- MITRE ATLAS AML.T0054 LLM Jailbreak
- MITRE ATLAS data, release 2026.09 (mitre-atlas/atlas-data)
Last reviewed: