Prompt injection occurs when an AI system interprets external text, a document, an email, a web page or other content as an instruction and begins acting against the user’s or organisation’s intended objective. An attack may change an answer, expose sensitive information, influence a decision or manipulate an AI agent into using connected tools in an unauthorised way.
The risk cannot be solved by writing a “stronger” system prompt. Protection needs several layers: untrusted content must be separated from instructions, the model must receive minimal privileges, tool calls must be verified by deterministic code, consequential actions must require human approval, and the system must be continuously tested and monitored.
What prompt injection means
A large language model receives information from several sources in the same context: system rules, the user’s request, conversation history, search results, documents retrieved by a RAG system and responses from tools. A person understands that the text inside an email is data, not a command to the email reader. That boundary is less reliable for a model because both rules and content are usually expressed in natural language.
OWASP LLM01:2025 defines prompt injection as input that changes model behaviour or output in unintended ways. The malicious instruction may be visible or hidden inside content the model can process. Retrieval-augmented generation and additional model training may improve relevance, but they do not remove the vulnerability by themselves.
Prompt injection is not the same as SQL injection or command injection. In a conventional injection vulnerability, a malicious string is interpreted as a database or operating-system command. Prompt injection first manipulates the model’s decision. Dangerous consequences arise when the surrounding application trusts that decision and passes it to a database, API, email service, file system or another tool without adequate verification.
Direct and indirect prompt injection
| Type | How malicious content enters the system | Typical example |
| Direct prompt injection | The attacker writes directly into the AI interface or API input | A customer tries to make a support assistant ignore its rules and reveal another customer’s data |
| Indirect prompt injection | The AI reads external content controlled by an attacker | An agent processes an email, PDF, web page or RAG document containing a hidden malicious instruction |
| Multimodal injection | The instruction is embedded in an image, document layer or another data format | A model analyses an image containing content designed to alter the task but difficult for a person to notice |
A direct attack is usually easier to associate with a particular user and request. An indirect attack is harder to detect because a legitimate employee may bring the hostile input into the system. The employee asks the AI to summarise a supplier document or read an incoming email without knowing that it contains an instruction addressed to the agent.
Jailbreaking is related, but usually refers to attempts to bypass the model provider’s own safety restrictions. For an enterprise risk assessment, the more useful question is broader: can untrusted input alter the application’s intended workflow, gain access to data or trigger an action in another system?
How an attack becomes a business incident
A successful incident usually requires a chain of weak controls rather than a single failure.
Untrusted content enters the AI context. It may come from a user, public website, email, uploaded file, customer comment or RAG knowledge base.
The model interprets the content as an instruction. The malicious text competes with system rules and the user’s real objective.
The model can access valuable data or tools. It may read a CRM, search documents, initiate a payment, send an email or modify a record.
The application trusts the model’s output. A tool call, link, SQL fragment or message recipient is not independently checked.
The action proceeds without meaningful approval or control. The impact becomes external: data is disclosed, a record is changed, a message is sent or a decision is distorted.
The severity of prompt injection therefore depends on four factors: the volume of untrusted input, the sensitivity of accessible data, the agent’s authority to act and the absence of independent checks. A chatbot that can only answer from public information and cannot change anything belongs to a different risk class from an agent with access to email, documents and payment systems.
Where enterprise systems face the highest risk
RAG assistants and internal knowledge bases
Retrieval-augmented generation locates relevant document fragments and adds them to the model’s context before generating an answer. If external files, uncontrolled comments or unreviewed imports can enter the knowledge base, an attacker may insert an instruction into a source that is later retrieved for an apparently unrelated query.
Retrieval does not make a document trustworthy. The system should retain the source identity, access level and trust status, and the answer must respect the same user permissions as the original data source.
Email, document and browsing agents
These agents are designed to read content controlled by third parties. A hostile instruction may appear in an email body, attachment, document metadata or web page. If the same agent can also search other files and send messages, indirect prompt injection gains a path from untrusted input to a sensitive action.
Customer support and self-service
Anyone can systematically probe a public assistant. Risk rises when the assistant can view customer profiles, orders or discounts and can change account state. Authorisation must never be delegated to the model. Every data request and action must be checked on the server against the real user’s identity and permissions.
Automated document assessment
AI systems may classify CVs, invoices, contracts, procurement bids or customer submissions. An instruction placed in a document may try to manipulate a score, priority or summary. In these workflows, document content is evidence to be evaluated, not a trusted definition of the task.
Why system prompts and filters are not enough
A system prompt is useful for defining behaviour, but it is not a security boundary. Attackers can change wording, split instructions across several inputs, switch languages or hide content inside another format. A blacklist recognises known phrases, not hostile intent.
A separate safety model or classifier can fail as well. It may reduce the number of attacks but must not become the only barrier before a consequential action. Secure architecture assumes that some malicious inputs will reach the main model and some harmful outputs will pass a filter.
API keys, passwords and other secrets must never be stored in the system prompt. Instructions can guide workflow, but their confidentiality is not equivalent to a secrets vault. Credentials should be managed in a dedicated store, and only server-side code should invoke a tool after checking permissions. Learning the structure of a system prompt must not give an attacker additional access.
Current agent-hijacking research supports this position. A 2026 NIST analysis of a large-scale red-teaming competition covered more than 250,000 attack attempts against 13 frontier models, and at least one successful attack was found for every target model. The practical conclusion is not that organisations should avoid AI, but that model robustness must not be treated as system security.
Protect the whole system, not only the model
1. Build a threat model
List every source that can supply content, the data the model can see, the tools it can call and the worst plausible outcome. Distinguish public, partner, employee and system-generated content. An “internal document” is not automatically a trusted instruction.
2. Separate instructions from data
Mark external content as untrusted and place it in a clearly bounded data field rather than concatenating it with system rules. Tell the model that commands found inside a document must not be followed. This reduces risk but does not replace permission and tool restrictions.
3. Apply least privilege
An agent should receive only the data and functions needed for the current task. An agent summarising email does not need permission to send messages or open a finance folder. Use separate, short-lived and narrowly scoped credentials instead of an employee’s complete session.
Separating read and write agents is safer still. One component produces a structured proposal; another receives only a verified command with explicit parameters.
4. Verify tool calls outside the model
The model may propose an action, but normal application code must authorise it. The server checks the user, allowed function, target object, recipient, amount, domain and other business rules. Use strict schemas with enumerated values, length limits and safe defaults instead of unrestricted text.
AI-generated SQL, commands, HTML, URLs or code must never be executed automatically without context-appropriate validation and isolation. Model output is untrusted input to the next component.
5. Require human approval for consequential actions
Approval must show the exact action, recipient, data involved and irreversible consequences. A generic “Continue” button creates an illusion of control. Payments, deletion, permission changes, external messages and sharing confidential files are typical actions that should not run autonomously.
6. Restrict execution and data egress
Run code and file processing in an isolated environment with resource, network and file-system limits. Allow connections only to necessary domains and prevent arbitrary external URL requests. If an attack manipulates the model, containment reduces its ability to reach sensitive resources or exfiltrate data.
7. Validate the final output
Before a response reaches a user or another system, check for signs of data disclosure, unauthorised links, invalid structure, personal data and deviation from the task. For structured processes, compare the response with source material and business rules. A classification or summary should remain traceable to the documents used.
Logging, monitoring and incident response
Without logs, an organisation may see a wrong output but be unable to determine which content, model, tool and permission caused it. Record a request identifier, input sources, retrieved documents, policy decisions, tool calls, approvals, errors and final status. Mask sensitive values in logs and restrict access to them.
Alerts can identify unusual tool calls, repeated blocked requests, access to unexpected data categories, new external domains and sudden changes in behaviour. The organisation must be able to disable a tool, revoke an agent’s credentials, isolate a document source and preserve evidence quickly.
The incident plan should name who can suspend the AI feature, who assesses possible data exposure, how responsible teams are informed and how security is retested after remediation.
Red-teaming and regression tests
Prompt injection defence is not a one-time audit. Models, system prompts, RAG documents, integrations and attacker techniques change. Before launch and after material changes, run attack scenarios that reflect the organisation’s actual data sources and tools.
The test set should cover direct attempts to override rules, instructions inside external documents, payloads split across inputs, other languages, malformed tool parameters, unauthorised data requests and attempts to trigger external actions without approval. Evaluate more than the model’s text. Verify whether the system blocked the tool, protected data, created the required log and informed the user correctly.
Keep failed attacks and successfully blocked cases as regression tests. A model or prompt update must not silently remove a previously working protection.
What to ask an AI solution provider
An organisation needs more than model-accuracy information. The provider should explain:
how system, user and external-source content is separated;
which permissions the agent account has and whether they can be limited per function;
which actions always require approval;
how tool parameters and model outputs are validated;
which logs, alerts and emergency shutdown controls are available;
how prompt injection testing and security updates are performed;
how the organisation can investigate an incident and export relevant evidence.
“Our model has safety filters” is not a sufficient answer when the system can access confidential data or take actions.
A practical implementation sequence
Start with one clearly bounded process in read-only mode. Inventory data sources, label untrusted content and create a permission matrix. Add server-side policies, structured tool-call validation, logs and adversarial tests. Introduce write operations or external communication only when approvals, containment and an emergency stop mechanism are in place.
A simple decision filter is useful: if the organisation cannot safely contain the worst action a compromised agent could take, that action should remain under human control.
Conclusion
Prompt injection exploits the difficulty AI systems have in reliably separating instructions from data. It becomes dangerous when a model receives broad context, sensitive data and permission to act while the surrounding application trusts its output without independent verification.
The security goal is not to promise that a model will never be deceived. It is to ensure that a deceived model cannot cross access boundaries, execute an unauthorised action or exfiltrate data without detection. That is why secure business process automation begins with permissions, data flows, validation and control points—not only with model selection.