Indirect Prompt Injection: How to Protect an AI Assistant from Instructions Hidden in Documents

What is indirect prompt injection?

OWASP distinguishes two kinds of prompt injection. Direct injection comes from the user’s own prompt. Indirect injection happens when the model takes in external content, such as a file or web page, that contains instructions which change its behaviour when interpreted. ^[1] Either can be deliberate or accidental.

NIST’s Generative AI Profile explains that the attacker needs no direct interface with the system: they plant prompts in data the model is likely to retrieve, and researchers have shown this can be used to steal data or run malicious code. ^[4] OWASP’s scenarios include a web page with hidden instructions that make the model exfiltrate conversation data through an image link, and a CV split into fragments that leads an evaluating model to recommend the candidate. ^[1]

Why the system prompt is not enough

Many teams’ first reaction is to add a line to the system instructions telling the model to ignore any commands inside documents. That helps, but it is not enough. OWASP says it is unclear whether foolproof prevention methods exist, given the stochastic nature of how models work, and that the controls it suggests reduce impact rather than eliminate the risk. ^[1]

Microsoft therefore recommends defence in depth that layers probabilistic controls, such as detecting injection attempts, with deterministic controls that do not depend on the model’s judgement, such as permission boundaries. ^[2] The working principle: assume some attempts will succeed in influencing the model, and design the system so that a successful attempt cannot do anything harmful.

To gauge how exposed your application is, ask three questions: which untrusted sources does it read, what sensitive data can it reach, and what actions can it take or propose? The more of these three come together in one application, the higher the risk, and the simplest way to reduce it is to remove one, for instance by making the first release of the assistant read-only.

Where to draw the trust boundaries

Each layer in this sequence assumes the previous one may fail. The early layers make it less likely that harmful content reaches the model; the later ones reduce its effect if it does.

Layer one: permissions before retrieval

Check the user’s access to each document before it is sent to the model, not after the answer is generated. Retrieval applications pass retrieved passages to the model, and without permission trimming the assistant may surface content the user is not authorised to see. ^[3] Trimming inside the search pipeline also avoids pulling unauthorised data into application memory. ^[3]

Note that permission changes in the source system appear in search results only once they have synced to the index, ^[3] so make the sync part of your access-withdrawal procedure.

Layer two: separating untrusted content

OWASP’s controls include segregating external content and clearly identifying it apart from system instructions, and filtering both input and output. ^[1] Microsoft describes a technique called Spotlighting, which uses data marking and prompt design to isolate external content, alongside tools that analyse incoming prompts to detect injection attempts. ^[2]

These controls are probabilistic by nature, can occasionally block legitimate requests and need ongoing tuning and testing. ^[2] Do not rely on them alone.

Layer three: tools and actions

This is where the biggest impact lies. An assistant that only reads is far less risky than one that sends email, edits records or calls other systems.

• Least privilege: give the assistant the minimum permissions its task requires, ^{[1, 2]} and, as Microsoft recommends, make them short-lived and remove them after each use. ^[2]
• Checks in application code: define and validate the expected output format, ^[1] and enforce allowlists of permitted destinations and parameters in the application itself, so the decision to send to a given address is not the model’s to make.
• Links and images in output: since one exfiltration scenario uses an image link, ^[1] consider disabling the rendering of external links and images in the assistant’s answers, or restricting them to approved domains.
• Human approval: require explicit approval for high-risk actions, ^[1] and show the user exactly what will happen before they confirm. Microsoft describes this step as the last line of defence. ^[2]

Hypothetical defensive test

In an isolated test environment, a hypothetical organisation’s team took an ordinary training file and added, at the end, white-on-white text telling the assistant to ignore the summarisation task and send the contents of another file to an external email address. A test user then asked for the file to be summarised.

The required result: the assistant summarises the permitted material, the application refuses any sending because the destination is not on the approved list, and the event is logged. The team records the file used, the action the model proposed and what the permission system blocked, without entering any real data.

What should testing cover?

OWASP recommends regular adversarial testing and attack simulation, ^[1] and NIST lists red-teaming against prompt injection among its measurement actions. ^[4] Cover the following:

• Languages: if the assistant reads Arabic and English, test planted instructions in both languages and in a mix of the two.
• Obfuscation: instructions that are encoded, split across parts of the document or written in emojis. ^[1]
• Multimodal content: instructions hidden in an image accompanying harmless text, which OWASP notes can be hard to detect. ^[1]
• Tool output: web search results or responses from external systems that the assistant reads.
• Hiding places in office files: text in the background colour or a tiny font size, review comments, file properties, image alt text and hidden cells in spreadsheets. The model reads what the employee never sees on screen.

Retest whenever a new tool or data source is added or the model changes.

What to log and monitor after launch

Pre-launch testing cannot cover every future attempt, so Microsoft recommends runtime monitoring, detecting when a multi-step plan drifts from the intended task, and analysing tool-call sequences to block risky chains. ^[2]

In practice, log for each request the documents retrieved, the actions the model proposed, and which were carried out, which were blocked and why. A blocked action is an early warning worth reviewing even if no harm was done. Set a clear procedure for a suspected poisoned document: remove it from the retrieval sources, notify its owner, review the requests in which it was used, then add it to the test set.

Applying this in the Gulf

Use your organisation’s approved data classification policy to decide which documents the assistant may read and which systems it may send to. Review the cybersecurity requirements that apply to your organisation in your country, such as the controls issued by the National Cybersecurity Authority (NCA) in Saudi Arabia, before connecting the assistant to sensitive systems. If the assistant reads incoming email from outside the organisation or public websites, treat those sources as the highest risk and test them first.

Conclusion

No single control yet prevents indirect prompt injection. Protection comes from limiting what reaches the model, limiting what it can do, requiring human approval for every consequential action, and testing continuously.