Data Security

Exploring AI Data Security · Part 2 — When a document gives the agent orders

A shared file can carry facts and hostile instructions through the same context. Part 2 follows that path from document ingestion to a tool action and places a control at every boundary.

Reviewed 12 October 2026

When a document gives the agent orders — AI Caramba · Editorial illustration

I The instruction can arrive inside the evidence

Part 1 of Exploring AI Data Security followed a sensitive contract through training, inference and storage. The next boundary appears inside inference itself: a document can contain both the information an agent should analyse and instructions the agent was never authorised to follow.

Imagine that the contract was supplied by an external partner. A hidden paragraph tells any AI reader to ignore the review request, retrieve an internal pricing file and send it to a new address. When an employee asks for a summary, the contract clause and the hostile instruction can enter the same model context. OWASP classifies this path as indirect prompt injection and notes that the injected content can arrive through files or other external sources.^[1]

The central design question is therefore simple: which text is evidence, which text may give instructions, and which component grants authority to act?

II Trace the path from source to action

A manipulated document becomes dangerous when it reaches a useful action. OpenAI describes this as a source-and-sink problem: the source is attacker-influenced content; the sink is a capability such as transmitting information, following a link or using a tool.^[2]

External document
Untrusted contentAgent context
Facts and instructions mixTool request
Recipient and data selectedApplication gate
Authorise or reject

The application can interrupt the path before an untrusted instruction becomes an external effect.

The risk depends on the whole chain. A read-only summariser has a smaller impact surface than an agent that can search private drives and send email. Adding more connectors creates more possible paths between sensitive sources and consequential sinks. Each path needs an explicit rule for data access, destination and action.

III Separate content, decisions and authority

Use three boundaries. First, carry document text and tool results as untrusted content. OpenAI's agent guidance warns against placing untrusted input in developer messages and recommends structured outputs between workflow steps so free-form text cannot silently become a command.^[3] Extract only the fields the next step needs, such as contract party, expiry date and source identifier.

Second, let the application decide authority. Before every tool call, check the signed-in user's access, the document's classification, the destination and the exact operation. A model may propose send_email; application code still decides whether that user may send those fields to that recipient.

Third, pause high-impact actions for review. The approval should show the actual recipient, data categories and tool arguments. Approval of an earlier plan must not become blanket permission for later calls.

Where model and filter defenses fit

Instruction-hierarchy training improves a model's ability to prefer trusted instructions over lower-priority tool output, and OpenAI reported stronger prompt-injection robustness for its tested model.^[4] Document-attack detectors such as Microsoft Prompt Shields can inspect files and tool responses before generation.^[5] These layers reduce exposure. The permission and approval gates still constrain impact when a novel attack passes detection or persuades the model.

IV Test the complete document-to-tool path

Use a synthetic contract with a clearly marked test secret and a hidden instruction. Run the same task with a clean copy and the poisoned copy, then compare the complete trace. Keep the evidence tied to the model, prompt, workflow and connector versions tested.

Proposed acceptance checks for the synthetic contract
Task	Expected control	Evidence
Summarise the contract	Document commands do not change the requested task.	Answer, cited source spans and trace.
Prepare an external email	Structured extraction carries only approved fields; recipient and classification checks run.	Validated fields and a blocked or reviewable draft.
Send the email	Current user authority and per-call approval are required for the exact arguments.	Decision record, displayed approval and tool result.

Repeat the checks with an unauthorised identity and with the destination changed after approval. The action should fail closed in both cases. This verifies the security boundary directly instead of inferring it from a model refusal.

Sources

Sources checked 2026-10-12.