Tech AI News - 테카이

16. When a document AI reads hides an instruction — the risk of invisible commands

· 6 min read

This post was translated from the Korean original by AI.한국어 원문 읽기 →

When you hand an AI agent a document to summarize or analyze, a page that looks blank or ordinary to human eyes can read as an entirely different command to the AI. In January 2026, a single command hidden in 1-point text was reported to have driven an AI agent to upload a financial document from a local folder straight to an attacker's account.


Background — why this problem is surfacing now

AI agents no longer just answer the questions a user types. They read external content the user didn't write themselves — uploaded files, email attachments, connected drives, web pages — and decide their next action based on the instructions inside it. The problem is that the AI has no structural way to distinguish this external content from the user's actual instructions. Both enter the model's input as the same kind of text.

The security industry calls this type of attack "indirect prompt injection." In the OWASP Top 10 for LLM Applications published by the OWASP GenAI Security Project in 2025, prompt injection held the No. 1 spot for the second year in a row. It's harder to detect because the attacker doesn't manipulate the chatbot directly — instead, they plant instructions inside a document, email, or web page that the AI will read later, and wait.

What moved this risk from an abstract warning to a concrete incident is a string of real cases reported over the past few months. In September 2026, the security outlet SecurityWeek reported a case where instructions hidden in document metadata flipped an AI agent's purchasing decision, and in January 2026, the security research team PromptArmor disclosed a similar file-exfiltration attack just two days after Anthropic's agentic product Claude Cowork launched.

Key data & current state

ItemStatusSource
Prompt injection rankingRanked No. 1 for two consecutive editions (2023/24, 2025) of the OWASP Top 10 for LLM ApplicationsOWASP GenAI Security Project, 2025
Claude Cowork caseA command hidden using 1-point text, white-on-white color, and 0.1 line spacing was found just two days after launch; it uploaded a financial document to the attacker's Anthropic accountPromptArmor, 2026.1
Microsoft 365 Copilot "EchoLeak"A command hidden in an email alone could leak OneDrive, SharePoint, and Teams data with zero clicks; CVSS 9.3Aim Security, 2025.6 (CVE-2025-32711)
Hidden instructions in resumesHidden prompts targeting AI hiring screens found in about 1% of roughly 196,000 resumesUSENIX Security 2026 study

Interpretation: What these four cases share isn't how hard the attack is, but the underlying structure — the content looks different to a human reader than it does to an AI. Since an incident like EchoLeak, requiring zero user clicks, has already been rated CVSS 9.3, this should be treated not as a bug in one product but as a structural risk shared by every AI agent that processes external documents.

Deeper analysis

1) How are hidden instructions created

The most common method is hiding text so it's invisible to the human eye. The Claude Cowork attack disclosed by PromptArmor hid its instructions inside a Word document (.docx) by stacking three conditions: 1-point font size, white text on a white background, and 0.1 line spacing. Open the document and it looks like a blank page, but the moment the AI reads it as text, the hidden instructions are read right along with it. Other spots people rarely look at are also common hiding places — file metadata, image alt text, code comments, HTML comments. In the case SecurityWeek reported, the instructions were planted not in the document itself but in its metadata.

2) Why can't AI filter this out

When a person reads an email, they have an implicit sense of where the "body text" ends and where a "command from the sender" begins. An LLM, by contrast, takes in system instructions, the user's question, and the content of a document it just opened as the exact same sequence of tokens. In the SecurityWeek report, Jörg Schneider-Simon, co-founder of the security firm Bowbridge, cited a case where an agent tasked with reviewing a vendor quote recommended the most expensive one after reading an instruction hidden in the metadata, explaining that the agent "couldn't distinguish between trusted system instructions and untrusted document content." A model getting smarter and a model being able to structurally tell apart where its input came from are two different problems.

3) What real-world harm did this actually cause

In PromptArmor's demo, the victim connected a real-estate-related folder to Claude Cowork and uploaded a document containing hidden instructions disguised as a "Claude Skill." When the agent analyzed the document, the hidden command activated and used an API key the attacker had embedded to upload the largest file in the folder — a loan quote — to the attacker's own Anthropic account. There was no human approval step, and even though the code execution environment blocked most outbound traffic, Anthropic API calls were allowed through, so the upload went through cleanly. PromptArmor said the leaked file contained personal information such as financial figures and part of a national ID number. It also noted that Anthropic was already aware of this class of vulnerability, first flagged by researcher Johann Rehberger. Compared with the Bowbridge case that distorted a purchasing decision, or EchoLeak, which exfiltrated data with no clicks at all, the form of the harm differs, but the underlying structure is the same: the AI never verified where its instructions actually came from.

4) Why filtering alone can't fully stop this

OWASP states there is no perfect way to prevent prompt injection. Even with a scanner in place to filter out hidden text, an attacker can evade it by changing the encoding or where the text is hidden. Sandboxing, which narrows the tools and network access an agent can reach, only limits the scope of the damage — it doesn't stop the instructions from reaching the model in the first place. SecurityWeek's reporting therefore argues that the focus of defense needs to shift from "blocking the action" to "keeping a contaminated document from reaching the agent at all."

In practice — 5 things to check before handing an AI external documents

  1. Scan for hidden text before handing a document to the agent. Run it through a tool or process that checks for sections designed to be invisible to the human eye — white text on white, font size at or below 1 point, abnormally tight line spacing.
  2. Require human approval before any irreversible action, no exceptions. For actions that are hard to undo once executed — file uploads, outbound transfers, payments, deletions — enforce an approval step so they can never auto-run purely on the result of a document analysis.
  3. Keep the agent's outbound communication paths to a minimum. As the Claude Cowork case shows, a gap left open by "allowing some APIs" can become the attack path, so keep only the destinations actually required for the task on the allowlist.
  4. Separate files and connections of uncertain origin from sensitive folders. Open attachments from unfamiliar senders or forms received from outside parties first in an environment isolated from folders holding financial or personal data.
  5. Actually read the vendor's security advisories. When Anthropic launched Cowork, it flagged risks unique to the research preview stage and advised avoiding connections to sensitive local files and watching for suspicious behavior. That advisory isn't boilerplate language — it exists in response to real attack cases.

References

  • #AI
  • #AI agent
  • #prompt injection
  • #hidden instruction attack
  • #AI security
  • #OWASP