Multimodal AI: New Senses, New Attack Surface
Last Updated: September 10, 2026 | By Mihail Sebastian | Agentic AI
Frontier AI models see images, hear audio, and read documents by default. Every new sense is a new input channel, and every input channel can be attacked.
Two years ago, a model that could look at a photo was a product announcement. Today it is the baseline. GPT, Claude, and Gemini all take images, audio, and documents as naturally as they take text, and nobody writes a headline about it.
In practice that means you photograph a paper form instead of retyping it, talk to the assistant while your hands are busy, and drop a chart into the chat instead of describing it. Multimodal AI is the technical term, but users just call it “the assistant,” because handling one input type at a time now reads as a limitation.
This article is not about the capability. It is about what the capability did to your security model, because every new sense the model gained is a new door into it.
Every Sense Is an Input Channel
Here is the uncomfortable symmetry: everything the model can perceive, an attacker can craft. A text-only model had one input channel to defend. A model that sees, hears, and reads files has several, and each one carries instructions as readily as it carries data.
Prompt injection was named for text, but nothing about it is text-specific. The attack works because the model draws no firm line between content to analyze and instructions to follow, and that weakness travels with the model into every modality it learns.
Images. An instruction can sit in an image as plainly visible text: a sign in a photo, a caption in a screenshot, a line in a scanned document. It can also be less obvious than that. Researchers have shown that images can be perturbed in ways a human eye does not register but that steer the model’s behavior, which means “the image looks clean” is not a security check.
Audio. A voice note or a meeting recording is another stream the model treats as input. Spoken instructions embedded in audio the model transcribes or interprets reach it the same way typed ones do, without passing through anything that filters typed text.
Documents. A PDF is the richest carrier of all: it holds visible text, invisible text, embedded images, and metadata. A document pipeline that feeds inbound files to a model gives everyone who can send you a file a channel into that model.
The defensive habit of scanning prompts for suspicious strings was always a partial answer. Against multimodal input it barely applies, because the payload may be pixels or waveforms rather than words.
Why Agents Raise the Stakes
If the model only describes what it sees, a successful injection produces a wrong description. Annoying, sometimes embarrassing, rarely an incident. The stakes change when the model is an AI agent that acts on what it perceives.
Consider a browser agent that navigates by reading screenshots of the page. A hostile page can render instructions anywhere in that screenshot, and the agent reads them with the same trust it gives the user’s request. The page has become a prompt, and the attacker controls the page.
Or take a back-office agent that processes incoming invoices and files them through tool calling. One poisoned PDF in the queue can carry an instruction the agent executes with its own permissions: reroute this, forward that, approve the exception. The blast radius is whatever the agent is allowed to do, which is why over-permissioned agents, what security frameworks call excessive agency, turn a crafted image into a real loss.
The pattern to internalize: multimodality multiplies the ways instructions get in, and agency multiplies what those instructions can cause. An organization deploying multimodal agents has both multipliers running at once.
What Defenders Do
None of this argues for turning the capability off; the productivity is real and your competitors are keeping it. It argues for treating new senses with the discipline you already apply to old inputs.
Start by noticing that you probably enabled this surface without deciding to. Many organizations picked up image and document input as a side effect of a model upgrade, and their threat model still describes the text-only system they reviewed a year ago. The first defensive act is updating the paperwork to match what the system now perceives.
Treat non-text input as untrusted, always. A screenshot, a voice memo, and an uploaded PDF deserve exactly the trust you would give a string pasted from an unknown website: none. Design the system so that content from these channels can request things but never silently authorize them, and put human approval in front of the actions that would hurt.
Red team in every modality you accept. If your agent reads images, your adversarial testing must include hostile images; the same goes for audio and documents. Agentic red teaming already maps every channel an agent reads as attack surface, and multimodal channels belong on that map from day one, not after the first incident.
Log what the model saw. When an agent misbehaves, the first investigative question is what input drove it, and with multimodal systems the answer may be an image nobody kept. Agent observability has to capture the actual inputs, all of them, or your incident review starts from a blank page.
Constrain before you trust. Least-privilege tool access and confirmation gates are not multimodal-specific, but they are what stands between a poisoned input and a consequence. The cheaper the injection is to attempt, and images are cheap, the more the permission boundary matters.
The Honest Capability Note
One more thing defenders should hold onto: these models still make unforced visual errors, with no attacker involved. They misread chart axes, mangle wide tables, drop rows during extraction, and transcribe a 7 as a 1 in a low-quality scan.
That failure mode is quieter than an attack and more common. A finance team that pipes extracted figures straight into a report is not being attacked; it is trusting a reader that gets things wrong at a low but real rate.
The rule is the same one auditors have always used: any extraction that feeds a consequential decision gets verified against the source by a person. Multimodal AI earned its place as the default interface. It has not earned exemption from the controls we put on every other system that reads untrusted input and sometimes reads it wrong.
