Advanced · Safety
Authority Confusion
Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.
TL;DR. An attack class where an attacker tricks an AI agent about who issued an instruction — exploiting gaps between the humans in the loop and the parties the agent actually listens to.
Technical Definition
An attack class where an attacker tricks an AI agent about who issued an instruction — exploiting gaps between the humans in the loop and the parties the agent actually listens to.
How it works
Authority confusion formalizes a failure mode that surfaced repeatedly as agents gained tools and multi-party context: the agent cannot reliably tell whose authority an instruction carries. A support agent reading a ticket thread may treat a customer's 'ignore previous policy' as an admin command; an email agent may act on instructions embedded by a sender. The [un]prompted II glossary (October 2026) grouped these under this name to drive a design response: explicit authority levels attached to context items, signed or channel-verified instructions for sensitive actions, and hard separations between what users may request and what operators may command. It pairs closely with context provenance — knowing where text came from is what lets you know who it speaks for.
Related Concepts
- Cross-Site Prompting — An attack where content on one trusted website injects instructions into an AI agent browsing a different, victim site — the prompt-injection analogue of cross-site request forgery.
- AI Red Teaming — A structured process of rigorously testing an AI system, often by adversarial attack simulations, to identify vulnerabilities, biases, and potential for misuse.
- Prompt Injection — An attack where untrusted input (a webpage, email, tool output) contains instructions that hijack an LLM's behavior in an agentic context.
- Computer Use — An agent capability where an LLM controls a computer GUI by viewing screenshots and emitting mouse/keyboard actions, automating arbitrary software.