The Alignment Fallacy
Most AI roadmaps bet on a single premise: train a model safe enough to trust in production. Better alignment, better guardrails, one more version, and the model behaves.
That bet is wrong. Even a perfectly aligned model cannot tell you who used it, on what data, under whose policy, or hand you an auditable record. Those are deployment facts, not model facts. They live entirely outside the weights.
The gap between model behavior and deployment governance is where real risk sits. Security engineering identified this trap in 1972. A US Air Force study defined the reference monitor — the component that decides whether an action is allowed — and set three conditions for trusting it: tamperproof, invoked on every access, and small enough to be completely verified.
A modern frontier model fails all three.
Why the Model Can't Govern Itself
Not tamperproof. Prompt injection, identified in 2022, works because instructions and data share the same text channel. A document the model reads can overwrite its instruction. Four years later, OWASP says it's unclear whether fool-proof prevention exists. A 2026 paper argues agents may always be vulnerable. A control you can rewrite with the input it inspects is no control.
Not reliably invoked. Give a model tools and it becomes a textbook confused deputy — a 1988 term for a trusted program tricked by an untrusted caller into misusing its authority. The model can't distinguish an instruction from data.
Not verifiable. You can't inspect billions of opaque parameters the way you audit a small enforcement kernel.
Fail these three tests and the conclusion isn't "align harder." It's that the model cannot be the enforcement mechanism. The enforcement must live elsewhere.
Safety vs. Governance
Safety and alignment ask whether a model tends to behave well — a disposition in the weights. Governance asks who used which model, on what data, under whose policy, and with what auditable record. That's a deployment property. Training can shape the first. It cannot supply the second.
Treating them as one problem is the category error underneath most AI risk conversations.
The Growing Surface
This lands on everyone who deploys AI, and it's expanding for two reasons.
First, models gain agency. When a model only produced text, a bad interaction meant a bad answer. Now models take actions through tools and protocols like MCP. Every tool call is a fresh decision about who acts, on what data, under whose authority. Those decisions multiply with every task and user. OWASP's 2026 tracking finds most agentic projects are coding agents, and prompt injection maps to six of the ten risks on its agentic top-ten list.
Second, accountability is now legal. California's SB 53 and the EU AI Act place duties on deployers. Data-protection law already holds organizations responsible for how personal data is used. When a regulator asks what your AI did, "we used an aligned model" is not an answer. A vendor's safety report won't tell them who inside your company sent which data to which model last Tuesday.
Public benchmarks don't help either. StrongREJECT, HarmBench, and AgentHarm grade the model on generic prompts, not your users, your data, or your policy. A high score is a model property, not an audit trail.
The Fix: A Layer Around the Model
If the model can't govern itself, the answer is a thin control layer that treats the model as an untrusted component and mediates everything it tries to do. This is the 1972 reference monitor rebuilt for an agent.
Here's how it works:
- The model reads its input and proposes an action — call this API, write this file, send this message.
- That proposal leaves the model as a request, not a command. It passes to a small policy engine outside the model, sitting between the agent and the resource it wants to reach.
- The engine checks the action against your actual policy and the narrow capabilities granted for this task.
- If the action clears, an enforcement point performs it; if not, it's refused. Either way, who acted, through which model, on what data, under which policy, and what was decided is logged.
This restores the three missing properties:
- Tamperproof: The policy lives in a small external engine, not in text the model can be argued out of.
- Always invoked: Nothing reaches a real resource without passing through it.
- Verifiable: An enforcement point small enough to read is one you can actually trust, unlike billions of opaque weights.
The strongest research defenses already work this way. CaMeL, from Google DeepMind and ETH Zurich, achieves provable security by containing the model exactly like this. It holds even when the underlying model stays vulnerable. The guarantee lives in the architecture, not the weights.
What This Means for Developers
Alignment is necessary and improving. It's simply the wrong layer for the question regulators, auditors, and users are asking. The answer has been on the shelf since 1972, and the industry keeps re-deriving it the expensive way.
The next model will be safer than this one. It still won't know who used it, on what data, or under whose policy. Closing that gap is the work.
Act now: Audit your AI deployments. If you rely solely on model alignment for safety, you have no governance. Build a thin enforcement layer — policy engine, audit log, access controls — around every model you put in production. Start with the OWASP Agentic Top 10 and the CaMeL architecture as reference.
