The Problem: Prompts Are Not Contracts
A user complained that their AI-generated newsletter used the word "delve" twice. That word had been in Ozigi's system prompt since day one. Gemini 3 used it anyway.
This isn't a bug. It's structural. Three things break the prompt-as-contract assumption in production:
- Attention dilution – The longer your system prompt, the less weight any single rule carries. By token 1,800, "don't use delve" competes with thousands of other instructions.
- Regression to training mean – LLMs pick high-probability tokens. A negative instruction is a soft constraint; training data is a hard prior.
- No inference-time ground truth – The model can't self-check compliance. Whatever leaves the softmax ships.
Prompts alone stop AI slop in about 80% of generations. The remaining 20% is where production reputation lives.
The Fix: A Code-Side Validator
Ozigi's solution is a two-layer defense. The banned lexicon lives both inside the prompt (prose rules for the LLM) and inside the code path (structured arrays for post-generation scanning).
A single source-of-truth file, anti-ai.ts, exports both the prose rulebook and the structured arrays. A dev-mode drift guard warns if the two diverge.
The validator runs four passes on every parsed draft:
- Pass 1: Vocabulary – Word-bounded, case-insensitive match against banned words like "delve", "tapestry", "robust".
- Pass 2: Phrases – Whole-token-sequence match against multi-word slop like "navigate the complexities".
- Pass 3: Openers and closers – Position-aware. Only fires if the term starts a sentence, paragraph, or post.
- Pass 4: Regex patterns – Structural tells: bold-colon prefixes (Architecture:), double-hyphen em-dashes, and contrast structures like "It's not X. It's Y."
Two precision details: code-block sanitization strips fenced blocks and inline code before scanning, and same-opener cadence detection flags three+ consecutive sentences starting with the same word.
The Retry Policy: One Bounded Retry
LLMs regress to the mean on every call. A second attempt usually fixes obvious tells. By the third, the model starts inventing new slop. By the fourth, you've spent 4× latency for diminishing returns.
Ozigi caps retries at one. The repair directive is critical: "Do not paraphrase, rewrite from source." Paraphrase prompts keep the same sentence skeleton; rewriting from source forces a different shape distribution.
After retry, the validator runs again. The system keeps whichever response has the lower slop score, even if neither is fully clean. The user always gets the best of two attempts, plus a lexiconWarnings payload for a "regenerate?" badge.
Latency Numbers from 48 Hours in Production
| Scenario | Frequency | Extra latency |
|---|---|---|
| Validator scan, draft is clean | ~88% | < 5 ms |
| Retry triggered, succeeds | ~10% | +1 LLM call (3–8 s social, 15–40 s long-form) |
| Retry triggered, fails to clean | ~2% | +1 LLM call, ship cleaner of two |
Worst case is roughly 2× generation time. The validator scan itself is regex over a few KB of text, sub-5ms even on a 2,000-word article.
Why Not a Humanizer API?
The team considered piping output through a third-party humanizer, then a "tuning" pass. They rejected it:
- Cost stack – Humanizer adds at least one round trip for every generation. The validator only pays on the ~12% of drafts that need it.
- Detection arms race – Humanizers are trained to fool AI detectors, not to sound human. They introduce typos and fragmented sentences.
- Meaning loss – Humanizers regularly invert sentences or drop technical specificity. A re-tune pass on mangled output adds noise.
- Ownership – A humanizer is a black box. The banned lexicon is a TypeScript file. When a user complains about a word, add it to the array and the drift guard catches any mismatch.
What You Should Do Now
If you ship LLM output to end users, add a code-side validator. Start with a small banned list of your most-hated AI tells. Use a single retry with a "rewrite from source" directive. Measure latency and slop scores. Your users will thank you.
