Verizon’s 2026 Data Breach Investigations Report found frequent employee AI tool use jumped from 15% to 45% in one year.
Every new tool, account, and agent is one more thing a security team must govern.
Leadership expects the CISO to secure AI as fast as the business adopts it. That starts with visibility into all AI usage, shadow AI included, across the agentic plane.
Governance is also shifting from recommendation to requirement. Any organization that handles sensitive data or is in a regulated industry must be able to explain every AI interaction to auditors.
You may already have an AI governance program in place with policies written and security at the table as a major stakeholder. Yet nothing on the request path enforces it or records why a model took the actions it did.
That is the AI control gap.
What You’ll Learn
- Judge whether you can see all AI usage, including personal accounts.
- Decide which agent tokens need scoping before an agent acts on them.
- Understand why weaponized language beats signature-based tools.
- Anticipate how AI governance shifts from recommendation to requirement.
- Plan the governance, enforcement, and proof auditors will ask to see.
What Is The AI Control Gap?
The AI control gap is the inability to govern the intent behind a model’s actions once identity verification succeeds. A user or agent passes every login and access check. Nothing then asks why the model takes each action.
PurpleSec’s AI security readiness framework splits this work into stages:
- Stage 1, Foundational Governance, writes the policies.
- Stage 2, Securing the Foundation, deploys the technical controls that enforce them.
The AI control gap is the ground between those two stages.
Policies set the rules, and vendor guardrails run inside the model. Neither sits between the user and the model to enforce those rules or log the exchange. That missing control shows up in breach data. In IBM’s 2026 study, 92% of organizations with an AI-related breach lacked proper AI access controls.
Secure Every AI Interaction With Defiance AI™
Understand the why behind every prompt, response, and agent action in real time, so you can confidently audit, question, and govern AI.
Cost 1: AI Sprawl Outruns Your Visibility
AI sprawl is AI spreading across the enterprise faster than anyone can govern it. New tools, models, and agents appear with no inventory, no owner, and no policy attached. The pace at which new models are released widens the gap. By the time security finishes reviewing one model version, the next one has already shipped.
Stanford’s 2026 AI Index counted 102 notable AI models released in 2025 alone. Netskope’s 2026 cloud report tracked more than 1,600 genAI apps, five times the count a year earlier.
Capability moves just as fast. METR estimates the length of tasks frontier agents can complete has doubled every 89 days since 2024. The model behind an agent approved last quarter may be far more capable today.
Without baseline visibility, AI risk is unknown and you cannot govern what you cannot see.
In IBM’s 2026 study, shadow AI was involved in 43% of security incidents, up from 20%. Those breaches averaged $5.39 million, up from $4.63 million.
| Do we see it? | Can we inspect the prompt and the response? | Do we know the intent? | Is there a record? | |
|---|---|---|---|---|
| Sanctioned enterprise AIthrough the corporate proxy | Seen | Partial | Blind | Partial |
| Personal accounts on public modelsfrom a corporate device | Partial | Blind | Blind | Blind |
| Public modelsover phone data or guest WiFi | Blind | Blind | Blind | Blind |
| AI agentscalling APIs with tokens | Partial | Blind | Blind | Partial |
Traditional network controls only go so far because they were not designed to mitigate AI threats.
CASB, SWG, and DLP see traffic that crosses the corporate edge, and only for apps their catalogs recognize. Prompts sent over phone data or guest WiFi never cross it. Most still classify apps and data patterns, but not the intent behind a prompt.
Blocking public models helps, but personal accounts still get through.
Among employees routinely using GenAI on corporate devices, 72% used non-corporate emails as account identifiers. Public models give the security team no way to inspect the conversation. Without the prompt and the response, the intent behind each request stays unknown.
Consumer terms add a second exposure. OpenAI says it may use ChatGPT content to train its models. Google trains on Gemini activity while the user’s Keep Activity setting is on. Whatever an employee pastes can become training data.
That leaves enterprise tiers as the only AI tools worth sanctioning.
OpenAI says it does not train on business data by default. Anthropic’s commercial terms bar training on customer content. Anything outside those tiers sends company data where it can be trained on, cannot be inspected, and leaves no record.
Where records exist, they are split across vendors and say nothing about intent. That gap in the record reappears in every cost that follows.
Add Your Heading Text Here
Your AI Usage Has Blind Spots
Your team is adopting AI to compete. Nobody can tell you which risks apply to your own usage. The AI Usage Risk Report answers that, on your traffic, in seven days.
Cost 2: Agents Do What Their Permissions Allow
In July 2025, a Replit AI agent reportedly deleted a live company database during an active code freeze. Its instructions said not to proceed without human approval.
The agent later admitted to running unauthorized commands anyway. The instruction asked for restraint, but only the credentials could have enforced it.
It happened again in April 2026. The founder of PocketOS reported that a Cursor agent running Claude Opus 4.6 deleted its production database and backups. The token had been created only to manage custom domains, yet it could delete the storage volumes holding the database.
Some platforms make permissions hard to narrow. Vercel’s access tokens can be limited to a team or a single project. Within that scope, each one can read and write. None is read-only, so an agent that only needs to read logs gets write access too.
What the token was created for
Add and remove custom domains
What the token allowed
Any call to the infrastructure provider's API, including deleting volumes
The fixes are tight permissions, deletion protection, and human approval for every destructive action. A confirmation prompt alone will not stop an agent because an agent with enough access can automate the click.
Those controls act before anything executes, which no instruction to the model can do.
Scoped agents can still fail. When one does, the first question is what it ran and who approved it? If the only record is the agent’s own log, you are left trusting the failed system to explain itself.
Cost 3: Weaponized Language Slips Past Signature-Based Defenses
Attackers no longer need malicious code. They weaponize language, writing instructions a model will follow. The UK’s NCSC notes that today’s models do not enforce a boundary between instructions and data in a prompt.
Any text the model reads can become a command.
- Prompt Injection: Instructions hidden in a prompt, document, tool, or agent message that hijack what the model does.
- Data Exfiltration: Crafted prompts that coax a model or agent into handing over sensitive data.
- Insider Misuse: Authorized users who word requests to pull data or slip past the controls their access should respect.
Signature-based defenses were built for a different threat. Signatures, web filters, and DLP rules match known code, patterns, and keywords. Language has no fixed signature. The same intent can be rephrased endlessly, then disguised through shadow prompting, prompt obfuscation, or adversarial prompt chaining.
That turns signature-based defenses for AI threats into a game of whack-a-mole.
Security teams must maintain an ever-growing library of malicious phrasings, and attackers need only one new permutation. Every time an agent drifts from its guardrails or policy, the library needs another entry.
NCSC guidance warns there are infinite ways to rephrase an attack around a deny-list. In Lakera’s Gandalf study, 246,401 of 279,675 attack prompts were unique.
Defenses built to catch prompt injection and jailbreaks fall behind too.
Researchers bypassed 12 recent defenses against prompt injection and jailbreaks, most at over 90% success. Most had originally reported near-zero attack success.
Malware is turning AI against signatures as well. Google’s threat intelligence team found an experimental strain, PROMPTFLUX, that asks Gemini to rewrite its own code. Google says the likely goal is evading signature-based detection.
Prompt injection hidden in a retrieved web page
Ordinary text to an allowed domain
No known signature
Passed1The conversation so far
2The decoded instruction
3The intent behind it
4The tool call it would trigger
BlockedData exfiltration request split across harmless-looking turns
Ordinary text to an allowed domain
No known signature
Passed1The conversation so far
2The decoded instruction
3The intent behind it
4The tool call it would trigger
BlockedInsider's reworded request for restricted data
Ordinary text to an allowed domain
No known signature
Passed1The conversation so far
2The decoded instruction
3The intent behind it
4The tool call it would trigger
FlaggedMitigating the weaponization of language means reading it the way the model does.
- Conversation Context: What came earlier in the exchange, since attacks spread across turns.
- Semantics: What each prompt and response actually asks for, however it is phrased or encoded.
- Intent: Why the user or agent is making the request, mapped to risk.
Relying on signatures alone leaves weaponized language unchecked. AI needs its own security layer, one that reads context and intent, not patterns. When an attack gets through, that layer also holds the exact exchange investigators need.
Build Your AI Security Roadmap
Turn abstract AI risks into actionable operational tasks for your team.
Cost 4: Proving Every AI Interaction As Requirements Arrive
AI rules increasingly expect organizations to explain how they use AI. That takes an immutable audit trail of every AI interaction, including the intent behind it. Policies, guardrails, and governance describe what should happen.
Recording what happened, and why, takes enforcement on each interaction.
The table below shows the pattern:
Requirements arrived within three years of first text, and some within months.
Sector | Rule | Announced | Required By |
Healthcare | Proposed Aug 4, 2022 | Within 300 days of Jul 5, 2024 | |
Healthcare | Proposed Apr 18, 2023 | Jan 1, 2025 | |
Healthcare | Proposed Jan 6, 2025 | Final rule targeted Jul 2027 | |
Insurance | Adopted Dec 4, 2023 | State by state, still adopting | |
Finance (New York) | Oct 16, 2024 | Sep 10, 2026 letter: risk assessments should consider AI | |
Defense | Final rule Sep 10, 2025 | Phased in from Nov 10, 2025 | |
Employment (Illinois) | Aug 2024 | Jan 1, 2026 | |
Cross-sector (Texas) | Final text Jun 2025 | Jan 1, 2026 | |
Cross-sector (California) | Approved Sep 23, 2025 | Jan 1, 2027 |
Pushback from the Federal government adds uncertainty.
Executive Order 14365 directs the Attorney General to form a task force that challenges state AI laws and as a result some state requirements may change.
The need to show how AI touched regulated data will not.
How Do Governance, Enforcement And Proof Close The Gap?
The missing layer is a control on every AI interaction that enforces governance and records what happened. Closing the gap takes three parts.
Governance turns policy into an operating model:
- Assign owners and seat a governance committee, so the work does not all fall on the CISO.
- Keep an AI incident response plan and run tabletop exercises.
- Prove the security posture improves over time.
Enforcement turns that policy into controls:
- See all AI usage, attributed to individuals and departments.
- Inspect prompts and responses in both directions and classify the intent behind each one.
- Allow, block, or monitor agent tools by name, category, vendor, or scope.
- Scope agent tokens to one job, such as a single repository.
Proof turns activity into evidence:
- Keep an immutable record of both directions of every exchange, including what was blocked.
- Capture what was sent, by whom, and to which tool in one record, not split across vendors.
- Keep the intent behind each action, which is the evidence auditors ask for.
Secure Every AI Interaction With Defiance AI™
Understand the why behind every prompt, response, and agent action in real time, so you can confidently audit, question, and govern AI.
Frequently Asked Questions
Are Enterprise AI Tiers Safe Enough On Their Own?
They solve one problem. The provider does not train on your business data by default. They do not show what employees send, what agents do, or why. Sanctioned tools still need role-based controls, inspection of prompts and responses, and a record of each interaction. A contract term is not an audit trail.
Should AI Agents Have Their Own Identities?
Yes. An agent that borrows a person’s credentials inherits everything that person can do. Its actions also look like theirs in the logs. Give each agent its own identity, a named owner, and a token scoped to one job. Every action then traces back to the agent and the human accountable for it.
How Is The Intent Behind An AI Action Determined?
An enforcement control reads the whole exchange, including the prompt, the response, and any tool call. It then classifies what the action is trying to do and maps that class to risk. Defiance AI™, for example, records classes such as extraction, manipulation, reconnaissance, and automation. See how intent classification works.
How Do You Test Whether Your AI Security Controls Actually Hold?
Test them against your own policy, using attacks that adapt to the defense. A vendor benchmark only shows a product passed someone else’s test. Red team the prompts, documents, and tools your agents actually touch, including obfuscated and chained prompts. Repeat after every model or tool change. PurpleSec’s AI red teaming checklist covers the scope.
What Makes An AI Audit Trail Immutable?
An immutable trail cannot be edited or deleted without detection, and it lives outside the system it describes. Each AI decision gets its own signed record with the prompt, response, intent, and action taken. Retention matches what each regulation requires. Logs an agent writes about itself do not qualify, because the failed system cannot vouch for itself.
Who Should Own AI Governance Besides The CISO?
Spread it across a governance committee that includes legal, privacy, compliance, IT, and the business units that deploy AI. The CISO owns enforcement and the evidence. A Chief AI Officer, where one exists, owns technical review. Business owners decide which AI use is worth the risk.
Should AI Controls Start By Observing Or By Blocking?
Observe first. Watching real traffic shows which tools, accounts, and agents are active, and which rules would fire. Inspection and risk scoring come next, then inline blocking with rules tuned to that traffic. Every stage should write the same record, so evidence starts on day one.
Does Following The NIST AI RMF Satisfy These Rules?
It helps, but a framework is not a control. NIST released its AI Risk Management Framework in January 2023 for voluntary use. Texas TRAIGA’s enforcement text does reference substantial compliance with NIST’s Generative AI Profile. Something still has to enforce the framework on each request.
Article by
Related Content
