The IP Leak Epidemic: The Hidden Cost of Cloud AI Prompts

The security review for a cloud AI rollout almost always asks the right first question, does the vendor train on our data, and almost always stops there. The vendor says no, or offers an opt-out, or points to a data processing addendum, and the review closes. That question is not wrong to ask. It is just a small fraction of the actual exposure, because "does the vendor train on it" describes one narrow fate for a prompt among several, and the other fates get almost no scrutiny at all.

A prompt is a data transfer. That is the whole fact, underneath every product description and marketing page. The moment an engineer pastes a function into a chat window, or a proposal writer drops in a draft containing unreleased pricing, or an analyst asks a model to summarize a document with a classification marking on it, the content of that paste has left the organization's boundary and arrived on infrastructure someone else operates. What happens to it next (training, logging, human review, retention, a subpoena, a breach at the vendor) is a separate question from whether it left, and the leaving is the part most governance programs never actually measure.

The training question is a distraction

"Do you train on our inputs" is easy to ask because it has a yes-or-no answer, and vendors have gotten good at giving the reassuring one. But training is only one of the things that can happen to a prompt after it's submitted, and arguably not the most consequential one for most organizations.

Prompts get logged for abuse monitoring and debugging, a legitimate operational need, and also a data store that now exists, with its own access controls, retention policy, and breach surface, none of which the submitting organization controls or can audit directly. Prompts get reviewed by humans in the ordinary course of quality assurance and safety tuning, which means the "no human sees your data" assurance is frequently narrower than it sounds: it usually means no human sees it for training purposes, not that no human sees it at all. Prompts sent through an API rather than a consumer chat interface often carry different contractual terms than the consumer product from the same vendor, and organizations that assume enterprise-tier protection because they're using enterprise-tier billing sometimes discover the actual data-handling terms are inherited from a different product line than they thought.

None of this requires a vendor acting in bad faith. It requires only that the vendor's business is operating a shared service well, at scale, for many customers, which necessarily means prompts pass through infrastructure, logging, and review processes designed for that scale, not for the confidentiality needs of any single customer's most sensitive draft.

What actually goes into the box

Ask a security team to enumerate what's in their source code repositories, their document management system, their CRM. They can do it, imperfectly, but with a real inventory and access logs to check the inventory against. Ask the same team what's gone into cloud AI prompts over the past year, and the honest answer is almost always "we don't know," because prompt content is unstructured, ephemeral from the user's perspective, and generated by employees making individual judgment calls about what's reasonable to paste, dozens of times a day, across every function in the company.

In practice that means the leakage is broad rather than exotic. Engineers paste proprietary algorithms and unreleased architecture into a coding assistant to get a better completion. Legal and business development paste unsigned contract drafts and deal terms into a summarization tool. Product teams paste roadmap documents into a model to get help writing the external version. Customer-facing teams paste account data, sometimes regulated, sometimes merely embarrassing to lose, to draft a response. None of these employees think of what they're doing as a data exfiltration event. Each of them is, technically, initiating one, and the aggregate is a continuous, decentralized outflow of exactly the material an organization would fight hardest to protect if it were leaving through any other channel.

This is the part that makes prompt leakage different in kind from a conventional breach. A breach is a discrete event with a forensic trail: a compromised credential, an exfiltration timestamp, a blast radius you can eventually characterize. Prompt leakage has no event to point to. It's continuous, distributed across thousands of individually reasonable-seeming actions, and by the time anyone thinks to ask "what have we sent," the honest answer requires reconstructing months of behavior from people who were not asked to keep a log, because nobody told them the paste was the sensitive act.

Why contractual assurances don't close the gap

The instinct once this risk is named is to fix it with paper: a data processing addendum, a no-training clause, a vendor security questionnaire, an approved-tools list. These are worth having. They are also, categorically, controls rather than constraints: statements about how a system is supposed to behave, revisable at the vendor's discretion, enforceable after the fact and only if a violation is somehow discovered. None of them changes what the network call is architecturally capable of carrying, and none of them helps at all with the much larger surface of shadow usage, the browser tab open to a consumer AI product that was never on the approved-tools list to begin with, used by an employee who found it more responsive than whatever sanctioned alternative procurement eventually delivered.

An approved-tools policy assumes the organization can see and enforce which tools get used. In practice, consumer AI products are a click away in any browser, require no procurement approval, and produce genuinely useful output, which is exactly why they get used quietly, by capable people trying to do good work faster, entirely outside whatever governance framework the DPA was supposed to anchor. The policy fails not because it was badly written but because it depends on a visibility the organization doesn't actually have.

The architectural alternative

The only version of this problem that resolves cleanly is one where there is no outbound path for the prompt to travel in the first place. Not a monitored path, not a logged-and-deleted path, no path. That means the model runs inside the boundary the sensitive material already lives in, so that the question "did this prompt leave the building" has the same answer every time: no, because there was never a network route by which it could.

That requirement looks different depending on what the sensitive workload actually is. A team that needs day-to-day AI assistance against a codebase, a document set, or an internal knowledge base needs an assistant with retrieval and reasoning that never has to phone home to be useful, where the model, the index, and the inference all sit on hardware inside the same boundary as the data. A team doing deeper R&D, fine-tuning a model against proprietary or regulated data, running experiments that would themselves be sensitive if their existence were known, needs a sealed environment built for that heavier workload, not a chat interface bolted onto a shared cluster.

Sealed appliance architecture is the structural answer to prompt leakage because it removes the ambiguity that makes the problem so hard to govern by policy. There is no vendor log to trust, no training opt-out to verify, no shadow-usage question to chase down, because the prompt itself never crosses the boundary that would put any of those questions in play. The inventory problem disappears along with the outbound path. If nothing leaves, there is nothing to reconstruct after the fact.

Where this sits in the E31 line

This is the reasoning behind building Forge and Czar as sealed appliances rather than as access-controlled endpoints into shared cloud infrastructure. Forge gives engineering teams repository-aware coding assistance without a single prompt or completion crossing the organization's boundary. Czar gives research and R&D teams a sovereign environment for training and fine-tuning against sensitive data that was never going to be permitted near a multi-tenant cloud service in the first place. Both sit on Substrate, the shared platform underneath Element 31's hardware line, and both exist because the honest fix for prompt-level leakage was never going to be a better contract. It was always going to be a boundary the data structurally cannot cross.