CMMC 2.0 and AI: How to Leverage LLMs Without Failing Federal Audits

Contractors in the defense industrial base have spent the last several years getting their arms around CMMC 2.0: scoping the CUI environment, mapping controls, hiring or training toward assessment readiness. Most of that work assumed a fairly static picture of "the environment": file shares, email, a handful of line-of-business applications, all sitting inside a boundary that a C3PAO could walk and diagram. Generative AI does not fit that picture cleanly, and a lot of contractors are now discovering that their engineers are pasting proprietary technical data into consumer chat interfaces months before anyone on the compliance side notices.

That is not a hypothetical. It is the default outcome of giving capable engineers a browser and a deadline. The question is not whether people at a CMMC-scoped contractor will want to use an LLM to speed up code review, document drafting, or requirements analysis. They already do, on personal accounts, on tools nobody assessed. The question is whether the organization gets ahead of that with an architecture that survives an assessor's questions, or gets found doing it the hard way, during an audit, by someone else.

Why "just use the enterprise tier" doesn't resolve the boundary question

The instinctive fix is to buy a business or government-cloud tier of a major AI vendor and call the problem solved. That helps. It is meaningfully better than shadow use of consumer tools, but it does not by itself answer the questions a CMMC assessment actually asks. CMMC's logic is boundary logic: CUI has to be identified, the environment that touches it has to be scoped, and every system in that scope has to be accounted for against the practice requirements. A cloud AI service, however well-contracted, is infrastructure you do not operate and cannot fully inspect. Bringing it inside your CUI boundary means extending that boundary onto someone else's multi-tenant infrastructure, which is exactly the kind of scope expansion assessors are trained to probe.

This is not a claim that cloud AI vendors are careless with data. Reputable ones are not. It is a narrower, structural point: a contractual assurance that data is handled correctly is a promise about behavior, and CMMC increasingly wants evidence about capability. Can this system send CUI outside the boundary is a different question from has this vendor promised not to, and an assessor evaluating your system security plan is going to ask the first one. Every additional cloud dependency inside CUI scope is another subprocessor, another data flow diagram, another thing your assessor has to take partly on faith.

What actually has to be demonstrable

Strip the framework language away and an AI tool operating on CUI has to answer a short list of questions in a way that produces evidence, not reassurance.

Where does the model run, and where does the data go. If a prompt containing regulated technical data leaves your network to reach an inference endpoint, that egress path is now part of your CUI boundary and has to be documented, justified, and controlled, the same as any other data flow. The cleanest answer is the one that doesn't require the question: inference that never leaves premises has no egress path to diagram.

What was the model trained or fine-tuned on, and can that be shown. Provenance matters more in an AI context than it does for most software, because the object under review isn't just code. It's weights, and weights encode whatever they were trained on. A fine-tuned model built on your program's data needs the same custody story as the data itself: who touched it, on what infrastructure, and with what controls in force at each step.

Who can change the system, and is every change recorded. Assessors care about configuration management because undocumented drift is where compliance quietly dies. An AI deployment that updates itself against a vendor's cloud schedule, outside your control and without a local record of what changed, is a configuration management gap by construction, not because anyone did anything wrong, but because nobody can produce the version history when asked.

What happened, and can you reconstruct it later. Audit logging isn't a nice-to-have layered on top of an AI tool; for CMMC purposes it is close to the point. If an incident response team or an assessor asks what a given user submitted to the model last quarter, "we'd have to ask the vendor" is not an acceptable answer, and depending on the vendor's retention policy it may not even be a possible one.

None of these are exotic requirements. They are the same accountability, configuration management, and audit expectations CMMC already applies to everything else in scope. AI tooling doesn't get a carve-out; it just makes the gaps more visible, because the systems are new enough that most organizations haven't built the same documentation reflexes for them yet.

The air-gapped answer, and its honest cost

An air-gapped AI appliance answers the boundary question by removing the boundary problem: if the system has no network path to anywhere outside your facility, there is no egress diagram to draw, no subprocessor to disclose, no cloud vendor's infrastructure to fold into your CUI scope. The inference happens on hardware inside the room the assessor is standing in. That is a structural answer, not a policy one, and it is the kind of answer CMMC's architecture rewards: a fact about what the system can do, rather than a commitment about what it won't.

It is worth being honest about what that costs. An air-gapped deployment does not get frontier-model updates on a vendor's release cadence; it gets them on whatever schedule your organization can validate and push signed updates through. It requires real hardware, provisioned and maintained, not a subscription. And it puts more of the burden of model selection and capability on your side of the fence, because there is no cloud endpoint to fall back on when a task exceeds what's running locally. For a contractor handling CUI or working toward higher CMMC levels, that tradeoff is usually the right one. But it is a tradeoff, not a free upgrade, and a system security plan should say so plainly rather than imply the constraints don't exist.

Where this points, in Element 31's catalog

For a contractor whose actual use case is engineers wanting AI assistance on proprietary code and technical documents without that material ever leaving a CUI-scoped facility, the relevant product here is Forge, Element 31's air-gapped coding copilot appliance, sealed and provisioned before it ships, running on the shared Substrate platform. It's the narrowest fit for the narrowest version of this problem: a coding assistant with no network path out. Organizations with a broader need, fine-tuning or training against sensitive program data rather than just querying an assistant, are better served by Czar, which is built for that sandboxed training workload specifically rather than as a side effect of a coding tool.

Neither replaces the compliance work. Buying sealed hardware doesn't write your system security plan, define your CUI boundary, or get you through an assessment on its own. That's still organizational work, done by people who understand your specific scope. What a sealed, air-gapped appliance does is remove one whole category of question from that work: the one where an assessor asks where the data goes, and the honest answer, structurally, is nowhere.