Repository-Scale Context: How Forge Handles Massive VRAM Context Windows

A codebase is not a long document. A five-hundred-page policy manual is linear: read front to back and the references mostly resolve as you go. A repository is a graph. A function in one file calls a helper in another, which depends on a type defined in a third, which is re-exported through a barrel file that half the codebase imports and almost nothing directly touches. Ask a coding assistant a question that spans that graph — "what breaks if I change this interface," "where else is this pattern used," "why was this abstraction introduced" — and the honest answer requires having enough of the graph in working memory at once to trace it. That requirement is what makes repository-scale context a distinct engineering problem from simply lengthening a chat window. It's the problem Forge is built around.

Why "just make the context window bigger" doesn't solve it

The instinct is to treat repository awareness as a context-length number: if the window is long enough, paste in enough files and the model will figure out the rest. That approach runs into two walls at once.

The first is capacity. A context window is ultimately bounded by VRAM: every token held in context requires a resident key-value entry per layer, per attention head, and that cost scales with sequence length regardless of how generous the advertised window is. A codebase of any real size, dumped in naively as raw text, will exhaust that budget long before it exhausts the useful portion of the repository. The second wall is relevance. Even a context window large enough to fit an entire mid-sized repository wastes most of its capacity on files that have nothing to do with the question being asked. That does not just cost memory, it dilutes the signal the model has to reason over, and retrieval quality, not raw window size, is what actually determines whether an answer is correct.

Repository-scale context, in other words, isn't a request for a bigger box. It's a request for the right contents of the box, assembled on demand, from an index that already understands how the repository's pieces relate to each other.

Indexing happens once, inside the boundary

Forge's approach starts before any query is asked. On onboarding, and continuously as commits land, Forge builds a local index of the repository: a structural map of files, symbols, call graphs, imports, and the relationships between them, along with the embeddings that make semantic retrieval over that structure possible. This indexing work happens once, inside the sealed appliance, rather than being reconstructed from scratch on every request by shipping source out to be re-read by a remote model. The index is what turns "the whole repository" from an undifferentiated pile of text into a queryable structure that retrieval can act on precisely.

That distinction matters for the air-gapped constraint as much as for performance. A design that relied on an external service to build or maintain semantic understanding of the repository would require the repository, or at minimum representative fragments of it, to leave the enclave at some point in that pipeline. Building and holding the index locally means the structural understanding of the codebase never has an occasion to exist anywhere but inside the boundary the appliance was bought to enforce.

Retrieval decides what earns a place in the window

When a query comes in, Forge doesn't attempt to load the repository into context. It retrieves the slice of the index that's actually relevant to the question, using the call graph and symbol relationships to pull in not just the file that matched the query but the definitions, callers, and dependencies that give that file meaning. A question about a function pulls in its signature, its callers, and the types it touches; it does not pull in an unrelated module three directories away just because the window had room.

This is where the context-window budget and the retrieval system have to be designed together rather than treated as separate concerns. A larger window that retrieval fills indiscriminately does not produce a better answer than a smaller window filled precisely. It produces a slower one, because every additional resident token adds to the key-value cache the accelerator has to attend over on every generation step. The design goal for a repository-scale coding assistant is not maximizing how much fits in VRAM at once. It's maximizing how much of what fits is actually load-bearing for the question being asked. Forge's retrieval layer is the component responsible for making that tradeoff well, request by request, rather than leaving it to chance or to the user's own guess about which files to paste in.

Multi-turn sessions and the cost of staying resident

A single query is the easy case. A working session — an engineer moving through a refactor across dozens of turns, each one building on context established earlier — is where repository-scale context gets expensive in a way that compounds. Each turn's key-value cache adds to what the appliance has to hold resident if the conversation's earlier context is going to remain usable, and a long session reasoning over a large codebase can accumulate a cache footprint that rivals the retrieved-content footprint itself.

Forge manages this the way any VRAM-constrained system serving concurrent, extended sessions has to: by treating KV cache growth as a first-class sizing input, not an afterthought discovered under load. Session context gets curated over the course of a long interaction rather than accumulated without bound. Context that's no longer load-bearing for the current turn gets retired, and the index gets re-queried when an earlier file becomes relevant again, rather than keeping every prior turn's full context permanently resident. The effect an engineer should feel is a session that stays fast and coherent through a long refactor, not one that visibly slows as the conversation lengthens because the appliance quietly ran out of fast memory and started paying a much steeper cost to keep going.

Multi-engineer, multi-repository reality

Real deployments rarely mean one engineer asking one question about one repository. A team running Forge typically has several engineers working concurrently, often across more than one repository the appliance has indexed, each session carrying its own retrieved context and its own KV cache. That concurrency is a second axis of the same sizing problem discussed above. The VRAM budget isn't being spent by one session's context window, it's being shared across however many sessions are active, and a system that only accounts for the single-session case will behave correctly in a demo and unpredictably in production.

This is part of why context-window capacity has to be treated as a deployment-sizing question, not a fixed spec that applies identically regardless of team size or repository scope. An appliance provisioned for a small team working on one moderately sized repository is solving a materially different memory-budgeting problem than one supporting a larger engineering organization across several codebases at once, even though both are described, at a glance, as "running Forge."

What this looks like from the seat of the engineer asking the question

None of this architecture is meant to be visible in daily use, and that's deliberate. The engineer asking "what else calls this function" or "trace how this request flows through the service" should get a fast, accurate answer that clearly reflects awareness of the whole repository, not a system that asks them to manually select which files matter, and not one that stalls while it works through an oversized context. The indexing, retrieval, and cache-management work described here exists specifically so that the felt experience is simple even though the underlying problem isn't: reasoning correctly over a codebase-sized graph, entirely inside a sealed appliance, without ever sending that codebase anywhere to be understood.

That combination is the actual point of Forge as a category. Not a chat window with a longer limit, but a system architected end to end — from indexing through retrieval through VRAM-aware session management — around the particular shape of the problem a real repository presents.