Network unplugged

There's a conversation that kills AI projects in regulated work, and it always arrives in the same shape. Someone from legal, compliance, or risk asks where the data goes. What happens to a privileged document after the request completes. Whether it's retained, who can subpoena it, whether it trains anything, which jurisdiction it sat in for the duration.

Every answer to those questions, in a hosted architecture, is a promise. Good promises — contractual terms, certifications, a vendor with real controls and a real audit trail. But promises, evaluated by someone whose job is to assume promises fail. So the project goes into a review cycle it frequently does not come out of, and the firm concludes that AI isn't available to them yet.

It's available. The framing is wrong. The answer that lands in that room isn't a better promise; it's the removal of the question.

A property, not a pledge

When the model runs on hardware you control, on a network segment with no egress, the data-handling question stops being contractual and becomes structural. There is no transmission to govern, no retention policy to trust, no third-party processor to paper over, because there is no third party. The control isn't that someone agreed not to keep the data. It's that the data had nowhere to go.

This is a categorically easier thing to review, and the reason is that it's demonstrable. A compliance officer does not have to take an engineer's word for the boundary — they can see the network policy, read the firewall rules, watch the request complete with the interface down. The reviewability is the whole point. Most security arguments in an AI system are arguments about behavior under conditions nobody can fully enumerate. This one is an argument about topology, and topology holds still while you inspect it.

It also survives the question that follows. When a client's client, or a regulator, or opposing counsel asks how the firm used AI on a matter, "the model ran inside our own environment and nothing left it" is an answer that doesn't require a vendor to co-sign.

What actually holds up locally

The honest version of this note has to say what works and what doesn't, because the failure mode here is enthusiasm.

Local models are strong now at the work that makes up most of the volume in professional services: pulling structured fields out of unstructured documents, classifying and triaging intake, redaction and PII detection, summarizing within a narrow and well-defined domain, drafting against a known template, answering questions over a private corpus with retrieval doing the heavy lifting, and cleaning up the long tail of data-entry work that currently consumes junior hours. This is not a consolation prize. In a practice drowning in document handling, that list is the business case by itself.

They are still meaningfully behind at the hardest end: long-horizon reasoning across a large body of material, synthesis where the quality difference is obvious to an expert reader, novel analysis that depends on broad world knowledge, and the top tier of code generation. If your use case lives there, a local-only design will disappoint you, and we'd rather say that before the hardware arrives than after.

The hardware itself has stopped being the obstacle. Capable open-weights models now run on a workstation under a desk, and the smaller ones on a well-specified laptop. The line moves in one direction, and it moves faster than most compliance review cycles complete.

The hybrid that stays honest

Very few practices are purely local, and they shouldn't have to be. The design that works is local by default, with escalation as a deliberate, recorded act.

That means the routing seam from the previous note, with a policy attached: classes of work that never leave, classes that may leave with the data de-identified, and classes that may leave in full with explicit authorization. What makes it defensible is the record — a log of what was escalated, when, by whom, and under which rule, kept as a matter of course rather than reconstructed when someone asks. A boundary you can only describe is a policy. A boundary you can produce the crossings for is a control.

The tempting shortcut is to leave the local tier as the fallback and route everything to the hosted model in practice, because it's easier and it always works. Resist it. An unexercised local path is a story you tell in a security review, not a capability you have.

Evals matter more here, not less

One caution. Trading a frontier model for a small local one narrows the margin you were relying on without knowing it. Prompts tuned against a large model often degrade in ways that are quiet rather than obvious — subtly worse extraction, a confident answer where the larger model would have hedged. In a domain where a missed clause or a wrong figure has consequences, quiet degradation is the dangerous kind.

So the golden sets and the eval harness aren't optional infrastructure for a local deployment; they're the mechanism that tells you the substitution was safe. Build them before you swap, keep them running after, and treat a model change on the local tier with the same seriousness as a schema migration.

The floor rose while everyone was looking up

The reason this is worth writing now is not nostalgia for local compute. It's that the capability floor rose quickly and most teams last measured it against a model generation that no longer exists, so a great deal of work is being sent out of the building by default, on an assumption that expired.

We're seeing more firms in sensitive verticals check that assumption, and find that a substantial share of what they wanted AI for runs on a machine they own, with the cable out. Not all of it. More than they expected.

If the compliance conversation is what's blocking you, start by asking what your work actually requires, and measure the local tier honestly against it. The answer is often that the hard problem was architectural, and architecture is solvable.

---

Need AI that clears a compliance review? Start a conversation.