Every smart glasses demo you've seen in the last two years follows the same script: point the camera at something, ask a question, get an answer a second later from a cloud model you've never met. It works. It's also a non-starter for anyone bound by HIPAA, ITAR, or basic common sense about where a live video feed from a patient's room should be allowed to go.

Over the past month I've been running a fully local inference stack on Mentra Live — open firmware, an always-on camera, and just enough onboard compute to make you respect how far edge silicon has come. Here's what the setup actually looks like.

The stack

MentraOS ships as an open-source fork with a clean audio/video capture bridge, which meant the hard part wasn't the glasses — it was picking a model small enough to run near the edge without becoming useless. I landed on a quantized Llama 3.2 checkpoint served from a local inference box, with the glasses talking to it over the local network rather than the public internet.

mentra-cli pair --device live-0231
mentra-cli route set --endpoint http://10.0.1.42:8080/v1/chat --mode local-only
mentra-cli route verify
> 0 requests routed externally in the last 24h

That route verify command became my favorite line in the whole project. It's the difference between a privacy claim and a privacy guarantee — you can actually watch the egress counter sit at zero while you use the device normally for a full day.

What broke, and what didn't

Latency was the first casualty. Cloud models return in 400-600ms; a local 3B-class model on modest edge compute runs closer to 1.5-2 seconds for a full response. For quick lookups — "what's this part number," "translate this sign" — that's fine. For anything conversational, it's noticeable, and it's the honest tradeoff you're making for keeping data local.

What didn't break was reliability. No dependence on a cellular or Wi-Fi uplink to a third party meant the device kept working in a warehouse basement with no signal, which is exactly the environment a field technician actually works in.

Why this matters beyond the demo

The MentraOS setup is a hobbyist-friendly version of the same architecture we deploy for enterprise partners: glasses talk to a private inference endpoint, the endpoint runs a model you chose, and nothing transits a cloud API in between. The difference at enterprise scale is the model gets bigger, the server gets racked in a real data center instead of sitting on my desk, and the compliance paperwork gets a lot more serious. But the core guarantee — zero bytes to third-party AI APIs — is identical.

If you're evaluating smart glasses for a regulated environment, the question to ask any vendor isn't "do you support on-device AI." It's "show me the network trace." If they can't show you a zero, keep looking.