This week I ran a fully local AI model on my MacBook.

M1 Pro, 16 GB RAM. Qwen 3.5 at 9 billion parameters.

No API. No cloud. No data leaving the machine.

It worked better than I expected.

I wasn't running a benchmark. I was trying to use it in my actual AI agent setup (Wiz) for real tasks. Memory recall, tool calling, simple generation. The results were genuinely usable for a non-trivial subset of what I normally do.

There are two different "local AI" stories that get conflated constantly: One is a local agent with a cloud model.

Your automation logic lives on the device, but the LLM is still Anthropic or OpenAI. This is why people started buying Mac Minis this year to host agent frameworks. Cheap hardware, cloud intelligence.

The other is a fully local LLM. The model itself runs on your device. No data exchange. No subscription dependency for the core capability.

The second path used to require serious hardware investment. Qwen 3.5 at 9B is the first model where that hardware requirement drops to a regular, several-year-old laptop.

I also tested 0.8B and 2B models on an iPhone 17 Pro. The capability is limited, but the signal is interesting: consumer smartphones have crossed the threshold where this is possible at all. That matters.

Three reasons this is worth paying attention to:

1. Privacy. When inference runs locally, there's nothing to log, no terms of service governing your queries, no API provider in the middle.

2. Cost. For high-volume agent workflows, routing simple tasks to a local model instead of a frontier model changes the economics significantly.

3. Resilience. An agent that can fall back to local inference doesn't depend on uptime, rate limits, or subscription tiers.

The gap between local and cloud AI capability is real and it's not closing overnight. But the direction is clear, and the hardware that makes it accessible is already in most people's pockets and on their desks.

I wrote up the full experiment here: