Why My AI Agents Live on My Mac, Not in a Data Center
There’s a thread climbing HN right now comparing anonymous request tokens from Opus 4.6 and 4.7. 490 points, hundreds of comments, everyone debating which model wins on reasoning benchmarks. I get it. But while that conversation is happening, I’m sitting in Tel Aviv with Tony, Tim, Tomer, Theo, and Talia running entirely on my M3 Max, processing prospect research and drafting sequences, and not a single token is touching Anthropic’s servers.
People think I’m being paranoid. I think they’re not thinking clearly about what “running AI agents” actually means operationally.
The Latency Argument Is Real, and It’s Not Close
When Talia, my chief of staff agent, kicks off a morning brief at 7am, she fans out tasks to the T4 simultaneously. Tony is pulling CRM enrichment, Tim is scanning for trigger events, Tomer is scoring accounts, Theo is drafting first-touch variants. That’s four parallel workstreams that need to coordinate results back into a coherent output within seconds.
Do that in the cloud and you’re stacking API round trips. Tel Aviv to US-East is around 120ms baseline. Multiply that across tool calls, agent handoffs, and context passing between agents that need to share state, and you’re looking at coordination overhead that genuinely degrades the output quality, not just the speed. Agents start working with stale information from each other because the sync lag is long enough that state has changed by the time a response arrives.
On local metal, agent-to-agent communication is effectively zero latency. Talia can pass a revised ICP filter to Tony and have his updated prospect list reflected in Tim’s trigger scan before Tim’s first tool call even resolves. That tight loop is what makes the coordination actually work.
Privacy Is a Business Argument, Not a Paranoia Argument
Everything my agents touch is commercially sensitive. Prospect lists, deal stage data, sequence copy, competitive positioning, account intelligence from customer conversations. When I run T4 plus Talia in the cloud, every one of those inputs is a training signal or a logging entry somewhere, governed by terms of service I don’t fully control.
My company sells to enterprise. Our buyers have procurement teams that ask hard questions about data handling. I can’t stand in front of a CISO and explain that my AI-assisted outbound process runs through a third-party API that may log inferences about their company’s tech stack. Running local means I can answer that question cleanly.
The HN crowd benchmarking Opus versions is solving for peak capability. I’m solving for sustainable, defensible operations. Those are different problems.
Reliability Is the One Nobody Talks About Enough
Cloud model APIs go down. Rate limits hit at the worst moments. Pricing changes mid-quarter and suddenly your cost model is broken. I’ve run T4 through a full quarter now and had exactly zero outages attributable to model availability. Because the model is on my machine. The only failure mode is hardware failure, and I have local redundancy for that.
There’s also a subtler reliability issue: model drift. When providers silently update a model version, agent behavior changes. Prompts that worked last month produce different outputs this month. Running pinned local models means my agents behave consistently across time. Tony in January is the same Tony in April.
The trade-off is real. Local models are not at frontier capability. Opus 4.7 on some Anthropic GPU cluster is probably better at raw reasoning than what I’m running locally. But “better at reasoning” in a benchmark and “better for my specific workflow with acceptable latency and zero privacy exposure” are not the same sentence.
The open question I keep sitting with: as local model capability closes the gap with cloud frontier models, what’s the argument for ever sending sensitive business data to someone else’s server? I’m genuinely not sure there is one.
