I Use Hybrid AI Every Day: A Local LLM That Knows When to Call Claude

I use hybrid AI every day. And I suspect, eventually, you will too.

There is a 2,300 dollar computer on a shelf in my office running a local LLM, OpenAI's open-weights gpt-oss-120b, on its own hardware with no subscription and no meter. It reads every email my companies receive. It finds and qualifies leads, manages outreach, drafts replies, books and reschedules meetings, and keeps my calendar in sync with my life. For three weeks this fall it helped coordinate my daughter's wedding.

The local LLM handles the constant, repetitive work, along with anything I want to keep private. But one of its most important jobs is knowing when to ask for help. When a task needs deeper reasoning, nuanced judgment, or better writing, it calls a cloud frontier model, currently Claude. Anything that requires my approval shows up on my phone, where I can approve it, edit it, or rewrite it.

My team at AGLedger has published the engineering twice: how the box is built and how the gate in front of the cloud works. This post is about the shape of the thing, and why I think it is the shape most business AI ends up taking.

#What a day looks like

The software on the box is a harness I built. I call it AgentSDR. It is not a product I bought; it is my own code, and it is what runs the models, the rules, and the approvals. It works four company mailboxes, plus my personal one, which it reads only to put committed times on my calendar. Over a normal day it sources prospects, runs outreach in my voice, sorts what comes back, drafts the replies that need drafting, and books the meetings that need booking. I see the parts that need me. I do not see the rest, and that is the point.

The work splits three ways, by the shape of the task rather than by vendor loyalty.

• The constant work runs on the box. Is this message written for us, or is it generic? Does this personal mail contain a commitment that belongs on the calendar? Which bucket does this lead go in? Temperature zero, a few hundred tokens, a verdict. Thousands of these a week, at the cost of electricity.
• The hard work runs on Claude. Anything a person will read, a reply, a follow-up, a judgment call about a thread, goes to Claude. That is where the money and the quality go, and because the box filtered the volume, I can afford the best model every time it matters.
• The rules run on neither. Send caps, send windows, deduplication, suppression lists, the daily floor. A large share of the decisions in an agent system are not model decisions at all, and putting a model behind a question that an integer comparison answers is a reliable way to spend money on nothing.

When I measured the first version of the gate in August, roughly 95 percent of scheduled cycles never reached the cloud. The two checks that stopped most of them used no model on either side: nothing new had arrived, or what arrived was mailbox warm-up traffic with a known marker in it. The local model only saw what survived those, and Claude only saw what survived the local model.

#Its most important job is knowing when to ask for help

Before I gave the box any job, we ran it through a scored battery. Its tool mechanics were clean: 110 trials, zero malformed calls. Asked about things that do not exist, it said not found 60 times out of 60. It does not make things up when it has something to look at.

It does make things up when asked to produce. An early drafting test had it invent invoice details and a signature. It cannot do arithmetic in its head and states a wrong product with the same confidence as a right one. None of that is disqualifying. It is a job description. The model is excellent at retrieval-shaped questions with a bounded answer and unreliable at open-ended generation. So it reads, sorts, and decides, and when the next step is writing or judgment, it hands the work up.

That is the whole discipline of hybrid AI in one sentence: give each model only the jobs it is safe to be wrong at.

The hand-off itself is built around the local model's fallibility, because a gate that can bury a real message is worse than no gate.

It fails open. If the box is down, slow, or returns something unparseable, the message escalates to Claude. The worst case is a wasted cloud run.

It judges one message per call. An email body is untrusted text written by whoever sent it. Batch ten into one prompt and any one body can talk about the other nine. One at a time, an injected instruction can only misclassify itself, and misclassifying itself lands on the safe side.

It never grades its own homework. When a verdict is wrong, a person corrects it. The model does not get to re-decide.

#Anything that needs me shows up on my phone

The frontier model does not get to act on its own either. A drafted reply, a meeting to book, an outreach sequence to start: each one lands on my phone as an approval. I can tap to send it, edit it, rewrite it, or skip it. Most days that is a few minutes of taps. The system does the work; I keep the decisions that are mine.

This is the part people skip when they describe AI agents, and it is the part that makes the rest tolerable. A model with good judgment and no approval step is a liability with a nice writing style. A model with an approval step is staff.

#Hard, deterministic controls around both

Neither model is trusted with the boundaries. Every send has a hard cap. Every send window is enforced in code. Suppression lists and deduplication run before any model sees a message. If the cloud agent has not run for a day, the local gate is bypassed and it runs regardless of what the box thinks, so a filter that drifts cannot drift far. These are integers and clocks, not prompts, and they do not have opinions.

Run what you can locally. Escalate the hard work to frontier models when needed. And put hard, deterministic controls around what either one is allowed to do. That is the whole design.

#For three weeks, it also ran a wedding

In September my daughter got married. The RSVP mailbox for the wedding ran on the same system, on the same box. Guests wrote in, the box read and sorted the mail, and the replies, in our family's voice, came from Claude. The same split: the local side read, the cloud side wrote, and the ones that mattered came to my phone first.

I switched it off in October, when I found the guest list it was working from was wrong. The model was never the problem. The spreadsheet was. That is also a hybrid AI lesson, and probably the more common one: the failures you will actually have are data failures, and no amount of model quality on either side of the gate fixes a bad source of truth.

#The box

None of this works if the local side costs what a workstation GPU rig costs. The box is a GMKtec EVO-X2 built on AMD's Strix Halo chip, the Ryzen AI MAX+ 395, with 96 GB of memory shared directly between the CPU and the graphics cores. The whole model sits in one pool. That suits gpt-oss-120b, a mixture-of-experts model with about 117 billion parameters in total and about 5 billion active per token: it needs a lot of memory and not much bandwidth. It loads once, uses about 65 GB with its full context, and generates faster than I can read.

When we bought it in May, the alternatives that could hold a model this size were a Mac Studio from 4,000 to 14,000 dollars on back-order, a multi-GPU workstation north of 5,000 dollars, or enterprise hardware in six figures. The EVO-X2 was 2,300 dollars and arrived in four days. A machine in that class running a near-frontier open model is the fact that makes hybrid AI a small-business option and not an enterprise project.

#Why I think business AI is headed here

The economics are lopsided in the right direction. The constant work in any business is high-volume and low-stakes: is this relevant, does this need a person, which bucket does this go in. That work is nearly free on hardware you own. The rare work is high-stakes and generative, and that is exactly where you want the best model money can buy.

The data stays in the building until it has to leave. Every message my companies receive is read first by a machine that phones nobody. Only what needs a real answer goes to the cloud, and what goes is the message that needs answering, not the whole inbox. For my mail that is a nicety. For a clinic, a law firm, or a lender, it is the difference between being allowed to use AI and not.

Two unlike systems are more reliable than one. When the cloud is down, the box keeps sorting and holds the queue. When the box is down, everything escalates and the only cost is money. Neither failure loses a message, and neither model can act outside the rails.

The line between local and cloud will move. Open models get better every quarter, and a year from now the box may be trusted with first drafts. Frontier models get better too, and some of what I send to Claude today will look like overkill. But the shape does not change: a cheap, private, always-on judgment layer that knows what it does not know, in front of an expensive, brilliant one that is only called when it should be, with a person and a set of hard rules around both.

I use it every day. I suspect, eventually, you will too. If you want the setup, ask.

The build and the gate are documented in full at agledger.ai: the box, with the recipe, and the gate, with the measurements and the sections on what the runs do not prove. If you want a tamper-evident record of what an agent like this intended and did, that is what AGLedger is for.