How to Reduce LLM Costs With a Self-Improving AI Agent

Part two: how to reduce LLM costs with a System 1 model that learns

In part one we made the case that your AI agents need two brains. A fast System 1 for snap decisions, and a slow System 2 for real reasoning. This post is about what happens next, and it is where the money is.

If you want to reduce LLM costs, a cheaper model is only half the answer. The other half is a System 1 that gets smarter every day it runs, so it takes more of the work off your expensive models every week. We built exactly that this week inside Henry, our AI chief of staff. Here is how it works, what it costs, and where the savings are real.

Watch the four minute walkthrough

What one public test showed

The idea of a self-improving System 1 is not theory. One public creator test ran Jev on a real inbox of 328 emails, sorting sponsor pitches, agency mail, personal notes and junk. Jev acted only when it was at least 90 percent sure. Everything else went to Opus, which handled the email and proposed one edit to the rulebook Jev reads on every call.

That last number matters most. A loop that accepts every change will drift. A loop that rejects bad changes is one you can trust. These are the creator’s own figures, not an independent benchmark, but they match what we see in our own early data.

The learning loop: the engine that reduces LLM costs every day

The learning loop turns every hard case into a permanent improvement. Here is the cycle, step by step:

• Jev answers first. It reads the situation, the question, and the rulebook, and returns a probability for every option in about a quarter of a second.
• Sure means done. If the top answer clears the bar, 90 percent in our case, Jev’s answer stands.
• Unsure means escalate. The big model handles the item, explains why, and proposes one edit to the rulebook.
• Every edit gets re-tested. The new rulebook runs against past decisions. It must be more right and break nothing, or it is thrown away.
• A person approves at first. For the first two weeks Chad signs off on every rulebook change. After that the re-test gate runs on its own.

Every version of the rulebook lives in version control, and every decision is logged with the rulebook version that made it. If something regresses, you can trace it to one edit and roll it back in seconds.

Start in shadow mode, not with a switch

Shadow mode means Jev watches before it works. For ten days it answers every request and logs what it would have done, while nothing changes. That gives you a labeled history from real work before Jev earns any authority, and it is the data the learning loop needs to re-test against.

Our first night already paid off. A short request like “commit and push” came back unclear at 0.44, because Jev sees one line, not the whole conversation. A person reads that instantly. Now it is a known gap with a clear fix: short follow-ups mean keep the current model. That is exactly the kind of lesson the loop exists to capture.

Use System 1 to pick the right Claude model

Most teams run everything on their biggest model by default. We kept Opus as Henry’s default and added one quiet question to every request: what is the cheapest Claude model that will do this well, and how much effort does it need?

• Recommend only. Henry shows a suggestion like “sonnet, low effort, 93 percent sure.” A person says yes or ignores it.
• A high bar. Below 90 percent, or when the answer is unclear, Jev stays silent and the default stands.
• Near zero overhead. The call runs in the background and adds about twenty milliseconds.
• Fails safe. Any error, timeout, or kill switch keeps the default model.
• Confidential stays local. A topic filter stops sensitive requests from ever leaving the machine.

Why not switch automatically? Because switching models mid-session breaks the prompt cache. One open source router experiment spent $106.73 against an $87.19 baseline over 309 requests for exactly that reason. Routing that ignores caching can cost you more than no routing at all.

The honest math on reducing LLM costs

How much you save depends entirely on which model makes the decision today. Here is the shape at one million decisions a day, assuming 1,500 input and 50 output tokens per decision at list prices, with Jev handling 80 percent after the loop matures and the rest going up to today’s model.

Illustrative figures, not a client’s numbers. The pattern is what matters. Hover the cards for what each case means in practice.

Where to start

Reduce LLM costs FAQ

How does a learning loop reduce LLM costs?

Every time the fast model is unsure, the expensive model handles the item and proposes a rule fix. Fixes that pass a re-test make the fast model right more often, so it handles a bigger share of the work each week. Fewer expensive calls means a smaller bill without anyone rewriting code.

What is shadow mode for AI agents?

Shadow mode runs a new model next to the current decider and logs its answers without acting on them. It builds a labeled history from real work, shows you how often the new model agrees, and gives the learning loop data to re-test against before anything goes live.

Is it safe to send company data to a decision model?

Only with the right route. Skip confidential topics entirely, and for anything sensitive use a zero data retention route. Several gateways, including OpenRouter, list Jev as a zero retention endpoint. Regulated companies should get retention terms in writing.

We build the loop, not just the agent

Cirrius designs AI agent pipelines that put every decision in the right place: Jev as System 1 for the snap calls, and Claude, OpenAI, Agentforce, and Salesforce as System 2 for the reasoning, with Salesforce as the system of record underneath it all. Then we add the learning loop, so the whole system gets cheaper every week it runs.

Want to find the System 1 decisions hiding in your business and start reducing LLM costs? Talk to our team.