What Does It Take to Run an AI Agent in a Real Business? Cost, Monitoring and Rollout

Imagine the Lagos fabric shop from the first post has an agent that did well in testing. On Friday the owner says: “Let’s switch it on for every customer on Monday.” Before she does, she has questions. What will it cost? What if the provider cuts it off? How will anyone know it is getting things wrong? Who can switch it off?

(This is an illustrative scene, not a real business.)

Why is running an agent harder than building one?

Because a demo only shows it can work. Anthropic’s engineers, writing about their multi-agent research system on 13 June 2025, put it this way: “the gap between prototype and production is often wider than anticipated”. In our reading, the same is true for a small agent. This post covers the gap in four parts: cost, limits, monitoring and rollout.

What does an AI agent cost to run?

Model providers such as Anthropic and OpenAI charge by the token, the small piece of text we met in the seventh post, and they price input and output separately. Anthropic’s pricing page, as of 24 October 2026, lists US dollar prices per million tokens (written “MTok”). It says “All prices are in USD” and points to claude.com/pricing for the most current figures. Four of its rows:

Model	Input per MTok	Output per MTok
Claude Haiku 5.5 (prompts up to 100k tokens)	$0.10	$0.50
Claude Sonnet 5.5	$2	$10
Claude Opus 5.5	$4	$20
Claude Fable 5.1	$10	$50

On these rows, output costs five times as much as input.

Illustrative example. Suppose the shop’s agent makes 10 model calls to handle one customer message. In our reading, each call sends the earlier steps again, as the second post explained. Say each call averages 5,000 input tokens and 500 output tokens. Later calls are usually bigger, because they carry the earlier steps. That is 50,000 input and 5,000 output tokens per task. These numbers are made up. At the prices above:

Model	Cost per task	Per 1,000 tasks
Haiku 5.5	$0.0075	$7.50
Sonnet 5.5	$0.15	$150
Opus 5.5	$0.30	$300
Fable 5.1	$0.75	$750

For Sonnet 5.5, 50,000 input tokens at $2 per million is $0.10, and 5,000 output tokens at $10 per million is $0.05. The same job costs 100 times more on the dearest row than the cheapest. In our reading, that does not mean choosing the cheapest, because it may fail more tasks. Test models on your eval set from the tenth post and compare cost per successful task.

The example leaves out four things:

• Caching. Prompt caching lets a provider reuse text that repeats at the start of requests. Anthropic says a cache hit costs 10% of the standard input price or less, depending on the model.
• Batch processing. Anthropic’s Batch API handles requests asynchronously with “a 50 percent discount on both input and output tokens”. In our reading, that suits overnight jobs, not a customer waiting for a reply.
• Tokenizers. Anthropic says newer models use a tokenizer that “produces approximately 30% more tokens for the same text” than earlier models, depending on content. So compare cost per task, not price per MTok.
• Agent overhead. As the ninth post noted, Anthropic’s June 2025 data was that agents typically use about 4 times more tokens than chat, and multi-agent systems about 15 times.

Spend limits stop a bill running away. OpenAI separates a spend alert (“Sends a notification; API traffic continues”) from a hard spend limit (“Affected API requests return a 429 error”). Anthropic says that at an organization’s tier spend cap, “API usage pauses until 00:00 UTC on the first day of the next month”, and that you can set a lower limit of your own. In our reading, set an alert well below the hard limit, so the owner hears about a surge before customers do.

What are rate limits, and what happens when you hit one?

A rate limit caps how much you can send to a provider in a period of time. Anthropic measures requests per minute and input and output tokens per minute, for each model class. OpenAI uses requests and tokens per minute and per day, among other measures, at organization and project level. Limits depend on your usage tier and model, so we do not quote numbers.

If you exceed a limit, you get an HTTP 429 error. HTTP is the protocol web services use to talk to each other, and 429 is its code for too many requests. Anthropic says to wait the number of seconds in the retry-after header when it is present. OpenAI’s rate-limits documentation also says to follow Retry-After. Anthropic says that “acceleration limits” can apply if usage sharply increases, and advises increasing traffic gradually. OpenAI’s rule of thumb is that, once traffic reaches 1 million input tokens per minute, you should increase it by no more than 50% every 15 minutes.

In our reading, the agent’s loop needs a plan for a 429, the same way it needs a step limit. The eighth post’s script caps the steps with MAX_STEPS. OpenAI’s guide advises: “Set limits on agent retries or actions.” It says to hand over to a person when they are exceeded. So: wait as told, retry a few times, then stop and tell a person.

What goes wrong once an agent is live, and what should you watch?

Anthropic’s engineers wrote that “Agents are stateful and errors compound”. They also wrote that “restarts are expensive and frustrating for users”, so they built systems that resume from where the agent was, alongside safeguards like retry logic and regular checkpoints. For diagnosis, they found that “Adding full production tracing let us diagnose why agents failed and fix issues systematically.” By tracing we mean recording each step an agent took. They also say they monitor decision patterns without monitoring the contents of individual conversations, to protect user privacy.

The tests from the tenth post are not enough on their own. Anthropic’s evals post says production monitoring “Catches issues that synthetic evals miss”, but is “Reactive; problems reach users before you know about them”. NIST’s AI Risk Management Framework says systems should be “tested before their deployment and regularly while in operation”, and its MANAGE 4.1 item calls for monitoring plans with “appeal and override, decommissioning, incident response, recovery, and change management”.

In our reading, a small business can start with five numbers:

1. Cost per task, against the estimate above.
2. Steps per task, because a sudden rise may mean the agent is looping.
3. Errors and 429s, to see limits and failures early.
4. How often a person overrides, edits or takes over.
5. Customer complaints, counted by hand if need be.

Keep the traces, review a sample weekly, and turn real failures into new eval tasks.

How do you roll an agent out safely?

Gradually. OpenAI’s guide says: “Start small, validate with real users, and grow capabilities over time.” In our reading, four stages work for most small teams:

1. Test. Run the agent on your eval set in a sandbox, as in the tenth and eleventh posts.
2. Draft only. The agent drafts and a person reviews and sends everything, like the draft and send split in the fifth post.
3. Live on low-risk tasks. Let it act alone on tasks where a mistake is cheap and easy to undo. Keep a person in front of anything sensitive or irreversible.
4. Widen slowly. Add task types or customers in small steps, and stop if the numbers from the last section get worse.

Anthropic’s research system used what it calls rainbow deployments, “gradually shifting traffic from old to new versions while keeping both running simultaneously”. In our reading, small teams can borrow the idea: when you change the prompt, model or tools, keep the old version ready and send a small share of work to the new one first. Keep an off switch, as the eleventh post said. NIST’s MANAGE 2.4 item asks for mechanisms to “supersede, disengage, or deactivate” systems that perform in ways inconsistent with their intended use.

Illustrative example. The shop could run week one in draft mode, with the owner approving every reply. In week two, the agent could send delivery-status answers by itself, while refunds still go to the owner. These steps are made up.

What did the series cover?

This is the last of 12 posts. The path, with links back:

• What an AI agent is: it decides its own next steps and uses tools.
• How agents work: the loop, and when it stops.
• What agents can and cannot do: benchmarks and compounding errors.
• Whether you need one: often a simple automation is enough.
• Tools and function calls: the model asks, and your code acts.
• The Model Context Protocol: connecting tools, and the trust risks.
• Memory, context and RAG: what the model can see.
• Building a first agent: a small loop with a step limit.
• Multi-agent systems: several agents cost more tokens.
• Evaluating agents: tests built from real failures.
• Security: injection, permissions and approval.

Where should you start?

In our reading, a short checklist, in order:

1. Pick one job where the path is predictable, the result is checkable and a mistake is cheap. The fourth post’s four questions help.
2. Check that a plain automation will not do. If it will, use it.
3. Build the smallest version, with a step limit and read-only tools, as in the eighth post.
4. Write 20 to 50 test tasks from real failures, and set a pass mark before you launch (tenth post).
5. Apply the safety checks from the eleventh post: least privilege, approvals, logs and an off switch.
6. Estimate cost per task from the vendor’s current price page, and set a spend alert and a retry limit.
7. Roll out in stages, and review the five numbers every week.

Vendor prices, limits and model names change, so recheck them before you commit.

What to remember

• Estimate cost per task, not price per token, and set spend alerts and retry limits before launch.
• Monitor cost, steps, errors, overrides and complaints, and turn real failures into new tests.
• Roll out in stages, starting with a person reviewing everything, and keep an off switch.

That is the end of AI Agents Explained.