Agent Harnesses
Written with AI. Verified by people. How we make books
This book is open to contributions, so anyone can check it or correct it. Its repository · Report an error · How to fix one
Build the code around a model that makes an agent you can trust with real tools: the loop, tools, context, permissions, sandboxing, subagents and tests, on a maintenance agent for a small Python library.
Who it's for For engineers who have called a model's API and built a demo agent, and now must make an agent that works on real tools without a person watching every step. Writes Python and knows JSON and HTTP. No model internals or MCP background needed.
Read on Kindle or any e-reader
The EPUB also works in Apple Books, Kobo and most e-readers.
Send to Kindle
Sends this edition's EPUB in English.
-
Approve our address
Amazon delivers to a Kindle only from addresses you've approved, once for each service that emails it. Add ours to your Approved Personal Document E-mail List.
books@rollingedition.comOpen Amazon's Personal Document Settings. On Amazon, it's under Manage Your Content and Devices, Preferences, Personal Document Settings.
Sent. It'll appear in your Kindle library in a few minutes.
If it doesn't arrive, it's almost always the approval in step 1: check that books@rollingedition.com is on your Approved Personal Document E-mail List.
Each new edition of your saved books will go to your Kindle too.
That isn't a Kindle address. It ends in @kindle.com: find it in Amazon's Personal Document Settings.
You've sent {count} books today, the most we send for one account in a day. Try again tomorrow.
We've sent as many books as we can today. Try again tomorrow.
The book didn't send. Try again in a few minutes.
This edition is too large to email.
Send to Kindle comes with Labs.
That isn't a Kindle address. It ends in @kindle.com: find it in Amazon's Personal Document Settings.
The book didn't send. Try again in a few minutes.
Or download the EPUB and drop it on Amazon's Send to Kindle page.
Kindle is a trademark of Amazon.com, Inc.
Labs
Work through this book in the Lab
Run every example exactly as the book did, change the code to see what happens, and try your own ideas on the book's code and data. Nothing to install, and your work is saved. Opens in about 30 seconds.
Run every example exactly as the book did, change the code to see what happens, and try your own ideas on the book's code and data. Nothing to install, and your work is saved. Labs runs it for you: 40 hours a month.
Checking your hours…
Checking for your Lab…
On your own machine, free: download the book's notebooks, with its code and data, from GitHub.
Contents
12 chapters, about 3 hours 41 minutes to read or 4 hours 17 minutes to listen to, with 64 examples.
-
1Give a model tools and a loop
By the end of this chapter you can run a model in a loop that gives it three tools: one reads a file, one runs the tests, and one finishes with a changed file.
-
2Stop the loop when it's done or stuck
By the end of this chapter you can say why any run stopped, from the one word the harness records for it, and add the two rules that end a run which has stopped making progress.
-
3Give the model tools it can't misuse
By the end of this chapter you can give a model tools whose arguments the harness checks before any tool runs, and write refusals the model can act on.
-
4One interface for scripted, replayed, local and hosted models
By the end of this chapter you can run the same loop on a scripted model, a replayed run, a local model and an OpenAI-compatible endpoint, without changing the loop.
-
5Keep the context useful as the task grows
By the end of this chapter you can work out how much room a run's prompt has, estimate a prompt's size when the harness cannot count it exactly, and compact a run's older steps without cutting a tool call away from its result.
-
6Gate the tools that change things
By the end of this chapter you can keep an agent's changes in its own copy until a person says yes, check that the yes covers the exact change they read, and hand a no back to the agent as the reason its run stopped.
-
7Run the agent's code in a sandbox, and keep secrets out of its view
By the end of this chapter you can run a command for the agent with a time limit, per-process memory, CPU and file-size caps, an environment built from an allowlist, and a scratch copy as its working folder.
-
8Hand a subtask to a child, and hand back a stop
By the end of this chapter you can give a subtask to a child loop with a step budget of its own, and get back a summary and a stop reason in place of the child's whole transcript.
-
9Connect the harness to MCP servers
By the end of this chapter you can connect the harness to MCP servers that other people run, see each server's tools under names that cannot clash with the harness's own, and send every call to one of those tools through the approval gate. Chapter 6 built that gate and said that MCP tools would go through its approvals.
-
10Test the agent, then measure it on a task set
By the end of this chapter you can test the harness on scripted replies, score the agent on the task set with one rule for done, and state the count with its range.
-
11Trace each run and count what it cost
By the end of this chapter, you can make every run write one record per step to a file that a later reader can parse, give each record IDs that tie it to its run and its call, and say what the run cost in tokens.
-
12When an agent is the wrong design
By the end of this chapter you can take any task in the case, sort it into a fixed pipeline, a single model call or an agent with gates, and say what the recorded runs show for it.