在任务完成或陷入停滞时停止循环
阅读约需 39分钟 分钟,6节中含4个示例。
读完本章,你应当能够根据测试框架为每次运行记录的那一个词,说明这次运行为何停止,并能添加两条规则,用来结束已经不再取得进展的运行。
第 1 章构建了一个循环,它以三种方式之一结束一次运行:模型调用 finish,运行达到 8 步的步数上限,或者模型两次不给出调用。这样的循环可以处理会自行完成任务的模型,却处理不了一直重复发送同一个编辑的模型,而找不到目标文本的编辑正容易引发这种情况。第 1 章的工具里没有编辑工具,所以这种情况出自任务集:测试智能体一节评分的 20 个任务,使用工具集 v1 运行,其中的工具包含编辑。在贯穿本书的案例 unitcraft(一个小型单位库,其可见测试套件会在每次改动之后给出一个计数)中,这类停滞的运行会一直跑到上限,而它结束的唯一记录,就是上限把它结束了。每浪费一步,就是一次模型调用,也是对话记录中审阅者必须阅读的一行。本章列出循环结束一次运行的各种方式,添加两条让停滞的运行提前结束的规则,并在任务集上衡量这两条规则的效果。
运行结束的方式
每次运行都因一个明确的原因而结束,harness 会把该原因写入这次运行的 ended 字段。其中两种原因由模型选择:调用 finish 结束运行,或者调用 ask_user 向人提出一个问题。三种是 harness 设定的限制:步数预算、重复调用规则和无进展规则。第六种 no_tool_call 由不给出任何调用的模型造成。这六种由循环设定,而权限关卡在第 6 章中又增加了第七种。下表列出这六种。
ended |
设定时机 | 体现于 |
|---|---|---|
finish |
模型调用 finish,运行随之结束 |
任务集计数 |
ask_user |
模型调用 ask_user 并附带一个问题,且未做任何更改 |
任务集计数 |
step_cap |
运行达到其步数预算 | budget-stops-a-loop |
repeated_call |
同一个调用连续出现三次 | repeated-call-stops |
no_progress |
连续两次编辑后,测试计数都没有变化 | no-progress-stops |
no_tool_call |
模型连续两次不给出调用 | 任务集计数,其值为 0 |
任务集计数是提问、停止与计数中最后一个示例打印出的各行。任务集使用的工具集 v1 中,finish 只接收一份摘要:它以一份摘要结束运行,摘要说明这次运行已对其临时副本做出的编辑,并且不写入任何文件。第 1 章的 finish 在结束运行之前还会写入一个文件。本章度量的工具集则不会这样做。
当同一步上有不止一种结束方式适用时,循环按固定顺序检查。先处理 finish 或 ask_user 调用,然后是重复调用规则,再后是无进展规则。预算在循环用完步数时才起作用,因此最后检查。这个顺序只在两种结束方式于同一步相遇时才有意义,本章其余部分只讨论单一情形。
没有调用的回复会得到一次重试。连续第二次没有调用的回复,会使运行以 no_tool_call 结束。本章没有任何示例会走到这一步,因此这里的内容都不依赖它。
每种结束方式都是一个值,而不是最后一条回复中的一句话,所以报告无需阅读那条回复,就能说明发生的是哪一种。在生产环境中,从外部很难看出这些结束方式之间的差别。以 step_cap 结束的运行,看起来像一次没有完成的缓慢运行,只读最终消息的审阅者无法把它与一次即将完成的运行区分开。ended 值能在报告中显示这一差别,当一次运行的花费超出其任务所需时,它是首先要检查的内容。
这些名称的意义不止于本章。权限关卡是 finish 调用所提议的更改等待人表示同意的地方。子代理与交接把一次停止转化为向人提出的问题,测试代理则对各种结束方式计数。它们都会读取这些值,所以在这里更改名称,那些地方也要随之更改。
我们列出的是 harness 能够观察到的结束方式,而不是运行可能失败的所有原因。一个给出错误答案并调用 finish 的模型,会以 finish 结束,只有检查(即为最终副本打分的隐藏测试)才能表明它失败了。这就是完成数来自检查而不是来自 ended 值的原因,下面的工作准则会再回到这一点。Agent SDK 的文档How the agent loop works以同样的方式描述其结束方式:达到轮次上限的运行,会以子类型为 error_max_turns 且没有最终响应的结果结束。在该 SDK 中,一个轮次是循环的一次完整通过,即一条模型回复及其工具调用,而 max_turns 统计这些通过的次数,再加上最后一条只有文本的回复所占的一次。在这里,这样的一次通过就是一步。本书把“轮次”一词留给与交办任务的人之间的一次交流。同一页还提醒,没有限制的循环在开放式提示下可能会运行很长时间。本书不使用该 SDK,所以我们引用那一页是为了借鉴其设计,而不是引用任何代码。
工作准则: 每次运行都要记录一个 ended 值,报告运行结果时绝不能缺少它。判定一次运行是否完成,要依据评分中的规则,绝不能只依据它的 ended 值。
这部分尚未翻译,因此以英文显示。
The budget counts steps
A model that never calls finish gives the loop no reason to stop, so the loop needs a count that does. A step is one call to the model and the reply it returns. The step budget is the most steps a run may take, and the loop stops when it reaches that number. Steps are counted, not timed: the loop reads no clock.
The task set, the 20 tasks this chapter's last example counts, runs on tool set v1 with a budget of 16 steps. Chapter 1 uses a smaller budget, 8.
The chapter 1 recording of T-01 shows the budget at work. The model, Qwen3-4B-Instruct-2507, is an open model, and its replies were recorded and are replayed here, so every run in this chapter gets the same output on every machine. In that recording the model did not finish within eight steps. Its last step was a malformed tool call, which the harness refused. Replay the run under the budget, with no other rule, and the output shows where the cap ends it:
1"""The chapter 1 recorded run of T-01 under its step budget of 8. The budget ends the run as step_cap, with no finish;2the last line shows the model's final step, a call the harness refused, and the budget ended the run after it."""3from lib.case import load_tasks4from lib.loop import run_task5from lib.models import ReplayModel6from lib.tools import CH17from lib.trace import IdCounter89task = load_tasks(only=["T-01"])[0]10run = run_task(ReplayModel("chapter-1-t01", "T-01"), task, CH1, max_steps=CH1.step_cap, ids=IdCounter())11print("budget (steps):", CH1.step_cap)12print("steps:", run.steps)13print("tool calls:", run.tool_calls)14print("ended:", run.ended)15print("last step:", run.trace.lines()[-1])16assert run.ended == "step_cap" and run.steps == CH1.step_cap, "the budget should stop the run on step 8"17assert not run.summary, "a budget stop carries no summary"
python budget-stops-a-loop.pybudget (steps): 8
steps: 8
tool calls: 8
ended: step_cap
last step: step 8: malformed: error: malformed call
The run ends as step_cap after 8 steps, with 8 tool calls, against a budget of 8. The two counts match because each step made one request. The line to keep is the last one. The model's final reply was a call the harness could not parse, and the budget was the only thing that ended the run. Nothing in the record says the model had finished.
A step budget does not bound time, and it does not bound tokens. A tool that hangs holds its step open until it returns, so the timeout belongs to the sandbox, in sandbox and secrets. A long reply uses more tokens inside a single step, and counting tokens is the work of context budgets. The budget bounds model calls and nothing else, so a run that reaches it has used its calls, and nothing more is known about why.
Malformed replies count too. A reply the harness cannot parse is still a model call, so it spends a step, and the budget counts it. A model that keeps producing calls the harness can't read reaches the budget as surely as one that keeps repeating a good call. Both end as step_cap, so only the last few steps show which of the two happened: a repeated good call, or calls the harness couldn't read.
Working rule: Set the step budget from runs on your own tasks, not from a guess. Take the most steps a passing run needed, leave room above it, and write in the code what that room is for.
同一个调用,再来一次
模型见过某个工具调用失败之后,可能一次又一次地发出同一个调用,而循环会逐个执行,直到预算耗尽并结束运行。人们容易以为模型会察觉自己在循环。Yao 等人的 ReAct 论文(2022 年,发表于 ICLR 2023)把重复列为一种常见的错误模式:在其问答实验中,模型反复生成它已经生成过的思考和动作。该论文的任务是问答和决策,不是代码,所以它只给了我们留意重复的理由,并没有给出重复的发生率。出处是 ReAct: Synergizing Reasoning and Acting in Language Models。检测重复必须由运行框架自己完成。
本书采用的规则很简单。连续三次相同的调用,就以 repeated_call 结束运行。两个调用的工具名相同、参数相同,即视为相同,参数比较时对键排序。排序很重要。两条回复可能携带相同的参数,只是键的顺序不同,此时原始文本不同,调用本身却没有不同。在本书固定的 Python 版本 3.12.15 下,对于同样的两个键、两种顺序,json.dumps 给出两个不同的字符串,即 plain dumps equal: False;而对键排序之后,得到的是同一个字符串,即 sort_keys dumps equal: True。这是我们运行过的一次探测,不是本书中的示例,请把它当作可以重复进行的检查。
本书把阈值定为三次,但这并不是固定值。它是一种判断,下面的工作准则会说明何时调整它。解析出的调用与前一个不同时,计数重新开始;回复中的调用块格式错误时也一样,因为它没有可供比较的已解析调用。回复在词元上限处被截断,或者回复中没有调用,则连续计数保持原样。循环对这些回复跳过连续检查,因此夹在其两侧的相同调用仍然一起计数。
T-01 从不连续三次发送同一个调用,所以这个示例重放 T-02,其中模型确实这样做了:
1"""The recorded run of T-02 with and without the repeated-call rule. Three identical calls in a row end the run as2repeated_call. The rule compares the name and the sorted arguments, so reordered keys still count as the same call."""3from lib.case import load_tasks4from lib.loop import Stops, run_task5from lib.models import ReplayModel6from lib.tools import V17from lib.trace import IdCounter89task = load_tasks(only=["T-02"])[0]101112def replay(stops: Stops):13 ids = IdCounter()14 ids.next("run") # the recording numbers its runs in task order, so T-02 is run-0002: skip T-01's number15 return run_task(ReplayModel("task-set", "T-02"), task, V1, max_steps=V1.step_cap, ids=ids, stops=stops)161718plain = replay(Stops())19print("without the rule, ended:", plain.ended, "at step", plain.steps)20run = replay(Stops(repeat_limit=3))21print("with the rule, ended:", run.ended, "at step", run.steps)22print("repeated call:", run.stuck_on)23print("first of the identical calls: step", run.streak_from)24print("identical calls in a row:", run.steps - run.streak_from + 1)25print("last two steps:", *run.trace.lines()[-2:], sep="\n ")26assert run.ended == "repeated_call" and run.streak_from == 11 and run.steps == 13
python repeated-call-stops.pywithout the rule, ended: step_cap at step 16
with the rule, ended: repeated_call at step 13
repeated call: edit_file
first of the identical calls: step 11
identical calls in a row: 3
last two steps:
step 12: edit_file: error: old text not found in unitcraft/format.py
step 13: edit_file: error: old text not found in unitcraft/format.py
没有这条规则时,运行以 step_cap at step 16 结束。有这条规则时,运行以 repeated_call at step 13 结束。关键在最后几步:同一个 edit_file 调用被再次发送,每次都以 old text not found in unitcraft/format.py 失败。相同调用中的第一个出现在 step 11,连续计数达到 3 之后,规则才结束运行。该示例对结束方式和停止的步骤都做了断言,所以任何改变停止位置的修改都会使运行失败,而不会悄悄改变打印出的输出。
如果模型交替进行读取和编辑,就永远不会触发这条规则,因为每次读取都会打断连续计数。下一条规则针对的就是这种情况。
这条规则针对的是格式正确的同一个调用原样重复出现。错误的回复是另一种失败,工具模式一节讲的是错误调用如何以模型能读懂的类型化错误返回。
工作准则: 按工具名和排序后的参数比较调用,连续三次相同的调用即结束运行。只有在你自己的轨迹显示,连续三次相同的调用其实仍在取得进展时,才提高阈值。然后在你的任务集上测量更高的数值,再决定是否保留。
编辑之间没有进展
如果模型交替进行读取和编辑,就不会连续三次发送同一个调用,因此重复调用规则无从计数,即使运行毫无进展也是如此。适合这种情况的信号是测试计数。每次编辑之后,测试框架会在临时副本上运行可见测试(即该用例自带的测试套件),并保留它所生成摘要中的计数行,例如 8 failed, 22 passed。模型看不到这次运行,测试框架将其留作自己的判断依据。
这正是 Building effective agents 中的指导所要求的那类信号。该指导指出,对智能体而言,在每一步从环境中获得真实依据至关重要,而测试计数就是这种真实依据。不过,这一指导只是 2024 年 12 月一篇文章中的观点,不是测量结果。
该规则会记录连续多少次编辑没有改变计数。第一次编辑与编辑之前的计数相比较。如果一次编辑使计数发生变化,记录就归零。读取不影响这个记录。连续两次就以 no_progress 结束运行。失败的编辑不会改动文件,所以它的计数保持不变,规则会把它算作没有进展。
对任务集记录中的 T-01 运行两次,它使用工具集 v1,其中的编辑都会失败:一次不设任何规则,一次把无进展规则设为两次编辑。请看每次运行在何处结束:
1"""The recorded run of T-01 under the no-progress rule. Two edits in a row that leave the test counts where they2were end the run as no_progress. The counts come from the harness's own run of the visible tests after each edit."""3from lib.case import load_tasks4from lib.loop import EDITS, Stops, run_task5from lib.models import ReplayModel6from lib.tools import V17from lib.trace import IdCounter89task = load_tasks(only=["T-01"])[0]101112def replay(stops: Stops):13 return run_task(ReplayModel("task-set", "T-01"), task, V1, max_steps=V1.step_cap, ids=IdCounter(), stops=stops)141516plain = replay(Stops())17print("without the rule, ended:", plain.ended, "at step", plain.steps)18run = replay(Stops(no_progress_limit=2))19print("with the rule, ended:", run.ended, "at step", run.steps)20edit_steps = [r["step"] for r in run.trace.records if r["tool"] in EDITS]21print("edits that left the counts unchanged, steps:", " and ".join(str(step) for step in edit_steps[-2:]))22print("counts after the last edit:", run.counts)23print("last two steps:", *run.trace.lines()[-2:], sep="\n ")24assert run.ended == "no_progress" and run.steps == 5
python no-progress-stops.pywithout the rule, ended: step_cap at step 16
with the rule, ended: no_progress at step 5
edits that left the counts unchanged, steps: 3 and 5
counts after the last edit: 8 failed, 22 passed
last two steps:
step 4: read_file: 23 lines
step 5: edit_file: error: old text not found in unitcraft/format.py
没有这条规则时,运行以 step_cap at step 16 结束。有这条规则时,运行以 no_progress at step 5 结束,结束于计数未变的两次编辑中的第二次。这些编辑位于 3 and 5。两次都以 old text not found 失败,因此都没有改变计数,计数一直停留在 8 failed, 22 passed。输出中的最后两步是一次读取和那第二次失败的编辑。读取位于两次编辑之间,不会使记录归零,所以规则可以在记录中并不相邻的编辑上触发。
规则只读取测试摘要中的一行:计数行,例如 8 failed, 22 passed。它不读取是哪个测试失败。因此,如果一次编辑修复了一个测试却破坏了另一个,计数不变,看起来就像没有进展。比较文件也无济于事。失败的编辑不会改动文件,但确实改动了文件的编辑仍可能使计数保持不变。我们选择计数,是因为测试框架无需读取测试名称就能比较它们。请在你自己的运行中检验这一选择:如果以一个失败替换另一个失败,就会表现为一次停止。
两次失败的编辑就足以停止这次运行,尽管第三次编辑也许会成功。我们选择了两次,本书并不断言以这种方式停止的运行本来有多大可能最终成功。在任务集中,有 11 次运行以这种方式结束,按照评分中的规则,其中没有一次算作完成。
失败的编辑正是工具模式要解决的失败。第 3 章改变了找不到目标文本的编辑所返回的内容:返回文件中最接近的几行,这样模型可以修正自己的文本,而不是再次发送同样的文本,正如指出最近行的编辑错误所示。
工作准则: 关注测试框架在每次编辑之后从测试中得到的计数,而不是模型自称改动了什么。连续两次编辑没有改变计数时,停止运行。将其记录为 no_progress,并把它停止时的计数与该次运行放在一起保存。
这部分尚未翻译,因此以英文显示。
Asking, stopping and counting the endings
Once a rule ends a run, nothing in the loop asks the person who gave the task what to do next.
Only ask_user asks. The model calls it with a question, the harness hands the question back, and no change is made. The stop rules work differently. A repeated_call or no_progress stop records the rule that fired and the call or the counts it stopped on, and the run is over. Turning such a stop into a question for a person is the handoff in subagents and handoffs. What a change may do without a person's yes is the gate in permission gates.
The last check in this chapter runs all 20 tasks twice, once as recorded and once with both rules, repeated calls and no progress. It prints counts only:
1"""The task-set recording's 20 runs, replayed as recorded and again with both stop rules. Counts only: which runs hit2the step cap, which rule ended the runs that stopped early, and whether any done task is cut short."""3from collections import Counter45from lib.case import load_tasks6from lib.loop import Stops, run_task7from lib.models import ReplayModel8from lib.tasks import score9from lib.tools import V110from lib.trace import IdCounter1112RULES = Stops(repeat_limit=3, no_progress_limit=2)131415def replay(stops: Stops) -> dict:16 """Every task in file order, so the run IDs match the recording's. Maps task ID to (ended, steps, done)."""17 ids, out = IdCounter(), {}18 for task in load_tasks():19 run = run_task(ReplayModel("task-set", task.id), task, V1, max_steps=V1.step_cap, ids=ids, stops=stops)20 out[task.id] = (run.ended, run.steps, score(task, run))21 run.close()22 return out232425ENDS = ("finish", "ask_user", "step_cap", "repeated_call", "no_progress", "no_tool_call")26plain, ruled = replay(Stops()), replay(RULES)27capped = sorted(t for t, (ended, _, _) in plain.items() if ended == "step_cap")28ruled_ends = Counter(ended for ended, _, _ in ruled.values())29done = sorted(t for t, (_, _, ok) in plain.items() if ok)30print("task set step cap:", V1.step_cap)31print("runs:", len(plain))32print("step cap without the rules:", len(capped))33print("step cap without the rules, tasks:", ", ".join(capped))34print("step cap with the rules:", ruled_ends["step_cap"])35print("ended early with the rules:", len([t for t in capped if ruled[t][0] != "step_cap"]))36for ended in ENDS:37 print(f" {ended}: {ruled_ends[ended]}")38print("done without the rules:", len(done))39print("done without the rules, tasks:", ", ".join(done))40print("done with the rules:", sum(1 for _, _, ok in ruled.values() if ok))41print("done with the rules, by finish:", sum(1 for ended, _, ok in ruled.values() if ended == "finish" and ok))42print("finish but not done, with the rules:", sum(1 for ended, _, ok in ruled.values() if ended == "finish" and not ok))43print("done with the rules, by ask_user:", sum(1 for ended, _, ok in ruled.values() if ended == "ask_user" and ok))44wrong = [t for t, (ended, _, ok) in plain.items() if ended == "ask_user" and not ok]45print("ask_user but not done, without the rules:", len(wrong))46print("of those, cut early with the rules:", len([t for t in wrong if ruled[t][0] != "ask_user"]))47assert len(capped) == 7 and all(ruled[t][2] for t in done), "a done task must survive the rules"48assert sum(ruled_ends.values()) == len(plain), "every run has one ending"
python stop-rules-over-task-set.pytask set step cap: 16
runs: 20
step cap without the rules: 7
step cap without the rules, tasks: T-01, T-02, T-06, T-10, T-11, T-13, T-16
step cap with the rules: 1
ended early with the rules: 6
finish: 6
ask_user: 1
step_cap: 1
repeated_call: 1
no_progress: 11
no_tool_call: 0
done without the rules: 5
done without the rules, tasks: T-04, T-05, T-09, T-14, T-18
done with the rules: 5
done with the rules, by finish: 4
finish but not done, with the rules: 2
done with the rules, by ask_user: 1
ask_user but not done, without the rules: 6
of those, cut early with the rules: 6
The rules cut stuck runs short without losing a done task. Without them, the budget ends 7 of the 20 runs; with them, only 1 still reaches it.
With the rules on, the most common end was no_progress, at 11 runs, and finish came next, at 6 runs.
Each done task still passes with the rules on, and the example checks it. Of the 5 done runs, 4 end with finish and pass their check, and 1 ends with the ask_user on T-18, which is the fifth. The other 2 finish runs fail their check, so they are not done.
Without the rules, 6 runs asked a question and were not done. With the rules, 6 of those runs ended earlier, and none of them asked. The one ask_user left with the rules on is T-18's, which the scorer counts as done; chapter 10 shows that it counts only through a word in a path. A stop that ends a run early doesn't make the run right. It ends the run before the wrong question is asked, and that run is still not done. The rules cut the waste; they can't tell a right question from a wrong one.
You can check the output yourself. The six endings with the rules on add up to the number of runs, as the lines of the output show, and the done count is the same with the rules and without them. If the done count ever moves, a rule has cut a run that was going to pass, and the rule is the thing to change. If a run in your own work ends stuck, read it in this order: the ended value, then the last few steps, then the first step where a call began to repeat. Those reads take less time than reading the whole transcript.
Working rule: Count each run by the ended value it recorded, and report the counts with the rules on and off, next to the done count. A stop can only turn a run into one that is not done, so when you change a rule, check that the done count holds before you keep the change.
本章的决定
下列每一项都是本书选定的默认做法,而不是定律。工作准则会说明何时应当改动其中某一项。改动应当先在你自己的任务集上进行,在别人依赖它之前,你可以先数一数它对已完成任务产生了什么影响。
- 停止规则是可选的,在循环中默认关闭。不启用停止规则的运行,发出的请求与第 1 章的循环相同。
- 重复调用指连续三次相同的调用,按工具名称和排序后的参数进行比较。
- 无进展指连续两次编辑都没有改变测试框架的测试计数。第一次编辑与编辑之前的计数相比较。
- 停止会让运行以一个结果结束。没有任何停止规则会去询问人,移交要等到子智能体与移交一章再讲。
- 步数预算统计模型调用的次数,包括格式错误的回复。本章不把它当作对时间或词元的限制。
- 六种结束方式的名称保持固定。门控会在第 6 章增加一种。
- 写明所依据的规则及其计数的停止,读者可以核对;只写明预算的停止,则会让读者把每一步重新查看一遍。
本章,在本版次中
版次 2026.11.2示例于 2026年11月20日 通过本章 4个示例 个示例中有 4 个通过
译自英文版图书,版次 2026.11.2。