在完成或卡住時停止迴圈
閱讀約需 39 分鐘 分鐘,6 節 中共有 4 個範例。
讀完本章後,你能從 harness 為每次執行記錄的一個詞,說明該次執行為何停止,也能加入兩條規則,讓不再有進展的執行結束。
第 1 章建立的迴圈,會以三種方式結束一次執行:模型呼叫 finish、執行達到 8 步的上限,或模型連續兩次沒有發出呼叫。這涵蓋了能順利完成的模型,卻沒有涵蓋一直重複送出同一個編輯的模型,而找不到目標文字的編輯正容易引發這種情況。第 1 章的工具沒有編輯功能,所以這種情況來自任務集:也就是測試代理程式所評分的 20 個任務,以工具集 v1 執行,其中的工具包含編輯。在貫穿全書的範例 unitcraft 裡,這是一個小型的單位函式庫,其可見的測試套件會在每次變更後給出一個計數;這類卡住的執行會一直跑到上限,而它結束的唯一紀錄,就是上限讓它結束。每浪費一步,就是一次模型呼叫,以及逐字稿中審查者必須閱讀的一行。本章會列出迴圈結束一次執行的各種方式,加入兩條能提早結束卡住執行的規則,並在任務集上衡量這兩條規則的效果。
執行結束的方式
每次執行都會因為一個明確的原因而結束,而 harness 會把這個原因寫入該次執行的 ended 欄位。其中兩種原因由模型決定:模型呼叫 finish 來結束執行,或是呼叫 ask_user 向使用者提出問題。三種是 harness 設定的限制:步驟預算、重複呼叫規則與無進展規則。第六種 no_tool_call 則是在模型沒有給出任何呼叫時設定。這六種都由迴圈設定,而 權限閘門 會在第 6 章再加上第七種。下表列出這六種。
ended |
設定時機 | 呈現方式 |
|---|---|---|
finish |
模型呼叫 finish,結束執行 |
任務集計數 |
ask_user |
模型呼叫 ask_user 並附上問題,且未做出任何變更 |
任務集計數 |
step_cap |
執行達到步驟預算 | budget-stops-a-loop |
repeated_call |
同一個呼叫連續出現三次 | repeated-call-stops |
no_progress |
連續兩次編輯後,測試計數都維持不變 | no-progress-stops |
no_tool_call |
模型連續兩次沒有給出呼叫 | 任務集計數,其值為 0 |
任務集計數是 詢問、停止與計數 中最後一個範例所印出的各行。任務集使用的工具集 v1 中,finish 只接受摘要:它會以一份摘要結束執行,內容是該次執行已對其暫存副本做出的編輯,而且不會寫入任何檔案。第 1 章的 finish 在結束執行之前還會寫入一個檔案,本章所衡量的工具集則不會。
當同一步驟中有不只一種結束條件成立時,迴圈會依固定順序檢查。先處理 finish 或 ask_user 呼叫,接著是重複呼叫規則,再來是無進展規則。預算在迴圈用完步驟時才適用,因此最後檢查。順序只在兩種結束條件於同一步驟相遇時才有影響,本章其餘部分只討論單一條件的情況。
沒有呼叫的回覆可以重試一次。連續第二次出現沒有呼叫的回覆,執行就會以 no_tool_call 結束。本章沒有任何範例會走到這一步,因此這裡的內容不依賴它。
每一種結束方式都是一個值,而不是最後一則回覆中的一句話,所以報告不必閱讀那則回覆,就能說明發生的是哪一種。在實際運作中,從外部很難看出這些結束方式的差異。結束於 step_cap 的執行,看起來像是一次沒有完成的緩慢執行;只讀最後一則訊息的審查者,無法把它與即將完成的執行區分開來。ended 值能在報告中顯示這項差異,而且當某次執行的成本高於其任務應有的水準時,它是第一個該檢查的項目。
這些名稱的重要性不限於本章。權限閘門 是 finish 呼叫所提議的變更等待使用者同意的地方。子代理與交接 會把停止轉為向使用者提出的問題,而 測試代理 則會統計各種結束方式。它們都會讀取這些值,因此在此更動名稱,其他地方也必須跟著更動。
我們列出的是 harness 能觀察到的結束方式,而不是執行可能失敗的所有原因。模型若給出錯誤答案並呼叫 finish,會以 finish 結束,只有 檢查,也就是為最終副本評分的隱藏測試,才會顯示它失敗了。這就是為什麼完成數來自檢查,而不是來自 ended 值,下方的實用原則會再回到這一點。Agent SDK 的文件 How the agent loop works 以相同的方式描述其結束方式:達到輪次上限的執行,會以 subtype 為 error_max_turns 的結果結束,而且沒有最終回應。在該 SDK 中,一輪(turn)是通過其迴圈的一次,也就是一則模型回覆加上它的工具呼叫,而 max_turns 計算的是這些次數再加上最後一則純文字回覆。在這裡,那一次通過就是一個步驟。本書把「輪」保留給與交付任務的人之間的一次交流。同一頁也警告,沒有限制的迴圈在開放式提示下可能執行很長的時間。本書並未使用該 SDK,因此我們引用該頁面是為了其設計,而不是為了任何程式碼。
實用原則: 每次執行都要記錄一個 ended 值,而且報告執行結果時絕不省略它。判定執行是否完成,要依 評分 中的規則,絕不能只看它的 ended 值。
此部分尚未翻譯,因此以英文呈現。
The budget counts steps
A model that never calls finish gives the loop no reason to stop, so the loop needs a count that does. A step is one call to the model and the reply it returns. The step budget is the most steps a run may take, and the loop stops when it reaches that number. Steps are counted, not timed: the loop reads no clock.
The task set, the 20 tasks this chapter's last example counts, runs on tool set v1 with a budget of 16 steps. Chapter 1 uses a smaller budget, 8.
The chapter 1 recording of T-01 shows the budget at work. The model, Qwen3-4B-Instruct-2507, is an open model, and its replies were recorded and are replayed here, so every run in this chapter gets the same output on every machine. In that recording the model did not finish within eight steps. Its last step was a malformed tool call, which the harness refused. Replay the run under the budget, with no other rule, and the output shows where the cap ends it:
1"""The chapter 1 recorded run of T-01 under its step budget of 8. The budget ends the run as step_cap, with no finish;2the last line shows the model's final step, a call the harness refused, and the budget ended the run after it."""3from lib.case import load_tasks4from lib.loop import run_task5from lib.models import ReplayModel6from lib.tools import CH17from lib.trace import IdCounter89task = load_tasks(only=["T-01"])[0]10run = run_task(ReplayModel("chapter-1-t01", "T-01"), task, CH1, max_steps=CH1.step_cap, ids=IdCounter())11print("budget (steps):", CH1.step_cap)12print("steps:", run.steps)13print("tool calls:", run.tool_calls)14print("ended:", run.ended)15print("last step:", run.trace.lines()[-1])16assert run.ended == "step_cap" and run.steps == CH1.step_cap, "the budget should stop the run on step 8"17assert not run.summary, "a budget stop carries no summary"
python budget-stops-a-loop.pybudget (steps): 8
steps: 8
tool calls: 8
ended: step_cap
last step: step 8: malformed: error: malformed call
The run ends as step_cap after 8 steps, with 8 tool calls, against a budget of 8. The two counts match because each step made one request. The line to keep is the last one. The model's final reply was a call the harness could not parse, and the budget was the only thing that ended the run. Nothing in the record says the model had finished.
A step budget does not bound time, and it does not bound tokens. A tool that hangs holds its step open until it returns, so the timeout belongs to the sandbox, in sandbox and secrets. A long reply uses more tokens inside a single step, and counting tokens is the work of context budgets. The budget bounds model calls and nothing else, so a run that reaches it has used its calls, and nothing more is known about why.
Malformed replies count too. A reply the harness cannot parse is still a model call, so it spends a step, and the budget counts it. A model that keeps producing calls the harness can't read reaches the budget as surely as one that keeps repeating a good call. Both end as step_cap, so only the last few steps show which of the two happened: a repeated good call, or calls the harness couldn't read.
Working rule: Set the step budget from runs on your own tasks, not from a guess. Take the most steps a passing run needed, leave room above it, and write in the code what that room is for.
同一個呼叫,再來一次
模型看過某個工具失敗之後,可能一再送出相同的呼叫,而迴圈會逐一執行,直到預算用盡才結束這次執行。人很容易認為模型會察覺自己陷入了迴圈。Yao 等人的 ReAct 論文(2022 年,發表於 ICLR 2023)把重複列為常見的錯誤模式:在它的問答實驗中,模型會一再產生它先前已經產生過的思考與動作。該論文的任務是問答與決策,並非程式碼,因此它只給了我們留意重複的理由,並沒有提供重複的發生率。來源是 ReAct: Synergizing Reasoning and Acting in Language Models。偵測重複必須由 harness 自己來做。
本書採用的規則很簡單:連續三次相同的呼叫,就以 repeated_call 結束這次執行。兩個呼叫若工具名稱相同,且引數在鍵排序後也相同,就視為相同。排序很重要。兩則回覆可能帶有相同的引數,只是鍵的順序不同,這時原始文字不同,呼叫本身卻沒有不同。在本書固定的 Python 版本 3.12.15 下,對於同樣兩個鍵、兩種順序,json.dumps 會給出兩個不同的字串,也就是 plain dumps equal: False;鍵排序之後則只得到一個字串,也就是 sort_keys dumps equal: True。那是我們執行過的一次探測,並不是本書中的範例,所以請把它當作可以自行重複的檢查。
三是本書採用的門檻,並非固定不變。這是一項判斷,下方的實用原則會說明何時該調整。解析後的呼叫只要與前一個不同,計數就會重新開始;含有格式錯誤呼叫區塊的回覆也一樣,因為它沒有可供比較的已解析呼叫。在 token 上限處被截斷的回覆,或沒有呼叫的回覆,則會讓連續計數維持原狀。迴圈對這類回覆會略過連續檢查,因此夾著這類回覆的前後相同呼叫,仍會合併計算。
T-01 從不連續三次送出相同的呼叫,所以這個範例重播的是 T-02,模型在其中確實會這麼做:
1"""The recorded run of T-02 with and without the repeated-call rule. Three identical calls in a row end the run as2repeated_call. The rule compares the name and the sorted arguments, so reordered keys still count as the same call."""3from lib.case import load_tasks4from lib.loop import Stops, run_task5from lib.models import ReplayModel6from lib.tools import V17from lib.trace import IdCounter89task = load_tasks(only=["T-02"])[0]101112def replay(stops: Stops):13 ids = IdCounter()14 ids.next("run") # the recording numbers its runs in task order, so T-02 is run-0002: skip T-01's number15 return run_task(ReplayModel("task-set", "T-02"), task, V1, max_steps=V1.step_cap, ids=ids, stops=stops)161718plain = replay(Stops())19print("without the rule, ended:", plain.ended, "at step", plain.steps)20run = replay(Stops(repeat_limit=3))21print("with the rule, ended:", run.ended, "at step", run.steps)22print("repeated call:", run.stuck_on)23print("first of the identical calls: step", run.streak_from)24print("identical calls in a row:", run.steps - run.streak_from + 1)25print("last two steps:", *run.trace.lines()[-2:], sep="\n ")26assert run.ended == "repeated_call" and run.streak_from == 11 and run.steps == 13
python repeated-call-stops.pywithout the rule, ended: step_cap at step 16
with the rule, ended: repeated_call at step 13
repeated call: edit_file
first of the identical calls: step 11
identical calls in a row: 3
last two steps:
step 12: edit_file: error: old text not found in unitcraft/format.py
step 13: edit_file: error: old text not found in unitcraft/format.py
沒有這條規則時,這次執行以 step_cap at step 16 結束。有了這條規則,執行則以 repeated_call at step 13 結束。重點在最後幾個步驟:同一個 edit_file 呼叫一再送出,每次都以 old text not found in unitcraft/format.py 失敗。相同呼叫中的第一個出現在 step 11,在規則結束這次執行之前,連續次數已達 3。這個範例會斷言結束的狀態與停止的步驟,因此若有改動讓停止點移位,執行會直接失敗,而不是悄悄改變印出的輸出。
模型若交替進行讀取與編輯,就永遠不會觸發這條規則,因為每次讀取都會中斷連續計數。下一條規則就是針對這種情況。
這條規則處理的是格式正確的同一個呼叫原封不動地再次出現。損壞的回覆是另一種失敗,而 tool schemas 談的是錯誤的呼叫如何以模型能讀懂的具型別錯誤退回。
實用原則: 以工具名稱與排序後的引數來比較呼叫,連續三次相同的呼叫就結束這次執行。只有在你自己的追蹤紀錄顯示,連續三次相同的呼叫其實仍在推進時,才提高門檻。接著先在你的任務集上測量較高的數字,再決定是否保留。
跨越多次編輯的無進展
交替進行讀取與編輯的模型,不會連續三次送出相同的呼叫,因此重複呼叫規則沒有東西可計數,即使整個執行過程毫無進展也是如此。適合這種情況的訊號是測試計數。每次編輯之後,測試框架(harness)會在暫存副本上執行可見測試,也就是該案例自身的測試套件,並保留它所建立之摘要中的計數行,例如 8 failed, 22 passed。模型永遠看不到這次執行,測試框架把它留作自己的決策依據。
這正是 Building effective agents 中的指引所要求的那類訊號。該指引指出,代理在每一步從環境取得真實基準(ground truth)至關重要,而測試計數就是這類真實基準。這份指引是 2024 年 12 月一篇文章中的意見,並非測量結果。
這條規則會記錄連續讓計數維持不變的編輯次數。第一次編輯要與尚未進行任何編輯前的計數比較。若某次編輯使計數改變,便將次數重設為零。讀取不會影響它。連續兩次,執行即以 no_progress 結束。失敗的編輯會讓檔案維持原狀,所以它的計數也不變,規則會把它算作無進展。
請以任務集錄製中的 T-01 執行兩次,它使用工具集 v1,且其編輯都會失敗:一次不設任何規則,一次將無進展規則設為兩次編輯。請看每次執行在哪裡結束:
1"""The recorded run of T-01 under the no-progress rule. Two edits in a row that leave the test counts where they2were end the run as no_progress. The counts come from the harness's own run of the visible tests after each edit."""3from lib.case import load_tasks4from lib.loop import EDITS, Stops, run_task5from lib.models import ReplayModel6from lib.tools import V17from lib.trace import IdCounter89task = load_tasks(only=["T-01"])[0]101112def replay(stops: Stops):13 return run_task(ReplayModel("task-set", "T-01"), task, V1, max_steps=V1.step_cap, ids=IdCounter(), stops=stops)141516plain = replay(Stops())17print("without the rule, ended:", plain.ended, "at step", plain.steps)18run = replay(Stops(no_progress_limit=2))19print("with the rule, ended:", run.ended, "at step", run.steps)20edit_steps = [r["step"] for r in run.trace.records if r["tool"] in EDITS]21print("edits that left the counts unchanged, steps:", " and ".join(str(step) for step in edit_steps[-2:]))22print("counts after the last edit:", run.counts)23print("last two steps:", *run.trace.lines()[-2:], sep="\n ")24assert run.ended == "no_progress" and run.steps == 5
python no-progress-stops.pywithout the rule, ended: step_cap at step 16
with the rule, ended: no_progress at step 5
edits that left the counts unchanged, steps: 3 and 5
counts after the last edit: 8 failed, 22 passed
last two steps:
step 4: read_file: 23 lines
step 5: edit_file: error: old text not found in unitcraft/format.py
沒有規則時,執行以 step_cap at step 16 結束。有規則時,執行以 no_progress at step 5 結束,停在兩次讓計數不變的編輯中的第二次。這些編輯位於 3 and 5。兩次都以 old text not found 失敗,所以都沒有改變計數,計數一直維持在 8 failed, 22 passed。輸出中的最後兩個步驟是一次讀取,以及第二次失敗的編輯。讀取位於兩次編輯之間,並不會重設計數,這也就是規則為何能在逐字記錄中並不相鄰的編輯上觸發。
規則只讀取測試摘要中的一行:計數行,例如 8 failed, 22 passed。它不會讀取是哪個測試失敗。因此,一次編輯修好了某個測試卻弄壞另一個,而計數相同,看起來就像沒有進展。比較檔案也沒有幫助。失敗的編輯會讓檔案不變,但確實改變檔案的編輯仍可能讓計數維持不變。我們採用計數,是因為測試框架不必讀取測試名稱就能比較它們。請在你自己的執行中檢驗這項選擇,在那裡,以一個失敗換掉另一個失敗的情形會顯示為停止。
兩次失敗的編輯就足以停止這次執行,儘管第三次編輯也許會成功。我們選了兩次,本書並未宣稱以這種方式停止的執行,之後繼續下去成功的頻率有多高。在任務集中,有 11 次執行以這種方式結束,而依據評分中的規則,其中沒有任何一次算完成。
失敗的編輯,正是工具綱要所處理的失敗。第 3 章會改變找不到其文字的編輯所回傳的內容:回傳檔案中最接近的幾行,讓模型能修正它的文字,而不是再次送出,如指出最接近行的編輯錯誤所示。
實用原則: 要觀察測試框架在每次編輯後從測試得到的計數,而不是模型所說它改了什麼。連續兩次編輯都讓計數不變時,就停止執行。將它記錄為 no_progress,並把它停止時的計數與該次執行一併保存。
此部分尚未翻譯,因此以英文呈現。
Asking, stopping and counting the endings
Once a rule ends a run, nothing in the loop asks the person who gave the task what to do next.
Only ask_user asks. The model calls it with a question, the harness hands the question back, and no change is made. The stop rules work differently. A repeated_call or no_progress stop records the rule that fired and the call or the counts it stopped on, and the run is over. Turning such a stop into a question for a person is the handoff in subagents and handoffs. What a change may do without a person's yes is the gate in permission gates.
The last check in this chapter runs all 20 tasks twice, once as recorded and once with both rules, repeated calls and no progress. It prints counts only:
1"""The task-set recording's 20 runs, replayed as recorded and again with both stop rules. Counts only: which runs hit2the step cap, which rule ended the runs that stopped early, and whether any done task is cut short."""3from collections import Counter45from lib.case import load_tasks6from lib.loop import Stops, run_task7from lib.models import ReplayModel8from lib.tasks import score9from lib.tools import V110from lib.trace import IdCounter1112RULES = Stops(repeat_limit=3, no_progress_limit=2)131415def replay(stops: Stops) -> dict:16 """Every task in file order, so the run IDs match the recording's. Maps task ID to (ended, steps, done)."""17 ids, out = IdCounter(), {}18 for task in load_tasks():19 run = run_task(ReplayModel("task-set", task.id), task, V1, max_steps=V1.step_cap, ids=ids, stops=stops)20 out[task.id] = (run.ended, run.steps, score(task, run))21 run.close()22 return out232425ENDS = ("finish", "ask_user", "step_cap", "repeated_call", "no_progress", "no_tool_call")26plain, ruled = replay(Stops()), replay(RULES)27capped = sorted(t for t, (ended, _, _) in plain.items() if ended == "step_cap")28ruled_ends = Counter(ended for ended, _, _ in ruled.values())29done = sorted(t for t, (_, _, ok) in plain.items() if ok)30print("task set step cap:", V1.step_cap)31print("runs:", len(plain))32print("step cap without the rules:", len(capped))33print("step cap without the rules, tasks:", ", ".join(capped))34print("step cap with the rules:", ruled_ends["step_cap"])35print("ended early with the rules:", len([t for t in capped if ruled[t][0] != "step_cap"]))36for ended in ENDS:37 print(f" {ended}: {ruled_ends[ended]}")38print("done without the rules:", len(done))39print("done without the rules, tasks:", ", ".join(done))40print("done with the rules:", sum(1 for _, _, ok in ruled.values() if ok))41print("done with the rules, by finish:", sum(1 for ended, _, ok in ruled.values() if ended == "finish" and ok))42print("finish but not done, with the rules:", sum(1 for ended, _, ok in ruled.values() if ended == "finish" and not ok))43print("done with the rules, by ask_user:", sum(1 for ended, _, ok in ruled.values() if ended == "ask_user" and ok))44wrong = [t for t, (ended, _, ok) in plain.items() if ended == "ask_user" and not ok]45print("ask_user but not done, without the rules:", len(wrong))46print("of those, cut early with the rules:", len([t for t in wrong if ruled[t][0] != "ask_user"]))47assert len(capped) == 7 and all(ruled[t][2] for t in done), "a done task must survive the rules"48assert sum(ruled_ends.values()) == len(plain), "every run has one ending"
python stop-rules-over-task-set.pytask set step cap: 16
runs: 20
step cap without the rules: 7
step cap without the rules, tasks: T-01, T-02, T-06, T-10, T-11, T-13, T-16
step cap with the rules: 1
ended early with the rules: 6
finish: 6
ask_user: 1
step_cap: 1
repeated_call: 1
no_progress: 11
no_tool_call: 0
done without the rules: 5
done without the rules, tasks: T-04, T-05, T-09, T-14, T-18
done with the rules: 5
done with the rules, by finish: 4
finish but not done, with the rules: 2
done with the rules, by ask_user: 1
ask_user but not done, without the rules: 6
of those, cut early with the rules: 6
The rules cut stuck runs short without losing a done task. Without them, the budget ends 7 of the 20 runs; with them, only 1 still reaches it.
With the rules on, the most common end was no_progress, at 11 runs, and finish came next, at 6 runs.
Each done task still passes with the rules on, and the example checks it. Of the 5 done runs, 4 end with finish and pass their check, and 1 ends with the ask_user on T-18, which is the fifth. The other 2 finish runs fail their check, so they are not done.
Without the rules, 6 runs asked a question and were not done. With the rules, 6 of those runs ended earlier, and none of them asked. The one ask_user left with the rules on is T-18's, which the scorer counts as done; chapter 10 shows that it counts only through a word in a path. A stop that ends a run early doesn't make the run right. It ends the run before the wrong question is asked, and that run is still not done. The rules cut the waste; they can't tell a right question from a wrong one.
You can check the output yourself. The six endings with the rules on add up to the number of runs, as the lines of the output show, and the done count is the same with the rules and without them. If the done count ever moves, a rule has cut a run that was going to pass, and the rule is the thing to change. If a run in your own work ends stuck, read it in this order: the ended value, then the last few steps, then the first step where a call began to repeat. Those reads take less time than reading the whole transcript.
Working rule: Count each run by the ended value it recorded, and report the counts with the rules on and off, next to the done count. A stop can only turn a run into one that is not done, so when you change a rule, check that the done count holds before you keep the change.
本章的決定
以下每一項都是本書選定的預設值,沒有一項是定律。實用原則會說明何時該更改其中一項;而更改應先在你自己的任務集上進行,讓你能先算出它對已完成任務造成的影響,再讓其他人依賴它。
- 停止規則是選用的,在迴圈中預設為關閉。沒有這些規則的執行,所發出的請求與第 1 章的迴圈相同。
- 重複呼叫是指連續三次相同的呼叫,以工具名稱與排序後的引數進行比較。
- 沒有進展是指連續兩次編輯,使測試框架的測試數量維持不變。第一次編輯是與任何編輯之前的數量相比較。
- 停止會以一個結果結束執行。沒有任何停止規則會詢問人,而交接要等到子代理與交接再處理。
- 步驟預算計算的是模型呼叫次數,包含格式錯誤的回覆。本章不把它當作時間或 token 的上限。
- 六種結束狀態的名稱維持固定。關卡會在第 6 章再增加一種。
- 指明其規則與計數的停止,是讀者可以檢查的;只指明預算的停止,則會讓讀者必須回頭走過每一個步驟。
本章,於本版次中
版次 2026.11.2範例於 2026年11 月 20 通過測試本章 4 個範例 個範例中有 4 個通過
譯自英文書,版次 2026.11.2。