為腳本式、重播式、本機與託管模型提供單一介面

閱讀約需 46 分鐘 分鐘,5 節 中共有 4 個範例。

版次 2026.11.2 範例於 2026年11 月 20 通過測試 64 個範例中有 64 個通過 變更內容

由 AI 翻譯。經人工以英文驗證。 我們如何製作書籍

譯自英文書,版次 2026.11.2。 閱讀原文

讀完本章,你可以用同一個迴圈,分別執行腳本式模型、重播的執行紀錄、本機模型,以及相容 OpenAI 的端點,而不必更動迴圈。你也能說明每個轉接器會保留回應中的哪些內容,又會從回應文字中讀出哪些內容。

第 1 章以腳本式模型,接著以已儲存的執行紀錄,為 unitcraft 執行了維護代理,迴圈兩次都不需修改。接下來的問題是:模型必須交出什麼,才能做到這一點。以 T-01 為例,該問題是 format_quantity 會捨去尾端的零,因此 3.5 kg 會印成 3.5 kg,而預設的兩位小數應得到 3.50 kg。每一個步驟都需要取回同樣的三樣東西:模型的文字、harness 能從該文字中讀出的工具呼叫,以及請求與回應所用的 token 數量。一份腳本式的字串清單可以提供這三項,磁碟上的檔案、載入記憶體的模型或 HTTP 端點也可以。轉接器就是讓上述每一種來源,在迴圈看來都一樣的程式碼,本章其餘部分會依迴圈遇到的順序介紹這四種轉接器。

此部分尚未翻譯,因此以英文呈現。

Ask every model the same question

The loop needs one thing from a model: given the messages so far and the tool schemas from the schema chapter, return what the model said. A Python Protocol names that contract. Every adapter in this book has the same method, and every adapter returns the same kind of value.

class Model(Protocol):
    def reply(self, messages: list[dict], tools: list[dict]) -> Reply: ...

The Reply is a small dataclass. It holds the model's raw text, the tool calls the adapter read from that text, any malformed blocks it could not read, the prompt and completion token counts, and the finish reason, which says whether the model stopped on its own or ran out of tokens. A token here is the unit that chapter 1 counts, from the pinned model's own tokenizer, so the counts belong to that model and mean nothing for another. The adapter never edits text. The loop acts on the calls, but the transcript keeps the text as the model wrote it, so a reader can see what the model actually said.

Keeping both the text and the calls matters because parsing can fail. A reply may hold a <tool_call> block whose JSON does not parse, or a block that is never closed. The adapter cannot quietly turn such a reply into a call, and it cannot throw the reply away, because the loop has to tell the model what went wrong. So the parse leaves the text as it is, records a short reason for each bad block beside it, and lets the loop decide what the model sees next. The local model section shows the parser at work.

A Protocol looks like an interface, but Python checks it by shape. Python's typing documentation says that a runtime_checkable protocol "will check only the presence of the required methods or attributes, not their type signatures or types." So isinstance can confirm that an object has a reply method, and nothing more: a class whose method is written reply(self, prompt) passes, and a class with no reply fails. The book's own Model is not decorated with runtime_checkable, and the same documentation says isinstance on such a protocol raises a TypeError. No example in this book calls isinstance on Model, so both claims are the documentation's and not run here.

Working rule: Test each adapter by calling reply() with the messages and tool schemas the loop sends, and check the fields of the Reply that come back. Use isinstance only to see that a method exists, never to show that its signature is right.

此部分尚未翻譯,因此以英文呈現。

Script the replies, or replay a stored run

Two adapters need no model at all. Chapter 1's scripted run used a scripted model, and its replayed run used a stored one. Both work for the same reason: the loop is the thing under test, and a fixed list of replies keeps the model out of it. This section says what the adapters add to that.

ScriptedModel takes strings or Reply objects. A string is parsed into a Reply for you, and its tool calls are read from the <tool_call> blocks in the text. A Reply object is passed through unchanged. That second form lets a test set the token counts or a malformed entry directly, instead of writing JSON that must fail in exactly the right way. If the loop asks for more replies than the script holds, the model raises an IndexError, so a short script fails loudly rather than returning nothing. The script also keeps a test readable. The reply that causes a behaviour sits in the test beside the assertion about it, and the reader does not have to open a recording to find it.

ReplayModel reads one run's rows from the recording's replies.jsonl, in step order. Each row holds the request's hash as it was sent, the reply's text and the token counts. When the loop asks for a step, ReplayModel hashes the request it was given, the sha256 of its messages and tool schemas in canonical JSON, and compares that hash with the row's. A difference raises ReplayMismatch, which names the step. The counts come from the stored row, so a replay never tokenizes again.

The rows come from a logger, not from a script. LoggedModel wraps a live model and appends each reply to a JSON-lines file before the loop sees it. A run that stops partway leaves its rows on disk, and the recording script removes them before it resumes.

Chapter 1's T-01 run goes through the loop twice in the first example: once on a ScriptedModel that holds the stored reply texts, and once on the ReplayModel for the same run. The scripted replies are read from the stored file, so both runs start from the same source.

same-loop-two-models.py Python 3.12.15 · pydantic 2.14.3 通過
"""Chapter-1-t01 runs through the loop twice: once with a ScriptedModel holding the stored reply texts, once with aReplayModel. The assert compares the two runs' step lines, and the lines printed below are the replayed run's."""import jsonfrom lib.case import load_tasksfrom lib.loop import run_taskfrom lib.models import RECORDED, ReplayModel, ScriptedModelfrom lib.tools import CH1from lib.trace import IdCountertask = load_tasks(only=["T-01"])[0]rows = [json.loads(line) for line in (RECORDED / "chapter-1-t01" / "replies.jsonl").read_text(encoding="utf-8").splitlines()]def run_once(model):    run = run_task(model, task, CH1, max_steps=CH1.step_cap, ids=IdCounter())  # counter-based IDs, never random (P-3)    try:        return list(run.trace.lines()), run.steps, run.ended    finally:        run.close()scripted = run_once(ScriptedModel([row["text"] for row in rows]))replayed = run_once(ReplayModel("chapter-1-t01", "T-01"))assert scripted == replayed, "the two adapters gave the loop different steps"lines, steps, ended = replayedfor line in lines:    print(line)print(f"steps: {steps}")print(f"ended: {ended}")
輸出python same-loop-two-models.py
step 1: read_file: 23 lines
step 2: run_tests: 8 failed, 22 passed
step 3: malformed: error: malformed call
step 4: malformed: error: malformed call
step 5: malformed: error: malformed call
step 6: malformed: error: malformed call
step 7: malformed: error: malformed call
step 8: malformed: error: malformed call
steps: 8
ended: step_cap

The loop made the same calls both times, and the example stops if the two runs differ in any step line. The run is the one that hit the step cap in chapter 1: it ended as step_cap after 8 steps. The loop cannot tell the two adapters apart, so whatever the loop does with a reply, it does the same for either source.

The second example checks the recording itself, against its manifest, and then replays it.

replay-byte-for-byte.py Python 3.12.15 · pydantic 2.14.3 通過
"""The stored chapter-1 run is the recording its manifest describes, and replaying it hands the loop each stored reply textexactly. The prompt and the check hash to the manifest's values, the first request hashes to the reply row stored with it,and the stored replies add up to the manifest's counts. The run is then replayed through the loop, which refuses any requestthat differs from its recorded hash, and each replayed text is compared with its stored text. No model file is loaded."""from contextlib import closingfrom lib.case import load_tasksfrom lib.loop import run_taskfrom lib.models import RECORDED_SAMPLING, ReplayModel, TextCapture, first_step, load_recording, request_hash, same_texts, stored_totalsfrom lib.prompts import prompt_sha256from lib.tasks import check_sha256from lib.tools import CH1from lib.trace import IdCountertask = load_tasks(only=["T-01"])[0]manifest, rows = load_recording("chapter-1-t01")messages, tools = first_step(task, CH1)assert manifest["prompt_sha256"] == prompt_sha256(CH1.system, CH1.descriptions()), "the prompt changed since recording"assert manifest["check_sha256"] == check_sha256(task), "the check changed since recording"assert rows[0]["request_sha256"] == request_hash(messages, tools), "the first request differs from the stored reply row"assert len(rows) == manifest["steps"], "the stored replies and the manifest disagree on the step count"prompt_tokens, completion_tokens = stored_totals(manifest, rows)assert manifest["sampling"] == RECORDED_SAMPLING, "the recording's sampling differs from the settings the adapters send"capture = TextCapture(ReplayModel("chapter-1-t01", "T-01"))with closing(run_task(capture, task, CH1, max_steps=CH1.step_cap, ids=IdCounter())) as run:    steps, ended = run.steps, run.endedcompared = same_texts(rows, capture.texts)assert steps == manifest["steps"] and ended == manifest["ended"], "the replay did not end as the manifest records"print(f"stored replies: {len(rows)}")print(f"prompt tokens: {prompt_tokens}")print(f"completion tokens: {completion_tokens}")for key in ("temperature", "top_k", "seed"):    print(f"{key}: {manifest['sampling'][key]}")print(f"reply texts compared exactly: {compared}")print(f"steps: {steps}")print(f"ended: {ended}")
輸出python replay-byte-for-byte.py
stored replies: 8
prompt tokens: 13009
completion tokens: 2128
temperature: 0
top_k: 1
seed: 4217
reply texts compared exactly: 8
steps: 8
ended: step_cap

Here the manifest is the reference, and the example makes four checks. The prompt's hash and the check's hash are compared with the manifest's values, where the check is the hidden test, H-01, that chapter 1 grades T-01 with. The first request's hash is compared with the one stored in its reply row in replies.jsonl, because the manifest holds no request hash. The stored replies must match the manifest's step count and token totals, and the sampling settings are printed from the manifest and compared with RECORDED_SAMPLING in the library, the values loopback_model sends. The stored run has 8 replies, 13009 prompt tokens and 2128 completion tokens. The example prints its sampling settings on their own lines, so a reader sees the temperature, the top k and the seed the recording used.

Then the run goes through the loop. ReplayModel refuses any step whose request hash differs from its row, and the example compares each replayed text with its stored text, exactly. It prints the count it compared, 8, and the run ends as the manifest says. The manifest holds no hash for each reply, so the stored texts are not checked against an independent value. The check shows that replay hands the loop each stored text unchanged. The prompt hash covers the system message and the tool descriptions together, so a changed tool description changes the recording's identity as surely as a changed line of code does.

Working rule: Replay a recorded run only with the prompt and the check it was recorded with. When either changes, its hash in the manifest no longer matches, the example fails, and the run has to be recorded again. Changing a stored reply to make the example pass hides the change that caused the failure.

本機模型,以及設定為何固定

LlamaModel 執行的是本機模型。它透過 llama-cpp-python 載入 GGUF 檔,這是 llama.cpp 讀取的單一檔案格式。轉接器只在建立模型時才匯入這個函式庫,因此各範例從不載入它。沒有任何範例會載入 GGUF 檔,本書也沒有針對 LlamaModel 的測試。錄製時使用了它,而範例讀取的是該次錄製所儲存的回覆。它的 reply() 會帶著訊息與工具來呼叫 create_chat_completion,並從回應中讀取文字與用量。

函式庫的預設值不是本書的設定。在固定的版本中,llama-cpp-python 的 create_chat_completion 預設溫度為 0.2,top k 為 40。若轉接器不更動這些值,取樣結果會與這個轉接器不同。LlamaModel 在每次呼叫時都傳入 temperature=0 與 top_k=1,因此取樣方式是寫在程式碼中,而不是沿用函式庫的預設。

傳回來的是文字。錄製時的伺服器以 python -m llama_cpp.server 的方式,在同一個模型檔上執行固定版本的 llama-cpp-python,並把第一步的工具呼叫放在訊息內容中傳回。端點錄製中儲存的本文含有該內容,而且完全沒有 tool_calls 欄位:

<tool_call>
{"name": "read_file", "arguments": {"path": "unitcraft/format.py"}}
</tool_call>

因此,函式庫的回覆是以文字攜帶呼叫,LlamaModel 並不是從結構化欄位讀取,而是解析這段文字。

解析只由一個函式負責,即 parse_tool_calls。它會找出每個 <tool_call> 區塊,將其當作含有 name 與 arguments 物件的 JSON 來讀取,並保留該呼叫。無法解析的區塊會成為一筆附有簡短原因的格式錯誤項目。迴圈會把該項目當作工具錯誤顯示給模型,因此錯誤的呼叫會被記錄為一個步驟,而不會被丟棄。第 1 章的 T-01 執行結果見哪裡會出錯,其中顯示了這一點:前兩個步驟之後,每個步驟都是 error: malformed call。

結束原因對迴圈很重要。stop 表示模型結束了它的回答,length 則表示回覆達到轉接器設定的 token 上限。length 的回覆可能在 <tool_call> 區塊中途停止,所以解析器也會把未關閉的區塊回報為格式錯誤,而不是當作完整的區塊來讀取。

計數是函式庫自己的數字。LlamaModel 會把 llama-cpp-python 回報的提示與生成 token 數複製到 Reply 中,錄製時也會儲存這些數字。它們描述的是固定檔案的分詞器,因此換成不同的檔案,同樣的訊息會得到不同的計數,某個模型的計數也無法與另一個模型的計數比較。

建構函式接收檔案路徑、情境大小、執行緒數、種子與 token 上限,除此之外不接收任何有關模型的內容。錄製程式從 recordings/local_model.py 讀取這些值,因此想載入不同檔案的讀者,要同時更改路徑與這些設定,然後重新錄製。取樣是一種選擇,而模型卡建議的是另一組。Qwen 模型卡寫道:「We suggest using Temperature=0.7, TopP=0.8, TopK=20, and MinP=0.」這些設定適合每次執行結果有所變化的回答。本書只錄製一次執行,之後再以儲存的回覆檢查後續的執行,所以採用溫度 0、top k 1 與固定種子 4217,如各份資訊清單所記錄的。這些設定讓重新錄製有最大機會得到相符的結果,但溫度 0 並不保證輸出會重現,因此本書的主張不超過其範例所呈現的內容。端點範例會檢查同一份錄製中的兩次請求是否得到相同文字。重播則完全不依賴取樣,因為它讀取的是儲存的文字。

實用原則: 以資訊清單所指明的取樣設定進行錄製,並把模型檔的雜湊值一併保存在資訊清單中。若你更改任何設定,回覆也會隨之改變,所以應重新錄製,而不是拿來與舊的執行結果比較。

以迴路位址測試的託管式端點

只要伺服器接受 OpenAI 的 chat-completions 請求格式,一個轉接器就能連上它,只需提供基礎 URL,以及主機要求時的金鑰。OpenAICompatibleModel 會把訊息、工具綱要與取樣設定以 POST 送到 /chat/completions。它使用標準函式庫的 urllib,不使用任何 SDK。沒有 SDK,就沒有東西遮住請求:request_body() 會回傳轉接器序列化並送出的本文,測試可以拿它與已錄製的內容比對。只有在轉接器被給定 top_k 的值時,才會送出該欄位。轉接器在建立時、尚未送出任何請求前,會拒絕兩件事:不是 http 或 https 的 URL,以及透過純 http 把金鑰送往迴路位址以外的任何主機。它也從不跟隨重新導向。重新導向會引發錯誤,因此金鑰不會被送到第二個 URL。

回應中的工具呼叫會出現在兩個位置之一。OpenAI 的參考文件把工具呼叫的 function.arguments 描述為模型以 JSON 產生的字串,並要求呼叫端「在呼叫你的函式之前,先在你的程式碼中驗證這些引數」。當回覆含有結構化的 tool_calls 欄位時,轉接器會解析該字串。欄位不存在時,它會退而使用 <tool_call> 文字,本地模型就是如此。parse_chat_body 會原樣保留伺服器送出的 message.content,把缺少的內容視為空文字,並把用量計數複製到 Reply 中,因此已儲存的本文每次都會解析成相同的 Reply。無論走哪一條路,引數都以字典的形式進入迴圈,而這個字典尚未經過檢查。這項檢查由綱要章中該工具的引數模型負責。

本書沒有任何範例會送出回傳結構化 tool_calls 欄位的請求,因此第一個分支只以程式碼形式存在,沒有任何範例執行它。本書不展示其他供應商在該欄位中回傳什麼,本章對此的說明也不超過 OpenAI 參考文件。

錄製時使用與本地模型相同的檔案與相同的取樣設定。它在迴路位址上執行固定版本 llama-cpp-python 自己的伺服器,並透過轉接器把 T-01 的第一步送出兩次。儲存的檔案包含每個依原樣送出的請求,以及每個原始回應本文。迴路位址上的 http.server 會依序提供這些本文,若任何請求的本文與錄製的不同,它就回應 409。這個替身會比對請求本文、每則訊息與每項設定。它不比對路徑、方法或標頭,因此端點路徑或 Authorization 標頭的變更仍會通過。若轉接器的變更改動了它送出的本文,請求會在讀取任何回覆之前失敗。儲存的本文保留了伺服器自己的 id 與 created 欄位。沒有任何東西會比對或印出它們。由於沒有載入任何模型,每次檢查本書的範例時,轉接器的 HTTP 與解析程式碼都會執行。

openai-compatible-loopback.py Python 3.12.15 · pydantic 2.14.3 通過
"""The OpenAI-compatible adapter, run live against a loopback stub that serves the bodies recorded from llama.cpp's ownserver. The line that matters is "raw text": the server put the call in the message content as text, with no tool_callsfield, so the adapter parses the text, and the tool's own model accepts the arguments. Needsrecordings/openai_compatible.py to have been run; until then this raises."""import jsonfrom lib.case import load_tasksfrom lib.models import first_step, loopback_model, serve_recordedfrom lib.tools import CH1task = load_tasks(only=["T-01"])[0]messages, tools = first_step(task, CH1)bodies: list[str] = []with serve_recorded("openai-compatible") as base_url:    model = loopback_model(base_url, on_body=bodies.append)    first = model.reply(messages, tools)    second = model.reply(messages, tools)assert first.text.lstrip().startswith("<tool_call>"), "the server should return the call as text"assert "tool_calls" not in json.loads(bodies[0])["choices"][0]["message"], "the recorded body has a structured tool_calls field"assert len(first.tool_calls) == 1 and first.tool_calls[0]["name"] == "read_file", "the first step is one read_file call"assert not first.malformed, "the call should parse"CH1.get("read_file").args.model_validate(first.tool_calls[0]["arguments"])  # raises if the arguments do not fit the toolassert first.text == second.text, "the two recorded responses differ"print(f"requests: {len(bodies)}")print("raw text:", json.dumps(first.text))print("structured tool_calls field: absent")print(f"tool calls: {len(first.tool_calls)}")print("call:", first.tool_calls[0]["name"], json.dumps(first.tool_calls[0]["arguments"], sort_keys=True))print(f"malformed: {len(first.malformed)}")print(f"prompt tokens: {first.prompt_tokens}")print(f"completion tokens: {first.completion_tokens}")print(f"finish reason: {first.finish_reason}")
輸出python openai-compatible-loopback.py
requests: 2
raw text: "<tool_call>\n{\"name\": \"read_file\", \"arguments\": {\"path\": \"unitcraft/format.py\"}}\n</tool_call>"
structured tool_calls field: absent
tool calls: 1
call: read_file {"path": "unitcraft/format.py"}
malformed: 0
prompt tokens: 501
completion tokens: 22
finish reason: stop

轉接器取回一個 read_file 呼叫,形式為文字。依工具的模型檢查其引數是呼叫端的工作,而這項檢查正是本節的重點,所以範例把它明確寫出。回覆帶有 1 個工具呼叫與 0 個格式錯誤的區塊,請求使用了 501 個提示詞 token。回覆以 stop 結束,這表示模型自行完成了回答,並未耗盡 token。

為錄製與範例建立轉接器的輔助函式 loopback_model,固定了伺服器啟動時使用的模型別名、每則回覆 1024 個 token 的上限、溫度 0 與 top_k 1。若呼叫端想要其他設定,就直接建立 OpenAICompatibleModel,此時它送出的請求本文會不同,替身會拒絕它。

代理伺服器的運作方式與任何 urllib 請求相同,唯獨迴路主機 127.0.0.1、::1 與 localhost 例外,它們會被直接連線。遠端 http URL 則經由環境所指定的代理伺服器。在拒絕範例中,有一個替身代表代理伺服器,並展示兩個面向:迴路請求從未到達它,而遠端請求帶著完整 URL 抵達。

adapter-refuses-unsafe-endpoints.py Python 3.12.15 通過
"""The OpenAI-compatible adapter refuses a redirect, a key over plain http to a remote host, and a non-http scheme. It sendsa remote URL through the environment's proxy and a loopback URL around it. Every request goes to a stub on 127.0.0.1, whichstands in for both the endpoint and the proxy, so no network is used. Each refusal prints one labelled line."""from lib.models import OpenAICompatibleModel, proxy_env, refusal, stub_endpointREMOTE = "http://remote-host.invalid/v1"MESSAGES = [{"role": "user", "content": "ping"}]with stub_endpoint(200) as proxy, stub_endpoint(302) as target, proxy_env(proxy):    loopback = f"{target.url}/v1"    model = OpenAICompatibleModel(loopback, "model", max_tokens=8, api_key="DUMMY-KEY")    reason = refusal(lambda: model.reply(MESSAGES, []))    assert target.seen == ["/v1/chat/completions"], "a redirect was followed"    print("redirect 302 on loopback:", reason)    target.code = 307    reason = refusal(lambda: model.reply(MESSAGES, []))    print("redirect 307 on loopback:", reason)    assert target.seen == ["/v1/chat/completions"] * 2, "a redirect was followed"    print(f"requests to the redirect target: {target.seen.count('/collect')}")    print(f"loopback requests through the proxy: {len(proxy.seen)}")    remote = OpenAICompatibleModel(REMOTE, "model", max_tokens=8)    reply = remote.reply(MESSAGES, [])    assert reply.text == "ok" and proxy.seen == ["http://remote-host.invalid/v1/chat/completions"]    print("remote request through the proxy:", proxy.seen[0])    print("remote reply text:", reply.text)    reason = refusal(lambda: OpenAICompatibleModel(REMOTE, "model", max_tokens=8, api_key="DUMMY-KEY"))    print("key over plain http to a remote host:", reason)    assert len(proxy.seen) == 1, "a request was made for the refused key"    OpenAICompatibleModel("https://remote-host.invalid/v1", "model", max_tokens=8, api_key="DUMMY-KEY")    print("key over https to a remote host: accepted, no request made")    OpenAICompatibleModel(loopback, "model", max_tokens=8, api_key="DUMMY-KEY")    print("key over http to loopback: accepted")    reason = refusal(lambda: OpenAICompatibleModel("file:///v1", "model", max_tokens=8))    print("file URL:", reason)
輸出python adapter-refuses-unsafe-endpoints.py
redirect 302 on loopback: the endpoint answered 302 with a redirect, and the adapter does not follow redirects
redirect 307 on loopback: the endpoint answered 307 with a redirect, and the adapter does not follow redirects
requests to the redirect target: 0
loopback requests through the proxy: 0
remote request through the proxy: http://remote-host.invalid/v1/chat/completions
remote reply text: ok
key over plain http to a remote host: the adapter sends an API key over https, or over http to a loopback host only
key over https to a remote host: accepted, no request made
key over http to loopback: accepted
file URL: the scheme 'file' is not http or https

範例也展示了每一種拒絕:重新導向、透過純 http 把金鑰送往遠端主機,以及檔案位址。重新導向的目標不會收到任何請求,另外兩種則在建立任何連線之前就被拒絕。每次拒絕都會印出造成它的規則。

本章不呼叫任何託管供應商。若你把轉接器指向其中一個,請使用 https,金鑰本來就需要它,並自備你的帳號與金鑰。本書兩者都不提供,其範例也從不呼叫模型或遠端主機。

實用原則: 把 function.arguments 當作 JSON 解析,依工具的引數模型檢查結果,並把失敗當作工具結果送回給模型。強制執行嚴格綱要的端點會讓這類失敗較少發生,但檢查仍然要執行。

一個迴圈,多種模型

迴圈在不同的轉接器之間不會改變。它呼叫 reply(),從 Reply 讀取工具呼叫,執行第一個呼叫,並把結果寫入下一個步驟。各轉接器的差別在於回覆從何而來,以及如何處理回覆的文字;迴圈只看得到各轉接器回傳的 Reply。

轉接器 回覆的來源 工具呼叫的讀取來源 示範範例
ScriptedModel 寫在程式碼中的字串 文字 same-loop-two-models
ReplayModel 某次執行所儲存的回覆 文字 same-loop-two-models、replay-byte-for-byte
LlamaModel 本機的 GGUF 檔案,透過 llama-cpp-python 文字 無;請見本機模型
OpenAICompatibleModel HTTP 端點,本書中使用 loopback 若有結構化欄位則讀取該欄位,否則讀取文字 openai-compatible-loopback、adapter-refuses-unsafe-endpoints

每一列說明一個轉接器的作用,最後一欄列出執行該轉接器的範例。有兩個範例透過迴圈驅動 ScriptedModel 與 ReplayModel,另有兩個範例直接呼叫 OpenAICompatibleModel 的 reply()。

在正式環境中,轉接器會在三個地方出問題,讀者應當分辨自己看到的是哪一種。第一是傳輸層。urllib 遇到錯誤狀態碼會擲出例外,而轉接器不會捕捉它。轉接器刻意拒絕重新導向,因此重新導向同樣會擲出例外。無論哪種情況,該步驟都不會產生 Reply,執行會因該錯誤而停止。第二是回覆的形狀:回覆可能可以成功解析,卻指名了迴圈中不存在的工具,此時第 3 章的名稱檢查會把它當作工具錯誤而拒絕。第三是已儲存的執行紀錄。若提示詞改變,replay-byte-for-byte 會在第一個斷言處失敗,且發生在任何步驟執行之前。若較新版本的迴圈在某個步驟提出了不同的問題,ReplayModel 會在該步驟擲出 ReplayMismatch,訊息中會指出是哪一個步驟。來自即時模型的格式錯誤工具呼叫則又不同:那是可預期的事件,迴圈會計數,並把它呈現給模型。

讓迴圈保持穩定的是一個剖析器與一個 Reply。每個轉接器,無論是即時的或儲存的,都交給迴圈相同的欄位,文字保持模型寫出的原樣,而與每個回覆一併儲存的請求雜湊,會讓被改動的問題在改動之處的那個步驟失敗。端點轉接器是唯一接觸網路的轉接器,在它指向真實主機之前,應重新檢查它的規則:不重新導向、不以純 http 傳送金鑰,而且代理只用於遠端 URL。

新增第五個轉接器需要三個步驟。撰寫一個具有 reply() 方法的類別,該方法接收迴圈的訊息與工具。以其他轉接器所用的同一個剖析器建構它的 Reply:文字使用 make_reply,回應本文使用 parse_chat_body,使每個轉接器都以相同方式讀取工具呼叫。接著為它加上協定一節所描述的測試。迴圈不會改變,其他任何轉接器也不會改變。

實用原則: 轉接器保留原始文字與計數,在單一處讀取工具呼叫,並回傳 Reply。它絕不把 SDK 物件交給迴圈。為每個轉接器撰寫測試,以迴圈的引數呼叫 reply(),並把提示詞或請求的改變視為重新錄製的理由。

本章,於本版次中

版次 2026.11.2範例於 2026年11 月 20 通過測試本章 4 個範例 個範例中有 4 個通過

譯自英文書,版次 2026.11.2。

引用本節

格式

各種格式皆引用英文原書,本譯本即依循該原書。

每種格式都會標示本版。其連結會開啟目前版次,節標題會說明該節內文最後一次變動的時間。存取日期與擷取日期即為你開啟此頁的當日。

本頁內容

第 4 章,共 12 章: 為腳本式、重播式、本機與託管模型提供單一介面

  1. Ask every model the same question
  2. Script the replies, or replay a stored run
  3. 本機模型,以及設定為何固定
  4. 以迴路位址測試的託管式端點
  5. 一個迴圈,多種模型

返回頂端所有章節