脚本化、回放、本地与托管模型的统一接口

阅读约需 46分钟 分钟,5节中含4个示例。

版次 2026.11.2 示例于 2026年11月20日 通过 64个示例中有64个通过 变化内容

AI 翻译。由人工以英文核验。 我们如何制作图书

译自英文版图书,版次 2026.11.2。 阅读原文

读完本章,你将能在脚本化模型、回放的运行记录、本地模型和兼容 OpenAI 的端点上运行同一个循环,而无需改动循环本身。你还能说明每个适配器从一次回复中保留了什么,又从回复文本中读出了什么。

第 1 章在脚本化模型上运行了 unitcraft 的维护智能体,随后又在一份已存储的运行记录上运行,循环两次都无需改动。接下来的问题是:模型要交出什么,才能做到这一点。以 T-01 为例,这个问题是 format_quantity 会丢掉末尾的零,因此 3.5 kg 被输出为 3.5 kg,而默认保留两位小数时应为 3.50 kg。每一步都需要拿回同样的三样东西:模型的文本、测试框架能从该文本中读出的工具调用,以及请求和回复所用的词元数。一个脚本化的字符串列表可以提供这三样东西,磁盘上的文件、加载到内存中的模型或 HTTP 端点也可以。适配器就是让这些来源在循环看来都一样的代码。本章其余部分按循环遇到它们的顺序,依次讲解这四种适配器。

这部分尚未翻译,因此以英文显示。

Ask every model the same question

The loop needs one thing from a model: given the messages so far and the tool schemas from the schema chapter, return what the model said. A Python Protocol names that contract. Every adapter in this book has the same method, and every adapter returns the same kind of value.

class Model(Protocol):
    def reply(self, messages: list[dict], tools: list[dict]) -> Reply: ...

The Reply is a small dataclass. It holds the model's raw text, the tool calls the adapter read from that text, any malformed blocks it could not read, the prompt and completion token counts, and the finish reason, which says whether the model stopped on its own or ran out of tokens. A token here is the unit that chapter 1 counts, from the pinned model's own tokenizer, so the counts belong to that model and mean nothing for another. The adapter never edits text. The loop acts on the calls, but the transcript keeps the text as the model wrote it, so a reader can see what the model actually said.

Keeping both the text and the calls matters because parsing can fail. A reply may hold a <tool_call> block whose JSON does not parse, or a block that is never closed. The adapter cannot quietly turn such a reply into a call, and it cannot throw the reply away, because the loop has to tell the model what went wrong. So the parse leaves the text as it is, records a short reason for each bad block beside it, and lets the loop decide what the model sees next. The local model section shows the parser at work.

A Protocol looks like an interface, but Python checks it by shape. Python's typing documentation says that a runtime_checkable protocol "will check only the presence of the required methods or attributes, not their type signatures or types." So isinstance can confirm that an object has a reply method, and nothing more: a class whose method is written reply(self, prompt) passes, and a class with no reply fails. The book's own Model is not decorated with runtime_checkable, and the same documentation says isinstance on such a protocol raises a TypeError. No example in this book calls isinstance on Model, so both claims are the documentation's and not run here.

Working rule: Test each adapter by calling reply() with the messages and tool schemas the loop sends, and check the fields of the Reply that come back. Use isinstance only to see that a method exists, never to show that its signature is right.

这部分尚未翻译,因此以英文显示。

Script the replies, or replay a stored run

Two adapters need no model at all. Chapter 1's scripted run used a scripted model, and its replayed run used a stored one. Both work for the same reason: the loop is the thing under test, and a fixed list of replies keeps the model out of it. This section says what the adapters add to that.

ScriptedModel takes strings or Reply objects. A string is parsed into a Reply for you, and its tool calls are read from the <tool_call> blocks in the text. A Reply object is passed through unchanged. That second form lets a test set the token counts or a malformed entry directly, instead of writing JSON that must fail in exactly the right way. If the loop asks for more replies than the script holds, the model raises an IndexError, so a short script fails loudly rather than returning nothing. The script also keeps a test readable. The reply that causes a behaviour sits in the test beside the assertion about it, and the reader does not have to open a recording to find it.

ReplayModel reads one run's rows from the recording's replies.jsonl, in step order. Each row holds the request's hash as it was sent, the reply's text and the token counts. When the loop asks for a step, ReplayModel hashes the request it was given, the sha256 of its messages and tool schemas in canonical JSON, and compares that hash with the row's. A difference raises ReplayMismatch, which names the step. The counts come from the stored row, so a replay never tokenizes again.

The rows come from a logger, not from a script. LoggedModel wraps a live model and appends each reply to a JSON-lines file before the loop sees it. A run that stops partway leaves its rows on disk, and the recording script removes them before it resumes.

Chapter 1's T-01 run goes through the loop twice in the first example: once on a ScriptedModel that holds the stored reply texts, and once on the ReplayModel for the same run. The scripted replies are read from the stored file, so both runs start from the same source.

same-loop-two-models.py Python 3.12.15 · pydantic 2.14.3 通过
"""Chapter-1-t01 runs through the loop twice: once with a ScriptedModel holding the stored reply texts, once with aReplayModel. The assert compares the two runs' step lines, and the lines printed below are the replayed run's."""import jsonfrom lib.case import load_tasksfrom lib.loop import run_taskfrom lib.models import RECORDED, ReplayModel, ScriptedModelfrom lib.tools import CH1from lib.trace import IdCountertask = load_tasks(only=["T-01"])[0]rows = [json.loads(line) for line in (RECORDED / "chapter-1-t01" / "replies.jsonl").read_text(encoding="utf-8").splitlines()]def run_once(model):    run = run_task(model, task, CH1, max_steps=CH1.step_cap, ids=IdCounter())  # counter-based IDs, never random (P-3)    try:        return list(run.trace.lines()), run.steps, run.ended    finally:        run.close()scripted = run_once(ScriptedModel([row["text"] for row in rows]))replayed = run_once(ReplayModel("chapter-1-t01", "T-01"))assert scripted == replayed, "the two adapters gave the loop different steps"lines, steps, ended = replayedfor line in lines:    print(line)print(f"steps: {steps}")print(f"ended: {ended}")
输出python same-loop-two-models.py
step 1: read_file: 23 lines
step 2: run_tests: 8 failed, 22 passed
step 3: malformed: error: malformed call
step 4: malformed: error: malformed call
step 5: malformed: error: malformed call
step 6: malformed: error: malformed call
step 7: malformed: error: malformed call
step 8: malformed: error: malformed call
steps: 8
ended: step_cap

The loop made the same calls both times, and the example stops if the two runs differ in any step line. The run is the one that hit the step cap in chapter 1: it ended as step_cap after 8 steps. The loop cannot tell the two adapters apart, so whatever the loop does with a reply, it does the same for either source.

The second example checks the recording itself, against its manifest, and then replays it.

replay-byte-for-byte.py Python 3.12.15 · pydantic 2.14.3 通过
"""The stored chapter-1 run is the recording its manifest describes, and replaying it hands the loop each stored reply textexactly. The prompt and the check hash to the manifest's values, the first request hashes to the reply row stored with it,and the stored replies add up to the manifest's counts. The run is then replayed through the loop, which refuses any requestthat differs from its recorded hash, and each replayed text is compared with its stored text. No model file is loaded."""from contextlib import closingfrom lib.case import load_tasksfrom lib.loop import run_taskfrom lib.models import RECORDED_SAMPLING, ReplayModel, TextCapture, first_step, load_recording, request_hash, same_texts, stored_totalsfrom lib.prompts import prompt_sha256from lib.tasks import check_sha256from lib.tools import CH1from lib.trace import IdCountertask = load_tasks(only=["T-01"])[0]manifest, rows = load_recording("chapter-1-t01")messages, tools = first_step(task, CH1)assert manifest["prompt_sha256"] == prompt_sha256(CH1.system, CH1.descriptions()), "the prompt changed since recording"assert manifest["check_sha256"] == check_sha256(task), "the check changed since recording"assert rows[0]["request_sha256"] == request_hash(messages, tools), "the first request differs from the stored reply row"assert len(rows) == manifest["steps"], "the stored replies and the manifest disagree on the step count"prompt_tokens, completion_tokens = stored_totals(manifest, rows)assert manifest["sampling"] == RECORDED_SAMPLING, "the recording's sampling differs from the settings the adapters send"capture = TextCapture(ReplayModel("chapter-1-t01", "T-01"))with closing(run_task(capture, task, CH1, max_steps=CH1.step_cap, ids=IdCounter())) as run:    steps, ended = run.steps, run.endedcompared = same_texts(rows, capture.texts)assert steps == manifest["steps"] and ended == manifest["ended"], "the replay did not end as the manifest records"print(f"stored replies: {len(rows)}")print(f"prompt tokens: {prompt_tokens}")print(f"completion tokens: {completion_tokens}")for key in ("temperature", "top_k", "seed"):    print(f"{key}: {manifest['sampling'][key]}")print(f"reply texts compared exactly: {compared}")print(f"steps: {steps}")print(f"ended: {ended}")
输出python replay-byte-for-byte.py
stored replies: 8
prompt tokens: 13009
completion tokens: 2128
temperature: 0
top_k: 1
seed: 4217
reply texts compared exactly: 8
steps: 8
ended: step_cap

Here the manifest is the reference, and the example makes four checks. The prompt's hash and the check's hash are compared with the manifest's values, where the check is the hidden test, H-01, that chapter 1 grades T-01 with. The first request's hash is compared with the one stored in its reply row in replies.jsonl, because the manifest holds no request hash. The stored replies must match the manifest's step count and token totals, and the sampling settings are printed from the manifest and compared with RECORDED_SAMPLING in the library, the values loopback_model sends. The stored run has 8 replies, 13009 prompt tokens and 2128 completion tokens. The example prints its sampling settings on their own lines, so a reader sees the temperature, the top k and the seed the recording used.

Then the run goes through the loop. ReplayModel refuses any step whose request hash differs from its row, and the example compares each replayed text with its stored text, exactly. It prints the count it compared, 8, and the run ends as the manifest says. The manifest holds no hash for each reply, so the stored texts are not checked against an independent value. The check shows that replay hands the loop each stored text unchanged. The prompt hash covers the system message and the tool descriptions together, so a changed tool description changes the recording's identity as surely as a changed line of code does.

Working rule: Replay a recorded run only with the prompt and the check it was recorded with. When either changes, its hash in the manifest no longer matches, the example fails, and the run has to be recorded again. Changing a stored reply to make the example pass hides the change that caused the failure.

本地模型,以及它的设置为何固定

LlamaModel 运行一个本地模型。它通过 llama-cpp-python 加载 GGUF 文件,这是 llama.cpp 读取的单文件格式。适配器只在构建模型时才导入这个库,因此各示例从不加载它。没有任何示例加载 GGUF 文件,本书也没有针对 LlamaModel 的测试。录制时使用了它,而各示例读取的是那次录制所存储的回复。它的 reply() 会用消息和工具调用 create_chat_completion,并从返回结果中读取文本和用量。

该库的默认值并非本书所用的值。在固定的发布版本中,llama-cpp-python 的 create_chat_completion 默认温度为 0.2,top k 为 40。若适配器不改动这些默认值,其采样方式就会与本书不同。LlamaModel 在每次调用时都传入 temperature=0 和 top_k=1,因此采样参数写在代码中,而不是从库中继承。

返回的内容是文本。录制时的服务器以 python -m llama_cpp.server 的方式,在同一个模型文件上运行固定版本的 llama-cpp-python,并把第一步的工具调用放在消息内容之中返回。端点录制中存储的响应体包含这段内容,却完全没有 tool_calls 字段:

<tool_call>
{"name": "read_file", "arguments": {"path": "unitcraft/format.py"}}
</tool_call>

可见,库的回复把调用当作文本携带,LlamaModel 并不从中读取结构化字段,而是解析这段文本。

解析由一个函数完成,即 parse_tool_calls。它找出每个 <tool_call> 块,将其作为 JSON 读取,其中包含 name 和 arguments 对象,并保留该调用。无法解析的块会变成一个带有简短原因的格式错误条目。循环会把该条目作为工具错误展示给模型,因此一次错误的调用会被记录为一个步骤,而不是被丢弃。第 1 章中 T-01 的运行见出错之处,它说明了这一点:在最初两步之后,每一步都是 error: malformed call。

结束原因对循环很重要。stop 表示模型结束了它的回答,length 表示回复达到了适配器设定的词元上限。length 的回复可能在 <tool_call> 块的中间停止,因此解析器也会把未闭合的块报告为格式错误,而不是将其当作完整的块来读取。

计数来自该库本身。LlamaModel 把 llama-cpp-python 报告的提示词和补全词元数复制到 Reply 中,录制时会存储这些数值。它们描述的是固定文件所用的分词器,因此对于相同的消息,换一个文件会得到不同的计数,一个模型的计数也不能与另一个模型的计数相比较。

构造函数接受文件路径、上下文大小、线程数、种子和词元上限,关于模型的其他内容一概不取。录制从 recordings/local_model.py 读取这些值,因此想加载其他文件的读者,需要同时修改路径和这些设置,并重新录制。采样是一种选择,而模型卡推荐的是另一组。Qwen 模型卡写道:“We suggest using Temperature=0.7, TopP=0.8, TopK=20, and MinP=0.”这些设置适合每次运行结果各不相同的回答。本书只录制一次运行,之后用存储的回复检查后续的运行,所以使用温度 0、top k 1 和固定种子 4217,这些都记录在清单中。这些设置让重新录制最有可能得到一致的结果,但温度 0 并不保证输出会重复,所以本书所声称的不超出各示例所展示的内容。端点示例检查了同一次录制中的两次请求是否给出了相同的文本。回放则完全不依赖采样,因为它读取的是存储的文本。

工作准则: 按清单中指明的采样方式录制,并把模型文件的哈希值一并保存在清单中。如果你更改了某项设置,回复也会随之改变,因此应重新录制,而不是拿它与旧的运行相比较。

托管式端点,在环回地址上测试

接受 OpenAI 聊天补全请求格式的服务器,只需一个适配器就能访问,只要给出基础 URL,并在主机要求时给出密钥。OpenAICompatibleModel 把消息、工具模式和采样设置通过 POST 发送到 /chat/completions。它使用标准库的 urllib,不使用任何 SDK。没有 SDK,请求就没有任何隐藏之处:request_body() 返回适配器序列化并发送的请求体,测试可以把它与记录下来的请求体比较。只有在适配器被给定 top_k 的值时,才会发送该字段。适配器在构建时、发出任何请求之前,会拒绝两种情况:不是 http 或 https 的 URL,以及通过明文 http 向环回以外的任何主机发送密钥。它也从不跟随重定向。重定向会引发错误,因此密钥绝不会被发送到第二个 URL。

响应中的工具调用出现在两个位置之一。OpenAI 的参考文档把工具调用的 function.arguments 描述为模型生成的 JSON 字符串,并要求调用方“在调用函数之前,先在你的代码中验证这些参数”。当回复含有结构化的 tool_calls 字段时,适配器会解析该字符串。当该字段缺失时,它会退而解析 <tool_call> 文本,本地模型就是这样处理的。parse_chat_body 按服务器发送的原样保留 message.content,把缺失的内容读作空文本,并把用量计数复制到 Reply 中,因此存储的响应体每次都会解析为同一个 Reply。无论哪种方式,参数都会以字典的形式进入循环,而这个字典尚未经过检查。该检查由模式一章中工具的参数模型完成。

本书中没有任何示例会发送返回结构化 tool_calls 字段的请求,因此第一个分支只以代码形式存在,没有被任何示例运行。本书没有展示其他提供商在该字段中返回什么,本章对此的说明也不超出 OpenAI 参考文档。

录制使用了与本地模型相同的文件和相同的采样设置。它在环回地址上运行固定版本的 llama-cpp-python 自带的服务器,并通过适配器把 T-01 的第一步发送了两次。存储的文件保存了每个原样发出的请求和每个原始响应体。环回的 http.server 按顺序提供这些响应体,对于请求体与记录不同的任何请求,它都回答 409。该桩服务比较请求体、每条消息和每项设置。它不比较路径、方法或请求头,所以端点路径或 Authorization 头的改动仍会通过。如果适配器的某项改动改变了它发送的请求体,请求会在读取任何回复之前失败。存储的响应体保留了服务器自己的 id 和 created 字段。没有任何东西会比较或打印它们。由于没有加载模型,每次检查本书的示例时,适配器的 HTTP 和解析代码都会运行。

openai-compatible-loopback.py Python 3.12.15 · pydantic 2.14.3 通过
"""The OpenAI-compatible adapter, run live against a loopback stub that serves the bodies recorded from llama.cpp's ownserver. The line that matters is "raw text": the server put the call in the message content as text, with no tool_callsfield, so the adapter parses the text, and the tool's own model accepts the arguments. Needsrecordings/openai_compatible.py to have been run; until then this raises."""import jsonfrom lib.case import load_tasksfrom lib.models import first_step, loopback_model, serve_recordedfrom lib.tools import CH1task = load_tasks(only=["T-01"])[0]messages, tools = first_step(task, CH1)bodies: list[str] = []with serve_recorded("openai-compatible") as base_url:    model = loopback_model(base_url, on_body=bodies.append)    first = model.reply(messages, tools)    second = model.reply(messages, tools)assert first.text.lstrip().startswith("<tool_call>"), "the server should return the call as text"assert "tool_calls" not in json.loads(bodies[0])["choices"][0]["message"], "the recorded body has a structured tool_calls field"assert len(first.tool_calls) == 1 and first.tool_calls[0]["name"] == "read_file", "the first step is one read_file call"assert not first.malformed, "the call should parse"CH1.get("read_file").args.model_validate(first.tool_calls[0]["arguments"])  # raises if the arguments do not fit the toolassert first.text == second.text, "the two recorded responses differ"print(f"requests: {len(bodies)}")print("raw text:", json.dumps(first.text))print("structured tool_calls field: absent")print(f"tool calls: {len(first.tool_calls)}")print("call:", first.tool_calls[0]["name"], json.dumps(first.tool_calls[0]["arguments"], sort_keys=True))print(f"malformed: {len(first.malformed)}")print(f"prompt tokens: {first.prompt_tokens}")print(f"completion tokens: {first.completion_tokens}")print(f"finish reason: {first.finish_reason}")
输出python openai-compatible-loopback.py
requests: 2
raw text: "<tool_call>\n{\"name\": \"read_file\", \"arguments\": {\"path\": \"unitcraft/format.py\"}}\n</tool_call>"
structured tool_calls field: absent
tool calls: 1
call: read_file {"path": "unitcraft/format.py"}
malformed: 0
prompt tokens: 501
completion tokens: 22
finish reason: stop

适配器取回一个 read_file 调用,形式为文本。对照工具的模型检查其参数由调用方负责,而这项检查正是本节的全部要点,所以示例把它明确写了出来。该回复带有 1 个工具调用和 0 个格式错误的块,请求使用了 501 个提示词词元。回复以 stop 结束,这表示模型自行完成了回答,而不是耗尽了词元。

为录制和示例构建适配器的辅助函数 loopback_model,固定了服务器启动时使用的模型别名、每条回复 1024 个词元的上限、温度 0 和 top_k 1。若调用方想要其他设置,可以直接构建 OpenAICompatibleModel,此时它发送的请求体会不同,桩服务会拒绝它。

代理的工作方式与任何 urllib 请求相同,只有环回主机 127.0.0.1、::1 和 localhost 例外,它们被直接访问。远程的 http URL 则经过环境所指定的代理。在拒绝示例中,一个桩服务代替代理,它展示了两个方面:环回请求从不到达它,而远程请求带着完整的 URL 到达。

adapter-refuses-unsafe-endpoints.py Python 3.12.15 通过
"""The OpenAI-compatible adapter refuses a redirect, a key over plain http to a remote host, and a non-http scheme. It sendsa remote URL through the environment's proxy and a loopback URL around it. Every request goes to a stub on 127.0.0.1, whichstands in for both the endpoint and the proxy, so no network is used. Each refusal prints one labelled line."""from lib.models import OpenAICompatibleModel, proxy_env, refusal, stub_endpointREMOTE = "http://remote-host.invalid/v1"MESSAGES = [{"role": "user", "content": "ping"}]with stub_endpoint(200) as proxy, stub_endpoint(302) as target, proxy_env(proxy):    loopback = f"{target.url}/v1"    model = OpenAICompatibleModel(loopback, "model", max_tokens=8, api_key="DUMMY-KEY")    reason = refusal(lambda: model.reply(MESSAGES, []))    assert target.seen == ["/v1/chat/completions"], "a redirect was followed"    print("redirect 302 on loopback:", reason)    target.code = 307    reason = refusal(lambda: model.reply(MESSAGES, []))    print("redirect 307 on loopback:", reason)    assert target.seen == ["/v1/chat/completions"] * 2, "a redirect was followed"    print(f"requests to the redirect target: {target.seen.count('/collect')}")    print(f"loopback requests through the proxy: {len(proxy.seen)}")    remote = OpenAICompatibleModel(REMOTE, "model", max_tokens=8)    reply = remote.reply(MESSAGES, [])    assert reply.text == "ok" and proxy.seen == ["http://remote-host.invalid/v1/chat/completions"]    print("remote request through the proxy:", proxy.seen[0])    print("remote reply text:", reply.text)    reason = refusal(lambda: OpenAICompatibleModel(REMOTE, "model", max_tokens=8, api_key="DUMMY-KEY"))    print("key over plain http to a remote host:", reason)    assert len(proxy.seen) == 1, "a request was made for the refused key"    OpenAICompatibleModel("https://remote-host.invalid/v1", "model", max_tokens=8, api_key="DUMMY-KEY")    print("key over https to a remote host: accepted, no request made")    OpenAICompatibleModel(loopback, "model", max_tokens=8, api_key="DUMMY-KEY")    print("key over http to loopback: accepted")    reason = refusal(lambda: OpenAICompatibleModel("file:///v1", "model", max_tokens=8))    print("file URL:", reason)
输出python adapter-refuses-unsafe-endpoints.py
redirect 302 on loopback: the endpoint answered 302 with a redirect, and the adapter does not follow redirects
redirect 307 on loopback: the endpoint answered 307 with a redirect, and the adapter does not follow redirects
requests to the redirect target: 0
loopback requests through the proxy: 0
remote request through the proxy: http://remote-host.invalid/v1/chat/completions
remote reply text: ok
key over plain http to a remote host: the adapter sends an API key over https, or over http to a loopback host only
key over https to a remote host: accepted, no request made
key over http to loopback: accepted
file URL: the scheme 'file' is not http or https

该示例还展示了每一种拒绝:重定向、通过明文 http 向远程主机发送密钥,以及文件地址。重定向的目标不会收到任何请求,另外两种在建立任何连接之前就被拒绝。每次拒绝都会打印导致它的规则。

本章不调用任何托管提供商。如果你把适配器指向某个提供商,请使用 https,密钥本来也需要它,并自备账号和密钥。本书两者都不提供,其示例也从不调用模型或远程主机。

工作准则: 把 function.arguments 解析为 JSON,对照工具的参数模型检查结果,并把失败作为工具结果送回给模型。强制执行严格模式的端点会让这类失败更少出现,但检查仍然要运行。

一个循环,多种模型

不同适配器之间,循环本身不变。它调用 reply(),从 Reply 中读取调用,运行第一个调用,并把结果写入下一步。各适配器的区别在于回复从哪里来,以及如何处理回复的文本;循环看到的,只是每个适配器返回的 Reply。

适配器 回复的来源 调用的读取位置 演示示例
ScriptedModel 写在代码中的字符串 文本 same-loop-two-models
ReplayModel 某次运行中存储的回复 文本 same-loop-two-models、replay-byte-for-byte
LlamaModel 本地 GGUF 文件,通过 llama-cpp-python 加载 文本 无;参见本地模型
OpenAICompatibleModel HTTP 端点,本书中使用回环地址 若有结构化字段则读取该字段,否则读取文本 openai-compatible-loopback、adapter-refuses-unsafe-endpoints

每一行说明一个适配器的作用,最后一列给出运行它的示例。有两个示例让 ScriptedModel 和 ReplayModel 通过循环运行,另有两个示例直接调用 OpenAICompatibleModel 的 reply()。

在生产环境中,适配器会在三个地方出问题,读者应当清楚自己面对的是哪一个。第一个是传输层。urllib 在遇到错误状态码时会抛出异常,适配器不会捕获它。适配器有意拒绝重定向,因此重定向同样会抛出异常。无论哪种情况,该步骤都不会产生 Reply,运行随错误而停止。第二个是回复的形状:回复可以解析成功,却指向循环中并不存在的工具,此时第 3 章中的名称检查会将其作为工具错误拒绝。第三个是已存储的运行。如果提示词发生变化,replay-byte-for-byte 会在第一个断言处失败,此时任何一步都尚未运行。如果循环的后续版本在某一步提出了不同的问题,ReplayModel 会在该步抛出 ReplayMismatch,消息中会指明是哪一步。来自在线模型的格式错误的工具调用则又不同:它是可预期的事件,循环会对其计数,并把它展示给模型。

使循环保持稳定的,是一个解析器和一个 Reply。每个适配器,无论在线还是存储的,都向循环交付相同的字段,文本保持模型写出时的原样,并且随每条回复存储的请求哈希值,会让被改动的问题在改动发生的那一步失败。端点适配器是唯一接触网络的适配器,在把它指向真实主机之前,应当重新检查它的规则:不重定向,不通过明文 http 发送密钥,仅对远程 URL 使用代理。

添加第五个适配器需要三步。编写一个带有 reply() 方法的类,该方法接收循环的消息和工具。用其他适配器所用的同一个解析器构建它的 Reply:文本使用 make_reply,响应体使用 parse_chat_body,这样每个适配器读取工具调用的方式都相同。然后为它配上协议一节所描述的测试。循环不变,其他任何适配器也不变。

工作准则: 适配器保留原始文本和计数,在一处读取工具调用,并返回 Reply。它绝不把 SDK 对象交给循环。为每个适配器编写一个测试,用循环的参数调用 reply();提示词或请求发生变化时,应当重新录制。

本章,在本版次中

版次 2026.11.2示例于 2026年11月20日 通过本章 4个示例 个示例中有 4 个通过

译自英文版图书,版次 2026.11.2。

引用本节

格式

每种格式均引用英文原书,本译本以其为准。

每种格式都会注明该版次。其链接会打开当前版次,节标题处会说明该节文本最后一次修改的时间。访问日期和检索日期即为您打开此页的当天。

本页内容

第 4 章,共 12 章: 脚本化、回放、本地与托管模型的统一接口

  1. Ask every model the same question
  2. Script the replies, or replay a stored run
  3. 本地模型,以及它的设置为何固定
  4. 托管式端点,在环回地址上测试
  5. 一个循环,多种模型

返回顶部全部章节