文件沒寫就拒答,並衡量兩種錯誤
閱讀約需 50 分鐘 分鐘,6 節 中共有 3 個範例。
讀完本章,你將能夠計算助理無法依據文件作答的兩種情形,在各自的分母上解讀這兩個比率,並判斷一條更常拒答的規則何時值得付出其代價。
本書中的助理負責回答關於 sablefile 的問題。sablefile 是一個命令列工具,用來讓專案中產生的檔案與其來源範本保持同步,而助理依據該工具的文件作答。當文件沒有涵蓋某個問題時,助理應當只回覆一句固定的話:「The docs don't say.」,不附加任何其他內容。這句話看起來是安全的答案。團隊在記錄中看到錯誤的回覆時,容易想讓助理更常拒答,因為拒絕看起來比較謹慎。然而拒絕也可能是錯的。如果文件其實回答了該問題,收到這句話的使用者就被辜負了,而且任何地方都不會顯示錯誤。如果文件並未回答,助理卻仍然作答,使用者得到的就是沒有任何依據的回覆。更常拒答,只是以其中一種錯誤換取另一種。只計算其中一種錯誤的團隊,會朝著一個看起來漂亮的數字調整,並掩蓋另一種錯誤的代價。
本章針對本書自己的回覆計算這兩種錯誤。接著,在未過濾的變體上測試最直接的修正方法:當排名第一的節的搜尋分數偏低時就拒答的防護機制。在該變體上,這個防護機制沒有幫助。這就是本章的結果,而後文大部分內容都在說明這個結果如何測量,以及它可以信賴到什麼程度。
此部分尚未翻譯,因此以英文呈現。
Two errors, both counted
Every question in the test set has a kind. An answerable question is one that the docs of the asked version answer. An unanswerable question is one they don't. A reply that declines an answerable question makes a false refusal, and a reply that answers an unanswerable question makes a false answer. The first chapter defined both in its decline check, and this chapter uses the same definitions on more items.
The two rates have different denominators, and the denominators carry the point. The false refusal rate is the number of answerable items declined, over the number of answerable items. The false answer rate is the number of unanswerable items answered, over the number of unanswerable items. Put both errors over one total, and a set with many answerable items makes the refusal count look small, while the answers that should have been declines get lost in the sum. Each rate needs its own count, and each count needs its own denominator.
Two kinds of item stay out of both rates. A version-dependent item asks about something that differs between versions, so its reply is judged against the reference for the asked version, and its own rate, version-correctness, is covered in chapter 9. An ambiguous item has two readings and is judged by the pass rate, not by either decline rate. A decline on either kind is not an error here. The code still counts those declines and prints them, so a decline on one of them can't go unnoticed.
The code counts a reply as a decline only when the reply is the decline sentence and nothing more. A reply that gives the sentence and then an answer is not a decline, so it counts as an answer in both rates. The same holds for a refusal in other words: the code matches the sentence exactly, so a refusal worded differently counts as an answer. On an answerable item that is not an error here, and on an unanswerable item it is a false answer, which is why the sentence is fixed in the prompt.
Working rule: Report the two decline rates apart, each over its own denominator, and leave version-dependent and ambiguous items out of both. Read the items behind each rate, not only the rate, because a rate can hide the few items that matter.
基準版本的表現
基準版本就是第一章所建構的助理。它使用提示 P1,要求每一項陳述都附上引用,且在各節文件沒有說明時回覆拒答句。它以 BM25 擷取前三節,只從所詢問版本的索引中擷取。它的回覆對每個項目只記錄一次,本範例讀取的就是那一次記錄下來的回覆。
三個變體各自只改動一件事。prompt-b 使用提示 P2,在給出答案之前先引述答案所依據的句子。k5 提供給模型的是前五節,而不是前三節。unfiltered 從所有版本的索引一併擷取,並在每一節的標頭標明其版本,使模型能看出某一節來自其他版本。每個變體對相同的項目各有一份回覆記錄,因此每一項比較都是針對同一批問題的回覆。
要為每份記錄計算兩種錯誤,需要一個範例:decline-rates。它讀取四份記錄在開發集與保留集上的結果,印出每個比率及其計數,並指出造成每種錯誤的項目。其輸出會顯示每份記錄拒答了哪些可回答的項目,以及回答了哪些不可回答的項目。
1"""Counts the two decline errors for each recording, on dev and on held-out: answerable items declined (false refusals)2and unanswerable items answered (false answers). Each rate line rests on its own count and denominator.34Version-dependent and ambiguous items are left out of both rates; the last line per split counts the declines among5them, which are not errors here."""6import lib # first: fixes the thread counts and seeds before anything else loads78from lib.contract import parse_reply9from lib.prompts import P1_SHA256, P2_SHA25610from lib.replay import ReplayModel11from lib.stats import pct12from lib.testset import load_items1314VARIANTS = ( # label, recording, the sha256 of the prompt it was made from15 ("baseline", "assistant-baseline", P1_SHA256), # P1, k=3, filtered to the asked version16 ("unfiltered", "assistant-unfiltered", P1_SHA256), # P1, k=3, sections from every version17 ("prompt-b", "assistant-prompt-b", P2_SHA256), # P2, k=3, filtered18 ("k5", "assistant-k5", P1_SHA256), # P1, k=5, filtered19)2021for label, recording, prompt_sha256 in VARIANTS:22 model = ReplayModel(recording, prompt_sha256) # refuses a recording made from another prompt23 for split in ("dev", "held-out"):24 items = load_items(split=split)25 declined = {item.id: parse_reply(model.reply(item.id, 0)).declined for item in items} # sample 0 only26 answerable = [item for item in items if item.kind == "answerable"]27 unanswerable = [item for item in items if item.kind == "unanswerable"]28 refused = sorted(item.id for item in answerable if declined[item.id])29 answered = sorted(item.id for item in unanswerable if not declined[item.id])30 other_kinds = ("version-dependent", "ambiguous")31 other = [item.id for item in items if item.kind in other_kinds and declined[item.id]]32 print(f"{label} {split} false refusal rate (answerable declined): {pct(len(refused), len(answerable))}")33 print(f"{label} {split} false refusals: {', '.join(refused) or 'none'}")34 print(f"{label} {split} false answer rate (unanswerable answered): {pct(len(answered), len(unanswerable))}")35 print(f"{label} {split} false answers: {', '.join(answered) or 'none'}")36 for item_id in answered: # what each false answer cites, and its text, so the chapter's claims can be checked37 reply = parse_reply(model.reply(item_id, 0))38 print(f"{label} {split} reply to {item_id} cites: {', '.join(reply.citations) or 'none'}")39 print(f"{label} {split} reply to {item_id} text: {reply.text}")40 print(f"{label} {split} declined, not counted (version-dependent and ambiguous): {len(other)}")
python decline-rates.pybaseline dev false refusal rate (answerable declined): 10.0% (4 of 40)
baseline dev false refusals: q-0052, q-0060, q-0063, q-0079
baseline dev false answer rate (unanswerable answered): 0.0% (0 of 13)
baseline dev false answers: none
baseline dev declined, not counted (version-dependent and ambiguous): 6
baseline held-out false refusal rate (answerable declined): 30.0% (6 of 20)
baseline held-out false refusals: q-0101, q-0102, q-0103, q-0105, q-0111, q-0118
baseline held-out false answer rate (unanswerable answered): 0.0% (0 of 7)
baseline held-out false answers: none
baseline held-out declined, not counted (version-dependent and ambiguous): 2
unfiltered dev false refusal rate (answerable declined): 12.5% (5 of 40)
unfiltered dev false refusals: q-0005, q-0052, q-0058, q-0060, q-0079
unfiltered dev false answer rate (unanswerable answered): 15.4% (2 of 13)
unfiltered dev false answers: q-0018, q-0039
unfiltered dev reply to q-0018 cites: remote-retries
unfiltered dev reply to q-0018 text: Yes, sablefile 1.1 can fetch a remote template from a private URL that needs a login token, as long as the token is included in the request. The retry behavior for such fetches is up to 5 times before giving up, and this applies to `sync` and `diff` operations [sec:remote-retries]. The `check` command does not fetch anything and thus does not apply to such scenarios.
unfiltered dev reply to q-0039 cites: check-exit-status
unfiltered dev reply to q-0039 text: When check finds a drifted file, it prints `DRIFT` and the path, and exits with status 1. [sec:check-exit-status]
unfiltered dev declined, not counted (version-dependent and ambiguous): 9
unfiltered held-out false refusal rate (answerable declined): 45.0% (9 of 20)
unfiltered held-out false refusals: q-0101, q-0102, q-0103, q-0105, q-0106, q-0111, q-0115, q-0116, q-0117
unfiltered held-out false answer rate (unanswerable answered): 0.0% (0 of 7)
unfiltered held-out false answers: none
unfiltered held-out declined, not counted (version-dependent and ambiguous): 3
prompt-b dev false refusal rate (answerable declined): 7.5% (3 of 40)
prompt-b dev false refusals: q-0052, q-0063, q-0079
prompt-b dev false answer rate (unanswerable answered): 0.0% (0 of 13)
prompt-b dev false answers: none
prompt-b dev declined, not counted (version-dependent and ambiguous): 4
prompt-b held-out false refusal rate (answerable declined): 25.0% (5 of 20)
prompt-b held-out false refusals: q-0101, q-0102, q-0103, q-0105, q-0111
prompt-b held-out false answer rate (unanswerable answered): 0.0% (0 of 7)
prompt-b held-out false answers: none
prompt-b held-out declined, not counted (version-dependent and ambiguous): 0
k5 dev false refusal rate (answerable declined): 7.5% (3 of 40)
k5 dev false refusals: q-0052, q-0062, q-0079
k5 dev false answer rate (unanswerable answered): 0.0% (0 of 13)
k5 dev false answers: none
k5 dev declined, not counted (version-dependent and ambiguous): 5
k5 held-out false refusal rate (answerable declined): 30.0% (6 of 20)
k5 held-out false refusals: q-0102, q-0103, q-0105, q-0106, q-0111, q-0116
k5 held-out false answer rate (unanswerable answered): 0.0% (0 of 7)
k5 held-out false answers: none
k5 held-out declined, not counted (version-dependent and ambiguous): 2
基準版本在開發集上的誤拒率為 10.0% (4 of 40)。它的誤拒項目是 q-0052、q-0060、q-0063 與 q-0079,每一項都是文件有涵蓋的可回答項目。它在開發集上的誤答率為 0.0% (0 of 13),也就是開發集中不可回答的項目,它一項都沒有回答。第一章同樣沒有發現誤答,但不可回答的項目只有四個,數量太少,不足以據以判斷比率的高低。基準版本另外拒答了 6 個開發集項目,它們屬於兩個比率都不計入的兩類。
第 2 章為這個基準版本計得 10 次過度拒答。該計數涵蓋文件有答案的每個項目,包括與版本相關的項目,因此其中既有上述的誤拒,也有對與版本相關及語意模糊的項目所做的拒答,而後者不在這個比率之內。
下表把四份記錄在兩個資料集上並列。每個儲存格列出一個比率及其計數。
| 記錄 | 開發集誤拒 | 開發集誤答 | 保留集誤拒 | 保留集誤答 |
|---|---|---|---|---|
| baseline | 10.0% (4 of 40) | 0.0% (0 of 13) | 30.0% (6 of 20) | 0.0% (0 of 7) |
| unfiltered | 12.5% (5 of 40) | 15.4% (2 of 13) | 45.0% (9 of 20) | 0.0% (0 of 7) |
| prompt-b | 7.5% (3 of 40) | 0.0% (0 of 13) | 25.0% (5 of 20) | 0.0% (0 of 7) |
| k5 | 7.5% (3 of 40) | 0.0% (0 of 13) | 30.0% (6 of 20) | 0.0% (0 of 7) |
先看 unfiltered 這一列,因為它是唯一在開發集上出現誤答的一列。它的誤拒率為 12.5% (5 of 40),基準版本的為 10.0% (4 of 40)。unfiltered 拒答了兩個基準版本回答了的可回答項目,即 q-0005 與 q-0058,並且回答了基準版本拒答的 q-0063。淨結果是四十個項目中多拒答一項。在開發集上,每一處與基準版本的誤拒差異都是四十個項目中的一項。就這些項目而言,一個項目不足以據以判斷哪份記錄較好,區間與樣本大小說明了這種大小的樣本能顯示什麼。
unfiltered 變體也回答了兩個不可回答的開發集項目,而基準版本一個也沒有回答。它的誤答率為 15.4% (2 of 13)。這是兩個項目的計數,沒有區間,因此本章把它當作計數來報告,而不當作經過測量的上升。這兩個誤答是 q-0018 與 q-0039。
兩個問題都針對 sablefile 1.1,而 1.1 的文件都沒有這兩個答案:遠端範本能否以登入權杖從私有位址取得,以及 diff 在發現差異時回傳什麼結束狀態。基準版本對兩者都拒答。unfiltered 的回覆則兩者都回答,且各自引用了相鄰的一節。對 q-0018 的回覆引用了 remote-retries,即遠端擷取的重試規則。對 q-0039 的回覆引用了 check-exit-status,並給出 check 的結束狀態,而這是與問題所問的 diff 不同的指令。範例會印出每一筆引用與每則回覆的文字,因此不必打開記錄就能查驗這些說法。unfiltered 變體改變的是擷取而非提示,所以應先查看引用。
讓開發集拒答朝另一個方向變動的變體是 prompt-b 與 k5。兩者在開發集上的誤拒都比基準版本少一項,而且都沒有回答不可回答的開發集項目。prompt-b 的誤拒率為 7.5% (3 of 40),k5 的為 7.5% (3 of 40)。就這些項目而言,各自都比基準版本少一項,而一個項目不足以據以下結論。
保留集的數字僅為完整起見而報告。prompt-b 的誤拒率為 25.0% (5 of 20),k5 的為 30.0% (6 of 20),基準版本的為 30.0% (6 of 20)。prompt-b 與基準版本之間的差距是二十個項目中的一項,就這些項目而言,不足以據以下結論。本章不選定任何變體,因為開發集上的差異各只有一個項目。
實用原則: 在更動提示之前,先逐一人工檢視開發集上的每個誤拒,因為其中某一項可能指出文件中真實存在的缺漏。在相同的項目上比較各變體,且只有在範例印出配對區間時才報告區間。沒有印出時,就報告這些項目上的計數,不對哪個變體較好下結論。保留集只執行一次,用來確認在開發集上做出的選擇。unfiltered 變體是此處的警訊:它在開發集上回答了基準版本拒答的不可回答項目。本章在沒有區間的情況下報告這一點,因此它是需要查核的理由,而不是經過測量的上升。
此部分尚未翻譯,因此以英文呈現。
A guard on recorded replies
The guard rests on a plain idea. BM25 scores each section by the words it shares with the question. When the docs have nothing on a question, the best-matching section should share few words with it, so its score should be low. A guard reads the best-matching section's score and declines when it falls below a threshold. The idea is a hypothesis, and it can fail. A question may share many words with a section that doesn't answer it, and the sweep below is the test.
The guard can be measured without running the model again. Retrieval is deterministic: the same question gives the same top sections and the same scores each time. So the top score for every item can be computed from the index, and the guard becomes a rule applied to replies that have already been recorded. Its grid and rule are constants in lib/guard.py. The guard only turns answers into declines. It can never undo a decline, so it can lower false answers and can only raise false refusals.
The guard's score must come from the index that made the replies it judges, and the unfiltered replies were made from sections of every version. So the guard reads the unfiltered recording and takes the top BM25 score over the all-version index. A score from the asked version's index would be measured on sections the unfiltered model never saw. The guard is tested on the unfiltered variant because the baseline answers no unanswerable dev item, so a guard has nothing to remove there, while the unfiltered variant answers two. This chapter makes no claim about a guard on the baseline.
The grid runs from 0.00 to 10.00 in steps of a quarter, above every dev score. For each threshold the rule adds the two rates, each over its own denominator, and picks the threshold with the smallest sum. That equal weighting is a choice, and the too-many-and-too-few section returns to it. A tie goes to the lower threshold, because it changes fewer replies. The rule is stated in lib/guard.py and picks on dev. The held-out split is not untouched, since the first table in what the baseline does already shows its unguarded rates. The guard's held-out run came once, at the frozen threshold.
The sweep runs in tune-guard-dev, which prints both rates at every step of the grid and then the rule's pick. It opens with the check that matters most: the recorded sections of each item must match what the index returns, or the scores would describe a retrieval that never made the replies.
1"""Sweeps the decline guard's threshold over dev replies of the unfiltered recording, printing both rates at each2threshold, and the threshold the dev rule picks. The line for 0.00 is the recorded replies with no guard, since no top3score is below zero; the rule's pick rests on the rate lines below it. Only dev is read here."""4import lib # first: fixes the thread counts and seeds before anything else loads56from lib.guard import GRID, GUARD_RECORDING, RULE, contexts_match, denominators, errors, pick_threshold, rows7from lib.stats import pct89dev = rows("dev")10answerable, unanswerable = denominators(dev)11print(f"recording: {GUARD_RECORDING} (all versions, dev items)")12print(f"answerable dev items: {answerable}")13print(f"unanswerable dev items: {unanswerable}")1415mismatches = contexts_match("dev")16print(f"dev items whose recorded sections differ from the retrieval: {mismatches}")17assert mismatches == 0, "the scores come from a retrieval that did not make the recorded replies"18highest = max(row.score for row in dev)19print(f"highest top score on dev: {highest:.3f}")20assert highest < GRID[-1], "the grid must reach above every dev score"2122print(f"rule: {RULE}")23for threshold in GRID:24 refusals, answers = errors(dev, threshold)25 print(f"threshold {threshold:.2f}, false refusal rate: {pct(len(refusals), answerable)}")26 print(f"threshold {threshold:.2f}, false answer rate: {pct(len(answers), unanswerable)}")2728refusals, answers = errors(dev, 0.0)29print(f"recorded false refusals (no guard): {', '.join(refusals) or 'none'}")30print(f"recorded false answers (no guard): {', '.join(answers) or 'none'}")31scores = {row.item_id: row.score for row in dev}32for item_id in answers: # the top score of each recorded false answer, so a reader can see where it falls on the grid33 print(f"top score of recorded false answer {item_id}: {scores[item_id]:.3f}")3435chosen = pick_threshold(dev)36assert chosen == 0.0, f"the dev rule now picks {chosen:.2f}, not 0.00"37refusals, answers = errors(dev, chosen)38print(f"threshold chosen by the dev rule: {chosen:.2f}")39print(f"at the chosen threshold, false refusal rate: {pct(len(refusals), answerable)}")40print(f"at the chosen threshold, false answer rate: {pct(len(answers), unanswerable)}")
python tune-guard-dev.pyrecording: assistant-unfiltered (all versions, dev items)
answerable dev items: 40
unanswerable dev items: 13
dev items whose recorded sections differ from the retrieval: 0
highest top score on dev: 9.949
rule: pick the threshold in GRID whose false refusal rate plus false answer rate is smallest, each rate over its own denominator; when two thresholds tie, take the lower one
threshold 0.00, false refusal rate: 12.5% (5 of 40)
threshold 0.00, false answer rate: 15.4% (2 of 13)
threshold 0.25, false refusal rate: 12.5% (5 of 40)
threshold 0.25, false answer rate: 15.4% (2 of 13)
threshold 0.50, false refusal rate: 12.5% (5 of 40)
threshold 0.50, false answer rate: 15.4% (2 of 13)
threshold 0.75, false refusal rate: 12.5% (5 of 40)
threshold 0.75, false answer rate: 15.4% (2 of 13)
threshold 1.00, false refusal rate: 12.5% (5 of 40)
threshold 1.00, false answer rate: 15.4% (2 of 13)
threshold 1.25, false refusal rate: 12.5% (5 of 40)
threshold 1.25, false answer rate: 15.4% (2 of 13)
threshold 1.50, false refusal rate: 12.5% (5 of 40)
threshold 1.50, false answer rate: 15.4% (2 of 13)
threshold 1.75, false refusal rate: 12.5% (5 of 40)
threshold 1.75, false answer rate: 15.4% (2 of 13)
threshold 2.00, false refusal rate: 12.5% (5 of 40)
threshold 2.00, false answer rate: 15.4% (2 of 13)
threshold 2.25, false refusal rate: 15.0% (6 of 40)
threshold 2.25, false answer rate: 15.4% (2 of 13)
threshold 2.50, false refusal rate: 17.5% (7 of 40)
threshold 2.50, false answer rate: 15.4% (2 of 13)
threshold 2.75, false refusal rate: 17.5% (7 of 40)
threshold 2.75, false answer rate: 15.4% (2 of 13)
threshold 3.00, false refusal rate: 20.0% (8 of 40)
threshold 3.00, false answer rate: 15.4% (2 of 13)
threshold 3.25, false refusal rate: 30.0% (12 of 40)
threshold 3.25, false answer rate: 15.4% (2 of 13)
threshold 3.50, false refusal rate: 32.5% (13 of 40)
threshold 3.50, false answer rate: 15.4% (2 of 13)
threshold 3.75, false refusal rate: 37.5% (15 of 40)
threshold 3.75, false answer rate: 15.4% (2 of 13)
threshold 4.00, false refusal rate: 40.0% (16 of 40)
threshold 4.00, false answer rate: 7.7% (1 of 13)
threshold 4.25, false refusal rate: 45.0% (18 of 40)
threshold 4.25, false answer rate: 7.7% (1 of 13)
threshold 4.50, false refusal rate: 55.0% (22 of 40)
threshold 4.50, false answer rate: 7.7% (1 of 13)
threshold 4.75, false refusal rate: 60.0% (24 of 40)
threshold 4.75, false answer rate: 0.0% (0 of 13)
threshold 5.00, false refusal rate: 62.5% (25 of 40)
threshold 5.00, false answer rate: 0.0% (0 of 13)
threshold 5.25, false refusal rate: 70.0% (28 of 40)
threshold 5.25, false answer rate: 0.0% (0 of 13)
threshold 5.50, false refusal rate: 77.5% (31 of 40)
threshold 5.50, false answer rate: 0.0% (0 of 13)
threshold 5.75, false refusal rate: 77.5% (31 of 40)
threshold 5.75, false answer rate: 0.0% (0 of 13)
threshold 6.00, false refusal rate: 80.0% (32 of 40)
threshold 6.00, false answer rate: 0.0% (0 of 13)
threshold 6.25, false refusal rate: 82.5% (33 of 40)
threshold 6.25, false answer rate: 0.0% (0 of 13)
threshold 6.50, false refusal rate: 87.5% (35 of 40)
threshold 6.50, false answer rate: 0.0% (0 of 13)
threshold 6.75, false refusal rate: 87.5% (35 of 40)
threshold 6.75, false answer rate: 0.0% (0 of 13)
threshold 7.00, false refusal rate: 92.5% (37 of 40)
threshold 7.00, false answer rate: 0.0% (0 of 13)
threshold 7.25, false refusal rate: 92.5% (37 of 40)
threshold 7.25, false answer rate: 0.0% (0 of 13)
threshold 7.50, false refusal rate: 92.5% (37 of 40)
threshold 7.50, false answer rate: 0.0% (0 of 13)
threshold 7.75, false refusal rate: 92.5% (37 of 40)
threshold 7.75, false answer rate: 0.0% (0 of 13)
threshold 8.00, false refusal rate: 92.5% (37 of 40)
threshold 8.00, false answer rate: 0.0% (0 of 13)
threshold 8.25, false refusal rate: 95.0% (38 of 40)
threshold 8.25, false answer rate: 0.0% (0 of 13)
threshold 8.50, false refusal rate: 95.0% (38 of 40)
threshold 8.50, false answer rate: 0.0% (0 of 13)
threshold 8.75, false refusal rate: 95.0% (38 of 40)
threshold 8.75, false answer rate: 0.0% (0 of 13)
threshold 9.00, false refusal rate: 97.5% (39 of 40)
threshold 9.00, false answer rate: 0.0% (0 of 13)
threshold 9.25, false refusal rate: 97.5% (39 of 40)
threshold 9.25, false answer rate: 0.0% (0 of 13)
threshold 9.50, false refusal rate: 97.5% (39 of 40)
threshold 9.50, false answer rate: 0.0% (0 of 13)
threshold 9.75, false refusal rate: 97.5% (39 of 40)
threshold 9.75, false answer rate: 0.0% (0 of 13)
threshold 10.00, false refusal rate: 100.0% (40 of 40)
threshold 10.00, false answer rate: 0.0% (0 of 13)
recorded false refusals (no guard): q-0005, q-0052, q-0058, q-0060, q-0079
recorded false answers (no guard): q-0018, q-0039
top score of recorded false answer q-0018: 3.762
top score of recorded false answer q-0039: 4.592
threshold chosen by the dev rule: 0.00
at the chosen threshold, false refusal rate: 12.5% (5 of 40)
at the chosen threshold, false answer rate: 15.4% (2 of 13)
The check passes: zero dev items differ, and the example asserts that count is zero. The highest top score on dev is 9.949, which sits below the top of the grid.
Start at the bottom of the sweep. At 0.00 no score falls below the threshold, so the line shows the recorded replies with no guard. The false refusal rate there is 12.5% (5 of 40). The false answer rate is 15.4% (2 of 13). Every line up to 2.00 reads the same. At 2.25 the false refusals begin to rise, to 15.0% (6 of 40).
The false answers don't fall until 4.00, where the false answer rate is 7.7% (1 of 13). At 4.75 the false answer rate is 0.0% (0 of 13), so no unanswerable dev item is answered. The false refusal rate at 4.75 is 60.0% (24 of 40).
No line of the sweep lowers both rates below the no-guard line. Each threshold that lowers the false answers raises the false refusals, and each threshold that leaves the refusals alone leaves the answers alone. On dev, the guard makes a trade and no improvement.
The sweep also shows the limit of the score. The top score of q-0018 is 3.762, and the top score of q-0039 is 4.592. A threshold catches q-0018 only from 4.00 up, where the false refusal rate is already 40.0% (16 of 40). It catches q-0039 only from 4.75 up. No cut catches both answered items with fewer than half of the answerable items declined. The likely cause is the score itself: it counts shared words, and a section about the same command can share many words with a question that section doesn't answer. That is a reading of the sweep, not a measured fact. Testing it would mean scoring the items by hand, which this chapter does not do.
The rule's pick is 0.00. At that threshold the false refusal rate is 12.5% (5 of 40). The false answer rate is 15.4% (2 of 13). Both are the same as the recorded replies with no guard. The pick is a tie-break, not a finding. The lines from 0.00 to 2.00 have the same sum, and the rule takes the lowest of them, so the guard the rule chooses is no guard.
Working rule: Build a guard's scores from the same index that made the replies, and check that match before you read any rate. Then read the whole sweep with both rates on every line. A pick that ties with doing nothing says the rule found no reason to change a reply, and that is an answer worth keeping.
拒答過多與過少
這份掃描結果呈現了團隊加入防護機制後會面對的取捨,也顯示了每一種衡量方式各會回報什麼。假設某個團隊只計算錯誤回答。能在開發集上擋下所有錯誤回答的最低門檻是 4.75,因此該團隊會選 4.75,並回報助理已不再回答它無法佐證的問題。掃描結果中 4.75 那一行顯示了這份回報所隱藏的代價。在 4.75 時,誤拒率為 60.0% (24 of 40)。報告中不會有任何地方提到這點,因為團隊從未計算另一種錯誤。
在實際上線的環境中,這正是需要留意的失效模式。使用者提出文件能夠回答的問題,卻得到拒答的句子,而回覆中沒有任何跡象顯示這個句子是錯的。這項代價唯一出現的地方就是誤拒次數。不計算它的團隊只看到錯誤回答為零,便直接上線。
這條規則對兩種錯誤給予相同的權重,而這是屬於團隊自己的選擇。團隊可能認為錯誤回答的代價遠高於誤拒,例如一個語氣篤定的錯誤回答會讓使用者去嘗試錯誤的修正方式。只要該團隊的權重大到足以改變加總結果,它在開發集上就會得到不同的選擇,而掃描結果顯示了方向:對錯誤回答給予較高的權重,會把選擇推向拒答較多的門檻。這樣的團隊應當在任何人查看掃描結果之前先寫下自己的權重,再把權重交給這條規則。本章不會以其他權重重新執行這條規則,也不會為此重新開啟保留的資料切分。在看過保留集的數字之後才選定權重,就等於在該切分上進行了第二輪調校。
門檻也屬於產生這些回覆的那個助理。分數取決於索引,而索引取決於各段落(節)如何切分以及收錄了哪些版本。其中任何一項改變,分數就會跟著變動,因此必須在開發集上重新選定門檻,而且在任何人查看比率之前,範例中的第一項檢查必須再次通過。
實用原則: 在調校門檻之前,先以團隊自己的說法寫下每種錯誤的代價,並在任何人查看掃描結果之前,把這些權重交給這條規則。
讀者應當接受什麼
保留集只執行了一次,門檻凍結在 0.00。執行的程式是 guard-held-out,它先斷言開發集規則(見防護機制)仍然選出這個值。因此,只要開發資料或規則有任何變動,執行就會在印出任何保留集數字之前停止。
1"""Applies the decline guard, with its threshold frozen from dev, to the held-out items of the unfiltered recording, and2prints both rates with 95% bootstrap intervals over items. The threshold is a constant chosen by tune-guard-dev's rule3on dev; the assert checks that the rule still picks it, so a change to the dev data or the rule fails here."""4import lib # first: fixes the thread counts and seeds before anything else loads56from lib.guard import denominators, errors, guarded, pick_threshold, rows7from lib.stats import bootstrap_mean, interval_pct, pct89# Picked on dev by tune-guard-dev (the rule in lib/guard.py), then frozen. Held-out is not used to change it.10FROZEN_THRESHOLD = 0.01112dev_pick = pick_threshold(rows("dev")) # reads dev only13assert dev_pick == FROZEN_THRESHOLD, f"the dev rule now picks {dev_pick:.2f}, not the frozen {FROZEN_THRESHOLD:.2f}"1415held = rows("held-out")16answerable, unanswerable = denominators(held)17refusals, answers = errors(held, FROZEN_THRESHOLD)18print(f"threshold (frozen from dev): {FROZEN_THRESHOLD:.2f}")19print(f"held-out answerable items: {answerable}")20print(f"held-out unanswerable items: {unanswerable}")21changed = [row.item_id for row in held if guarded(row, FROZEN_THRESHOLD) != row.declined]22print(f"held-out replies the guard changed to declines: {len(changed)} of {len(held)}")2324# One 0/1 flag per item: the bootstrap over these flags is the bootstrap of the rate, over items.25refused_flags = [float(guarded(row, FROZEN_THRESHOLD)) for row in held if row.kind == "answerable"]26answered_flags = [float(not guarded(row, FROZEN_THRESHOLD)) for row in held if row.kind == "unanswerable"]27assert len(refused_flags) == answerable and len(answered_flags) == unanswerable2829refusal_lo, refusal_hi = bootstrap_mean(refused_flags, level=0.95) # 2,000 resamples, seeded (lib/stats.py)30answer_lo, answer_hi = bootstrap_mean(answered_flags, level=0.95)31print(f"false refusal rate: {pct(len(refusals), answerable)} interval {interval_pct(refusal_lo, refusal_hi)}")32print(f"false refusals: {', '.join(refusals) or 'none'}")33print(f"false answer rate: {pct(len(answers), unanswerable)} interval {interval_pct(answer_lo, answer_hi)}")34print(f"false answers: {', '.join(answers) or 'none'}")
python guard-held-out.pythreshold (frozen from dev): 0.00
held-out answerable items: 20
held-out unanswerable items: 7
held-out replies the guard changed to declines: 0 of 27
false refusal rate: 45.0% (9 of 20) interval [25.0%, 70.0%]
false refusals: q-0101, q-0102, q-0103, q-0105, q-0106, q-0111, q-0115, q-0116, q-0117
false answer rate: 0.0% (0 of 7) interval [0.0%, 0.0%]
false answers: none
凍結後的防護機制改變了 0 of 27 筆保留集回覆,所以保留集的輸出就是沒有防護機制時記錄的輸出。在保留集上,錯誤拒答率為 45.0% (9 of 20) interval [25.0%, 70.0%]。錯誤作答率為 0.0% (0 of 7) interval [0.0%, 0.0%]。
錯誤拒答率與第一個表格中未過濾變體的相同,而且保留集的結果在主要論點上與開發集一致:在規則所選的門檻下,防護機制沒有任何作用。如果團隊只看保留集上的錯誤作答率,會看到零,並因此斷定助理不會回答它無法支持的問題。錯誤其實出在拒答上,而任何門檻都無法消除這些錯誤,因為防護機制只會增加拒答。
保留集的錯誤作答率為 0.0% (0 of 7) interval [0.0%, 0.0%]。
它印出的區間沒有寬度。這個區間是對保留集中七個無法回答的項目重新抽樣。其中沒有任何一項被錯誤作答,所以每次重新抽樣都沒有錯誤,自助法(bootstrap)也就無從產生變動。忠實性說明了這種零事件情況的推理;本範例並未印出精確的界限,因此這個區間不能當作界限來讀。它無法區分錯誤作答率是零,還是一個這七個項目剛好沒有碰到的小比率。七個項目只能顯示其中沒有任何一項被錯誤作答。
開發集的情況也沒有比較好。在開發集上,未過濾的回覆在沒有防護機制時得到 15.4% (2 of 13)。門檻為 4.00 時得到 7.7% (1 of 13),少了一個項目。
錯誤拒答的區間之所以寬,原因不同:二十個項目留下一段資料無法區分的比率範圍。保留集的數字為 45.0% (9 of 20) interval [25.0%, 70.0%]。區間與樣本數說明了建立這些區間所用的逐項目自助法,以及成對差異需要多少項目。Kohavi 等人針對網路上的對照實驗提出了相同的看法。在他們的指南網路上的對照實驗:調查與實務指南中,檢定力不足的測試被列為常見的錯誤。七個項目的保留集對這麼小的比率而言檢定力不足,本章明白指出這一點,而不讓零代表確定性。
綜合來看,這些結果說明了具體的事。在未過濾的變體上,於開發集規則所選的門檻下,防護機制不改變任何回覆,所以本章不加入它。在開發集上,兩種錯誤都仍然存在於未過濾的回覆中。在保留集上,代價是錯誤拒答,而本章中沒有任何做法能降低它們。這個結果是一項決定,也是測量所支持的決定。
本章的決定如下:
- 兩種拒答率都放進每一次比較,各自除以自己的分母。
- 依版本而定的項目與模糊的項目不計入兩種比率,它們的拒答另行計算。
- 拒答規則在開發集上挑選,事先寫下其權重,讀完整個掃描結果,並在執行保留集之前凍結所選的值。
- 保留集每條規則只執行一次。零事件上的區間要附上這項但書來報告,而不是當作零的發現。
- 檢索分數防護機制不加入未過濾的變體。它在開發集上的選擇不改變任何回覆,在保留集上的執行也不改變任何回覆。
實用原則: 只有當防護機制在保留集上的執行,朝著開發集掃描所承諾的方向改變回覆時,它才值得保留。當它沒有做到時,應保留這個結果,因為它顯示了助理有哪些錯誤,也顯示檢索分數上的門檻無法修正這些錯誤。
本書範圍之外
本章不討論攻擊。使用者設法讓助理說出不該說的話,屬於另一個資安問題,有其自身的威脅與防禦方式,並且已規劃另一本專書處理,書名為 Securing LLM Applications and Agents。本章的拒答率衡量的是針對文件所提出的誠實問題,這些數字無法說明助理在使用者刻意施壓、企圖破壞時會有什麼表現。
其他研究者已在別的模型上衡量過同樣的取捨,其主張值得了解,但不必借用他們的數字。XSTest 是一組看起來不安全、實則安全的提示所構成的測試集。其作者之所以建立它,是因為模型會拒絕那些用詞與不安全提示相近、或涉及敏感主題的安全提示(Röttger 等人,2024)。OR-Bench 發表於 ICML 2025,衡量了許多模型,並指出經過安全調校的模型所增加的安全性,往往伴隨著更多的過度拒答(Cui 等人,2024)。一篇關於語言模型棄答(abstention)的綜述,為模型拒絕回答這件事給出了通用名稱,並依查詢、模型與人類價值三個面向整理棄答行為(Wen 等人,2024)。一項 2026 年的醫療保健基準測試,則以其自身的提示集,回報了該領域中對無害提示的過度拒答(Zhang 等人,2026)。
這些論文是在其他測試集上測試其他模型,因此它們的比率無法直接套用到這個助理身上。能夠沿用的是它們呈現的形態,也就是本章要在自身案例上檢驗的主張。在這組測試集上,防護機制要降低錯誤答案,只能靠多拒答一些本來可以回答的題目,這是以錯誤答案換取錯誤拒答的取捨。提示與檢索方面的調整可以同時改變這兩種錯誤,未經過濾的變體就做到了。本章回報的結果,僅限於本書自行執行的實驗。
實用原則: 不要因為某篇論文指出模型會過度拒答,就直接採用某項拒答規則;應先在自己的助理與自己的題目上衡量該規則,並同時計入兩種錯誤。論文的數字不要放進自己的結果裡,而應引用該論文,說明是什麼想法促成了這項衡量。
本章,於本版次中
版次 2026.11.3範例於 2026年11 月 21 通過測試本章 3 個範例 個範例中有 3 個通過
譯自英文書,版次 2026.11.3。