搜尋的完整流程

閱讀約需 78 分鐘 分鐘,10 節 中共有 11 個範例。

版次 2026.11.10 範例於 2026年11 月 19 通過測試 132 個範例中有 132 個通過 變更內容

由 AI 翻譯。經人工以英文驗證。 我們如何製作書籍

譯自英文書,版次 2026.11.10。 閱讀原文

讀完本章後,你可以為商品目錄建立一條粗略但完整的搜尋流程,從購物者的查詢一路到排序好的商品清單,並且能用一個數字衡量流程中的某項改動是否有幫助。後續每一章都會選取這條流程的一個部分,把它做得穩固。

在一家線上家具與居家用品商店裡,購物者會在搜尋框中輸入像是 clear shower curtain 或 48 inch kitchen hood 這樣的查詢。本書全程以這家商店為例,並稱之為「商店」。商店需要兩個答案:它的搜尋品質有多好,以及某項改動是否讓搜尋變得更好。無論面對哪一種搜尋系統,你都會遇到這兩個問題,因此本書把每一種方法都拿到商店上試驗,讓你能看到各方法對真實查詢造成的影響。

商店已經具備回答這些問題所需的資料:一份商品目錄、一組真實查詢,以及人們對於哪些商品符合哪個查詢所做的判斷。因此在本章中,你會以三種方式搜尋目錄(比對字詞、比對語意,以及兩者結合),用較慢的模型將最佳結果重新排序,並依據這些判斷為每個版本評分。並非每個階段都會提高分數。這正是本章的主要教訓:每一項改動都要衡量。

此部分尚未翻譯,因此以英文呈現。

Load the store's data

最近一次變動於第 2026.11.10 版

Before you can improve a search system, you need three things: the items it searches, the queries people type, and judgments that say which items match which query. Search engines call the items documents. In this book they are products. A judgment is a person's label for one pair of a query and a product, and a grade is the number that stands for a judgment's label. Without judgments, you can look at results and form opinions, but you can't tell a fix from a regression.

The store's data, which the book's examples search and score, is real. It is a sample of WANDS, a public product-search dataset that Wayfair published with a paper at ECIR 2022 (Chen et al., 2022), under the MIT licence (licence). The full dataset has 42,994 products, 480 queries and 233,448 judgments (repository). The queries are a stratified sample of real shopper queries from a quarter's search log in 2021, and each label is the majority vote of up to three trained annotators (Chen et al., sections 4.1 and 5). The script prep/wands_sample.py draws the book's sample, and the sample's data card, examples/data/wands/README.md, says how. Setup code that many examples need lives in the book's shared modules, short files in examples/lib/ that you can open and read. lib.store, one of those modules, loads the sample, and the next example prints what comes back:

load-the-store.py Python 3.12.15 · numpy 2.5.5 · pandas 3.1.1 通過
"""Loads the store's sample and prints its size, how its queries split, and how its judgments are labelled. The prose quotes the split from the `train queries` and `test queries` lines."""import pandas as pdfrom lib.store import DATA, load_labels, load_products, load_queriesproducts = load_products()queries = load_queries()labels = load_labels()                                   # one row per judged (query, product) pairconflicts = pd.read_csv(DATA / "label_conflicts.csv")    # pairs WANDS labelled two waysprint(f"products: {len(products):,}")print("queries:", len(queries))print("train queries:", (queries["split"] == "train").sum())print("test queries:", (queries["split"] == "test").sum())print(f"judged pairs: {len(labels):,}")print("pairs with conflicting labels:", len(conflicts))for label in ["exact", "partial", "irrelevant"]:    print(f"{label} pairs: {(labels['label'] == label).sum():,}")print(f"products judged for at least one query: {labels['product_id'].nunique():,}")print()print(queries.head(4).to_string())
輸出python load-the-store.py
products: 12,000
queries: 80
train queries: 50
test queries: 30
judged pairs: 13,077
pairs with conflicting labels: 7
exact pairs: 3,252
partial pairs: 7,972
irrelevant pairs: 1,853
products judged for at least one query: 11,917

                               query        query_class  split
query_id
6                acrylic clear chair      Dining Chairs  train
10                   king poster bed               Beds  train
20          foutains with brick look          Wallpaper  train
36        laundry basket with wheels  Hampers & Baskets  train

The sample has 12,000 products and 80 queries. It splits the queries once, before any ranker runs: 50 train queries for tuning and 30 test queries for reporting. Every score in this book is reported on the test queries, and nothing is tuned on them. A score on queries you tuned on measures how well you fit those queries, not how well search works.

WANDS judged some pairs more than once. The sample keeps one row per pair, 13,077 in all. When a pair's labels differ, it keeps the lower grade and lists the pair in label_conflicts.csv, which holds 7 pairs. Code that scores a ranking reads the judgments through lib.store.qrels, which returns them as {query_id: {product_id: grade}}. Chapter 2 reads the judgments closely.

Each judgment is one of three labels, defined by the dataset's authors (Chen et al., section 3.2). Exact means the product fully matches the search query. Partial means it "only matches the target entity of the query, but does not satisfy the modifiers for the query." Irrelevant means the product is not relevant to the query. The book's grades are 2, 1 and 0. Of the judged pairs, 3,252 are exact, 7,972 partial and 1,853 irrelevant. In this book, a relevant product is one judged exact for the query, unless a sentence says otherwise.

Three warnings come with the store's data. First, each query's query_class is WANDS's predicted class for it, not a judgment (Chen et al., Table 1): foutains with brick look is classed as Wallpaper. Treat it as a hint. Second, nobody judged every product for every query. That would have taken about 60 million judgments, so WANDS judged candidates gathered from products shoppers clicked or added to their carts and from several search systems, and stopped when few new exact matches turned up (Chen et al., sections 4.2 and 4.3). A product without a judgment for a query is unjudged: unknown, not irrelevant. Third, nearly every product in the sample, 11,917 of 12,000, is judged for at least one of its queries, so the sample holds fewer distractors than a full catalog would: products that compete for a query's top places without matching it.

Working rule: Split your queries into train and test before you change anything. Read test results to understand a ranker, but choose and tune every change on train queries, and report on test. Treat unjudged products as unknown. If you have no judgments yet, collect some before you change the ranking.

以 BM25 對商品排序

最近一次變動於第 2026.11.10 版

搜尋商店的商品目錄包含兩項工作。檢索(retrieval)是從整個目錄中找出可能符合查詢的商品。排序(ranking)則是把檢索到的商品依序排好,最佳者在前。詞彙式方法兩者都是靠比對詞彙來完成:比對查詢的詞與每項商品文字中的詞,並為重疊的部分計分。

BM25 是一種詞彙式計分函式,由三個部分組成。商品若包含查詢中的詞,分數就較高,若是罕見的詞,加分更多,因為出現在少數商品中的詞,更能說明那些商品的特性。同一個詞重複出現,所增加的分數會越來越少,直到上限為止,BM25 的文獻稱之為飽和(saturation)。此外,長文本相對於短文本會被打折,因此不能靠包含更多的詞取勝;參數 b 決定折扣的幅度,範圍從 0(不打折)到 1(完全折扣)(Robertson and Zaragoza, 2009)。第 5 章會以手工方式建構這個公式。

本書的 BM25 以 bm25s 0.3.16 執行,這是一個 BM25 的 Python 函式庫。預設情況下,它採用 Lucene 搜尋函式庫所使用的 BM25 變體計分,設定為 k1=1.5(飽和設定)與 b=0.75(bm25s source)。它的分數與教科書上的公式略有差異,不過一項針對三個新聞專線文件集的重現性研究發現,BM25 各變體在效果上沒有顯著差異(Kamphuis et al., 2020)。計分之前,bm25s 會先將文字切分為詞元(token)。它會將詞元轉為小寫,保留兩個或以上字元的詞,移除如 and 之類的英文停用詞,而且不會將詞還原為共同的詞幹(bm25s tokenizer)。本書使用的排序器 lib.lexical.LexicalSearch,會對每項商品的名稱、類別(class)、分類(category)與描述建立索引,並將這些內容串接成單一文字。

商店的測試查詢 clear shower curtain 可以展示 BM25 的運作。輸入這個查詢的消費者想要的是透明的簾子,但目錄中有許多並不透明的簾子,還有同樣含有這些詞的配件。以下是 BM25 排名前 10 的結果:

這十項中大多數是浴簾或浴簾內襯,但最後幾項是浴簾掛鉤。BM25 在掛鉤的名稱中找到「shower curtain」這些詞,並給了它們幾乎與浴簾一樣高的分數,因為它比對的是詞,而不是事物:它並不知道掛鉤不是簾子。這裡印出的內容完全沒有說明這些浴簾是否透明。要知道這一點,就需要商店的判斷標註。

實用原則: 先以函式庫的預設值執行 BM25,作為你的基準。它不需要訓練資料,也不需要模型,而且出錯時,你能看出是哪些詞造成的。在你能於訓練查詢上衡量出變化之前,不要調整它的參數。當消費者與商品對同一件事物使用不同的詞時,再加入語意檢索。

對照判斷檢查結果

最近一次變動於第 2026.11.8 版

在你知道購物者想要什麼之前,排名本身意義不大。判斷(judgment)正好提供這項資訊,而且是逐一配對的。就這個目的而言,判斷比點擊是更好的證據。WANDS 的作者指出:「User behavior log data suffer from positional biases and would favor the rankings similar to the production system」(Chen et al.,第 3.1 節)。購物者會點擊最先顯示的商品,因此點擊大多只是在確認你現有的排名。

接下來,把 BM25 針對 clear shower curtain 傳回的前 10 項商品,逐一與其判斷並列。若某項商品在這個查詢下沒有人判斷過,就會顯示為 unjudged:

judge-first-results.py Python 3.12.15 · bm25s 0.3.16 · numpy 2.5.5 · pandas 3.1.1 · scipy 1.18.1 通過
"""Puts BM25's top 10 for `clear shower curtain` beside each product's judgment, and counts the labels. The prose's reading of the top 10 rests on the labelled rows and the `in the top 10` lines after them."""from lib.lexical import LexicalSearchfrom lib.store import load_products, load_queries, qrelsLABELS = {2: "exact", 1: "partial", 0: "irrelevant"}products = load_products()query_id = 246query = load_queries().at[query_id, "query"]judgments = qrels()[query_id]                        # {product_id: grade} for this querytop = LexicalSearch(products).search(query, k=10).index# a product with no judgment for this query is unjudged, not irrelevantresults = [LABELS.get(judgments.get(product_id), "unjudged") for product_id in top]print("query:", query)print()for rank, (product_id, label) in enumerate(zip(top, results), start=1):    print(f"{rank:>2}  {label:<10}  {products.at[product_id, 'name'][:55]}")print()for label in ["exact", "partial", "irrelevant", "unjudged"]:    print(f"{label} in the top 10: {results.count(label)}")print("products judged for this query:", len(judgments))print("exact products judged for this query:", sum(grade == 2 for grade in judgments.values()))
輸出python judge-first-results.py
query: clear shower curtain

 1  partial     rodas vinyl shower curtain
 2  exact       woolley vinyl single shower curtain
 3  exact       woolley vinyl solid color single shower curtain
 4  exact       siniard single shower curtain
 5  exact       simmon vinyl single shower curtain liner
 6  partial     71 '' x71 '' 4pcs shower curtain sets beach theme showe
 7  partial     newburg check 3d vinyl single shower curtain + hooks
 8  irrelevant  gilded pineapple shower curtain hooks
 9  irrelevant  double shower curtain hooks
10  irrelevant  stainless steel shower curtain hooks

exact in the top 10: 4
partial in the top 10: 3
irrelevant in the top 10: 3
unjudged in the top 10: 0
products judged for this query: 253
exact products judged for this query: 12

BM25 的前 10 名包含 4 項完全相符(exact)的商品、3 項部分相符(partial)的商品、3 項不相關(irrelevant)的商品(也就是配件),以及 0 項未經判斷的商品。第一名的結果屬於部分相符:依據資料集的定義,它符合購物者所要找的物品,也就是浴簾,但不符合修飾語 clear(透明)。

判斷也能告訴你,最理想的前 10 名會是什麼樣子。評審針對這個查詢標註了 253 項商品,其中 12 項為完全相符。因此理想的排名可以讓十個位置全部都是完全相符的商品,而 BM25 只填入了 4 項。這個差距,就是指標必須衡量的對象。

用這種方式閱讀結果,可以看出排序器會犯哪一類錯誤。在這個範例中,BM25 比對到的是配件的字詞,卻漏掉了修飾語。對許多查詢取平均值會掩蓋這一點,而你親眼看到的錯誤,正是你能夠修正的錯誤。

實用原則: 在計算任何指標之前,先閱讀五到十個查詢的前幾名結果,並與其判斷並列對照。留意其中的模式:配件、漏掉的修飾語、錯誤的尺寸。當前幾名中有數項未經判斷時,在你信任該查詢的分數之前,先請人為這些商品做判斷。

此部分尚未翻譯,因此以英文呈現。

Score one ranking with nDCG@10

最近一次變動於第 2026.11.10 版

To compare rankings, you need one number per ranking. It should reward matching products, and exact ones more than partial ones, and count a product near the top more than one near the bottom, because shoppers read from the top. Normalized discounted cumulative gain at 10, nDCG@10, does all three.

In words: each of the top 10 results earns a gain equal to its grade: 2 for exact, 1 for partial, 0 for irrelevant. The gain is divided by a discount that grows with position: log2 of the position plus one, so the first result counts in full and the third counts half. Adding up the discounted gains gives the discounted cumulative gain, DCG. Dividing by the DCG of the best possible top 10 for that query, the ideal DCG, puts the score between 0 and 1, where 1 means no better top 10 exists. An unjudged product earns 0.

DCG@10  = sum over positions i from 1 to 10 of  grade(i) / log2(i + 1)
nDCG@10 = DCG@10 / ideal DCG@10

The WANDS paper scores search with nDCG@10 too, with gains of 1.0, 0.5 and 0 for exact, partial and irrelevant (Chen et al., section 6). The book's grades are twice those gains, which doubles both DCG and ideal DCG, so nDCG doesn't change.

Take BM25's ranking of clear shower curtain: a partial product first, then exact and partial curtains, then three irrelevant hooks. Compute DCG, the ideal DCG and their ratio by hand, then compare with lib.metrics, the shared module of the book's metrics:

ndcg-for-one-query.py Python 3.12.15 · bm25s 0.3.16 · numpy 2.5.5 · pandas 3.1.1 · scipy 1.18.1 通過
"""Computes nDCG@10 for one test query step by step, then checks it against `lib.metrics`. The prose's point rests on the `ndcg@10` line and on the library line that prints the same value."""import mathfrom lib.lexical import LexicalSearchfrom lib.metrics import ndcg_at_kfrom lib.store import load_products, load_queries, qrelsproducts = load_products()query = load_queries().at[246, "query"]                      # "clear shower curtain"judgments = qrels("test")[246]                                # {product_id: grade}ranking = list(LexicalSearch(products).search(query, k=10).index)grades = [judgments.get(product_id, 0) for product_id in ranking]     # unjudged counts as 0# positions count from 1, so the first result is discounted by log2(2) = 1dcg = sum(grade / math.log2(position + 1) for position, grade in enumerate(grades, start=1))best = sorted(judgments.values(), reverse=True)[:10]                  # the best possible top 10ideal = sum(grade / math.log2(position + 1) for position, grade in enumerate(best, start=1))print("grades:", grades)print("best grades:", best)print(f"dcg@10: {dcg:.3f}")print(f"ideal dcg@10: {ideal:.3f}")print(f"ndcg@10: {dcg / ideal:.3f}")print(f"ndcg@10 from lib.metrics: {ndcg_at_k(ranking, judgments, 10):.3f}")
輸出python ndcg-for-one-query.py
grades: [1, 2, 2, 2, 2, 1, 1, 0, 0, 0]
best grades: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2]
dcg@10: 5.586
ideal dcg@10: 9.087
ndcg@10: 0.615
ndcg@10 from lib.metrics: 0.615

The ranking earns a DCG of 5.586. The best possible top 10 holds only exact products and earns 9.087. Their ratio, 0.615, matches what ndcg_at_k in lib.metrics returns: 0.615. The partial product at position 1 costs the most of any single result, because it holds the place with no discount. Each hook costs less, because it sits lower, but together the hooks cost more.

That one number sums up what the judgments showed: a ranking that found the right kind of product and made two kinds of mistake. You can compute it for any ranking of any judged query, and compare.

Working rule: Use nDCG@10 as your headline metric when your judgments are graded and the first page of results matters most, as it does in a store's search. When you have only yes-or-no labels, or when shoppers look at one result only, another metric fits better. Chapter 3 builds the alternatives and says when to choose each.

為每個測試查詢評分

最近一次變動於第 2026.11.10 版

單一查詢只是軼事。要評判一個排序器,應在一組固定的查詢上評分,再取平均:在本書中,這組查詢就是測試查詢。一組查詢中,每個查詢的排名合起來稱為一個執行結果(run),寫成 {query_id: [product_id, ...]},最佳者在前。lib.metrics.evaluate 會依據判斷(judgments)為執行結果評分,並為每個查詢回傳一列。

除了 nDCG@10 之外,evaluate 也會回報 judged@10:前 10 名中,對該查詢有任何判斷的產品所占的比例。由於未經判斷的產品會視為等級 0,judged@10 偏低就表示分數有一部分是猜測。它無法分辨一個產品是真的不相關,還是相關但沒有人判斷過。

現在,對每個測試查詢執行 BM25,為每個查詢保留前 100 名產品,並為這份執行結果評分:

bm25-test-scores.py Python 3.12.15 · bm25s 0.3.16 · numpy 2.5.5 · pandas 3.1.1 · scipy 1.18.1 通過
"""Scores BM25 on every test query, keeping each query's top 100, and prints the averages and the spread. The prose's point rests on the mean and judged@10 lines and on the three lowest queries printed after them."""from lib.lexical import LexicalSearchfrom lib.metrics import evaluatefrom lib.store import load_products, load_queries, qrelsLABELS = {2: "exact", 1: "partial", 0: "irrelevant"}products, queries, judgments = load_products(), load_queries("test"), qrels("test")run = LexicalSearch(products).run(queries, k=100)       # {query_id: [product_id, ...]}scores = evaluate(run, judgments)                        # one row per test queryscores["query"] = queries["query"]print("test queries:", len(scores))print(f"mean ndcg@10: {scores['ndcg@10'].mean():.3f}")print(f"mean judged@10: {scores['judged@10'].mean():.3f}")print(f"lowest ndcg@10: {scores['ndcg@10'].min():.3f}")print(f"median ndcg@10: {scores['ndcg@10'].median():.3f}")print(f"highest ndcg@10: {scores['ndcg@10'].max():.3f}")print("queries with ndcg@10 of 1:", (scores["ndcg@10"].round(3) == 1).sum())print()lowest = scores.nsmallest(3, "ndcg@10")print(lowest[["query", "ndcg@10", "judged@10"]].round(3).to_string())print("lowest query:", lowest["query"].iloc[0])print(f"judged@10 of the lowest query: {lowest['judged@10'].iloc[0]:.3f}")print()# query 133, outdoor light fixtures, is the second-lowest scorer in the table aboveprint(f"judged@10 for {queries.at[133, 'query']}: {scores.at[133, 'judged@10']:.3f}")print("bm25 top 5 for", queries.at[133, "query"])for product_id in run[133][:5]:    label = LABELS.get(judgments[133].get(product_id), "unjudged")    print(f"  {label:<10}  {products.at[product_id, 'name'][:60]}")
輸出python bm25-test-scores.py
test queries: 30
mean ndcg@10: 0.725
mean judged@10: 0.927
lowest ndcg@10: 0.212
median ndcg@10: 0.687
highest ndcg@10: 1.000
queries with ndcg@10 of 1: 4

                            query  ndcg@10  judged@10
query_id
46           beach blue headboard    0.212        0.5
133        outdoor light fixtures    0.286        0.2
292       decorative white pillow    0.467        1.0
lowest query: beach blue headboard
judged@10 of the lowest query: 0.500

judged@10 for outdoor light fixtures: 0.200
bm25 top 5 for outdoor light fixtures
  exact       malikah 1 -bulb outdoor bulkhead light
  unjudged    60 watt equivalent e12 led candle light bulb
  unjudged    5w ( 40 watt equivalent ) , b11 led , dimmable light bulb e1
  unjudged    60 watt equivalent , a19 led , non-dimmable light bulb , war
  unjudged    40 watt equivalent e12 dimmable led edison candle light bulb

在 30 個測試查詢上,BM25 的平均 nDCG@10 為 0.725,平均 judged@10 為 0.927。這個平均值是商店第一個量測到的結果,之後每個排序器都要與它比較。

平均值掩蓋了很大的差異。分數從 0.212 到 1.000,中位數為 0.687,並有 4 個查詢拿到滿分 1。

得分最低的查詢 beach blue headboard,其 judged@10 為 0.500。另一個低分查詢 outdoor light fixtures,其 judged@10 為 0.200:BM25 的前 5 名中,有一個完全符合的戶外燈具,接著是與查詢共用字詞的燈泡,而這些燈泡沒有人針對此查詢做過判斷。燈泡不是燈具,因此在這裡等級 0 大概是合理的。但指標無法知道這一點,換成另一個排序器或另一個查詢,未經判斷的產品也可能是好的結果。

實用原則: 回報測試查詢的平均 nDCG@10 時,要在旁邊附上平均 judged@10,並且每次為執行結果評分時,都要檢視得分最低的查詢。當某個排序器的 judged@10 低於另一個時,先查明原因,再比較兩者的 nDCG:差距可能來自缺少判斷,而不是結果較差。

此部分尚未翻譯,因此以英文呈現。

Search by meaning with embeddings

最近一次變動於第 2026.11.10 版

Lexical retrieval fails when the shopper's words and the product's words differ, or when shared words mean different things, as with the hooks that BM25 ranks for clear shower curtain. Semantic retrieval compares the meanings of the query and the product instead of their words. It rests on embeddings. An embedding is a list of numbers, a vector, that a neural network computes for a piece of text. The network is trained so that texts with similar meanings get vectors pointing in similar directions. You embed every product once, ahead of time. At search time, you embed the query and return the products whose vectors are closest to it. With every vector scaled to length 1, closeness is the dot product of two vectors, which equals the cosine of the angle between them.

Embedding the products ahead of time is what makes this fast. In the paper that introduced Sentence-BERT, finding the most similar pair among 10,000 sentences took about 65 hours on a V100 GPU when BERT read every pair. Embedding each sentence once and comparing the embeddings took about 5 seconds, almost all of it spent computing the embeddings: the comparison itself took about 0.01 seconds (Reimers and Gurevych, 2019).

The store's embeddings come from the open model Snowflake/snowflake-arctic-embed-m-v1.5. Examples never call a model, so that every example runs offline and gives the same output each time. Instead, a recording, here the script recordings/embeddings.py, runs the model once and saves its vectors, and the examples read them offline through lib.recorded. We chose this model because it is open and under the Apache 2.0 licence, so anyone can download the pinned version and rerun the recording. The recording adds to each query the instruction that the model's card asks for ("Represent this sentence for searching relevant passages:") and embeds products as they are (model card). The book's semantic retrieval, built as lib.semantic.SemanticSearch, multiplies every product vector by the query vector and sorts the results. Chapter 6 builds it step by step.

Every product is a vector of 768 numbers, and semantic retrieval compares the query's vector with each one; here are the ten closest, with their judgments:

On this query, semantic retrieval does better than BM25. Its top 10 holds 5 exact products, against BM25's 4, and 0 irrelevant ones. Its nDCG@10 is 0.793, against BM25's 0.615. Notice how close the similarities are: the first and the tenth differ by 0.035. Semantic retrieval always returns something, ranked by small differences.

One query can mislead, so compare the two rankers on every test query:

compare-lexical-semantic.py Python 3.12.15 · bm25s 0.3.16 · numpy 2.5.5 · pandas 3.1.1 · scipy 1.18.1 通過
"""Scores BM25 and semantic retrieval on every test query and counts the queries each one is better on. The prose's point that the test set is too small to show a pattern rests on the `better` and `size in inches` lines."""from lib.lexical import LexicalSearchfrom lib.metrics import evaluatefrom lib.semantic import SemanticSearchfrom lib.store import load_products, load_queries, qrelsproducts, queries, judgments = load_products(), load_queries("test"), qrels("test")bm25 = evaluate(LexicalSearch(products).run(queries, k=100), judgments)semantic = evaluate(SemanticSearch().run(queries, k=100), judgments)for name, scores in [("bm25", bm25), ("semantic", semantic)]:    print(f"{name} ndcg@10: {scores['ndcg@10'].mean():.3f}")    print(f"{name} judged@10: {scores['judged@10'].mean():.3f}")# per test query, rounded to three decimals: queries that differ only beyond the third place count as tieddifference = (semantic["ndcg@10"] - bm25["ndcg@10"]).round(3)print("semantic better:", (difference > 0).sum())print("bm25 better:", (difference < 0).sum())print("tied:", (difference == 0).sum())sized = queries["query"].str.contains(r"\d+\s*(?:inch|in\b|\"|'')")             # a size in inchesprint("test queries with a size in inches:", sized.sum())print("of those, bm25 better:", (difference[sized] < 0).sum())print()print("queries where bm25 is better:")print(queries.loc[difference[difference < 0].index, "query"].to_string())
輸出python compare-lexical-semantic.py
bm25 ndcg@10: 0.725
bm25 judged@10: 0.927
semantic ndcg@10: 0.830
semantic judged@10: 0.990
semantic better: 21
bm25 better: 7
tied: 2
test queries with a size in inches: 2
of those, bm25 better: 2

queries where bm25 is better:
query_id
63                                     monthly calendar
72                                   fold in half table
94             kohler whitehaven farmhouse kitchen sink
285    48" sliding single track , barn door for laundry
292                             decorative white pillow
293                            wood floor protector mat
482                                48 inch kitchen hood

Averaged over the test queries, semantic retrieval is ahead: an nDCG@10 of 0.830 against BM25's 0.725. Its judged@10, 0.990, is also higher than BM25's 0.927, so part of BM25's deficit could come from products nobody judged. For outdoor light fixtures, at least, those products are light bulbs, and a score of 0 is fair.

Semantic retrieval is better on 21 queries, BM25 on 7, and 2 are tied. The store has only 30 test queries, so chapter 3 tests whether a gap like this one is larger than chance.

BM25 is better on 2 of the 2 test queries that give a size in inches. Exact tokens such as sizes and model numbers may favour matching words, but a count this small can't show a pattern. On the train queries that contain a digit, BM25 is better on 1 of 5 (Find the queries each retriever wins), and the embedding model matches digits too, as text (Check queries that contain digits).

A theoretical result shows where single vectors fall short. Weller et al. prove that an embedding's dimension limits which combinations of top results a single vector can return. On LIMIT, a dataset they built to test that limit, BM25 finds 93.6% of the relevant items in its top 100, and the best single-vector model 18.9%. On a variant of LIMIT with words replaced by synonyms, BM25's score drops by more than 89% (Weller et al., 2025). Lexical and semantic retrieval fail in different places.

Working rule: Add semantic retrieval when shoppers describe what they want in their own words. Don't assume lexical retrieval wins on exact terms such as sizes, brands and model numbers: before you rely on it there, count which retriever wins the train queries that contain them. On this data, semantic retrieval alone scores highest so far, so it is the ranker to beat.

以倒數排名融合法融合兩個排名

最近一次變動於第 2026.11.10 版

由於詞彙檢索與語意檢索會在不同的地方失敗,混合檢索(hybrid retrieval)會結合兩者各自產生的排名,此處指 BM25 與語意檢索的排名,期望同時保有兩者的優點。兩者的分數無法直接相加,因為它們的尺度不同(第 7 章會說明原因)。倒數排名融合法(reciprocal rank fusion,RRF)只使用排名,藉此避開這個問題。對每個包含某商品的排名,該商品得到的分數是 1 除以「一個常數與其排名之和」,再把各排名的分數加總:

RRF(product) = sum over rankings r of  1 / (k + rank of product in r),   with k = 60

常數 k 會讓頂端的差距變得平緩:當 k = 60 時,第 1 名得到 1/61,第 10 名得到 1/70。因此,在兩個排名中都接近前段的商品,會勝過只在其中一個排名拿到第一名的商品。作者在前導研究中將 k 固定為 60,並發現「這個選擇並不關鍵」(Cormack et al., 2009)。他們融合了兩類執行結果:提交給 TREC(Text REtrieval Conference 的共同評測)的結果,以及單一搜尋系統的 30 種設定。在這些實驗中,RRF 以平均精確率均值(mean average precision,MAP)衡量,平均勝過最佳的單一執行結果 4% 至 5%;MAP 是對各查詢取平均,而每個查詢的值則是各相關結果所在位置的精確率之平均。這與融合一個詞彙排名和一個語意排名的情形不同,而後來一項針對混合搜尋的研究發現,經過調校、對正規化分數做加權和的方法勝過 RRF(Bruch et al., 2023)。第 7 章會在訓練查詢上調校這兩種方法。此處的 RRF 不經調校。

RRF 簡短到可以手寫,只需兩個函式:rrf_scores 為單一查詢的商品計分,reciprocal_rank_fusion 則為每個查詢排序商品,分數相同的商品依 product_id 的順序排列,以免某個排名因為先被傳入而在平手時佔優勢。hybrid-rrf 範例會融合 BM25 與語意檢索對每個測試查詢的前 100 名,檢查 clear shower curtain 排在最前面的兩個商品的分數,並為混合執行結果評分。

BM25 的分數為 0.725,語意檢索為 0.830,所以較弱的排序器得到同樣的一票。它的一票會拖累融合結果,還是兩者能彌補彼此的遺漏,進而勝過語意檢索?在閱讀輸出之前,請先選定三種答案之一:高於語意檢索、介於兩者之間,或低於 BM25。

hybrid-rrf.py Python 3.12.15 · bm25s 0.3.16 · numpy 2.5.5 · pandas 3.1.1 · scipy 1.18.1 通過
"""Fuses BM25 and semantic retrieval with reciprocal rank fusion, by hand for one query and then for every test query. The prose's point rests on the hybrid's `ndcg@10` line and on the two scores printed for the first product, which agree."""from lib.lexical import LexicalSearchfrom lib.metrics import evaluatefrom lib.semantic import SemanticSearchfrom lib.store import load_products, load_queries, qrelsdef rrf_scores(rankings, k=60):    """Each product scores the sum of 1 / (k + rank) over the rankings that contain it."""    scores = {}    for ranking in rankings:        for rank, product_id in enumerate(ranking, start=1):            scores[product_id] = scores.get(product_id, 0) + 1 / (k + rank)    return scoresdef reciprocal_rank_fusion(runs, k=60, depth=100):    """For each query, products in order of their RRF scores, best first; equal scores by product_id."""    fused = {}    for query_id in runs[0]:        scores = rrf_scores([run[query_id] for run in runs], k)        # equal scores fall back to product_id, so neither ranking wins ties by being passed first        fused[query_id] = sorted(scores, key=lambda product_id: (-scores[product_id], product_id))[:depth]    return fusedproducts, queries, judgments = load_products(), load_queries("test"), qrels("test")bm25, semantic = LexicalSearch(products).run(queries, k=100), SemanticSearch().run(queries, k=100)hybrid = reciprocal_rank_fusion([bm25, semantic])for place, product_id in zip(["first", "second"], hybrid[246]):       # query 246, "clear shower curtain"    # list.index counts from 0, and a rank counts from 1    bm25_rank, semantic_rank = bm25[246].index(product_id) + 1, semantic[246].index(product_id) + 1    print(f"{place} product: {products.at[product_id, 'name']}")    print(f"{place} product's bm25 rank: {bm25_rank}")    print(f"{place} product's semantic rank: {semantic_rank}")    # the formula with the two ranks in it, where k = 60    print(f"{place} product's score by hand: {1 / (60 + bm25_rank) + 1 / (60 + semantic_rank):.5f}")    print(f"{place} product's score from rrf_scores: {rrf_scores([bm25[246], semantic[246]])[product_id]:.5f}")hybrid_scores, semantic_scores = evaluate(hybrid, judgments), evaluate(semantic, judgments)print(f"hybrid ndcg@10: {hybrid_scores['ndcg@10'].mean():.3f}")print(f"hybrid judged@10: {hybrid_scores['judged@10'].mean():.3f}")difference = (hybrid_scores["ndcg@10"] - semantic_scores["ndcg@10"]).round(3)print("better than semantic:", (difference > 0).sum())print("worse than semantic:", (difference < 0).sum())print("tied with semantic:", (difference == 0).sum())
輸出python hybrid-rrf.py
first product: woolley vinyl solid color single shower curtain
first product's bm25 rank: 3
first product's semantic rank: 1
first product's score by hand: 0.03227
first product's score from rrf_scores: 0.03227
second product: woolley vinyl single shower curtain
second product's bm25 rank: 2
second product's semantic rank: 2
second product's score by hand: 0.03226
second product's score from rrf_scores: 0.03226
hybrid ndcg@10: 0.812
hybrid judged@10: 0.980
better than semantic: 9
worse than semantic: 14
tied with semantic: 7

融合後的第一個商品 woolley vinyl solid color single shower curtain,在 BM25 的排名中是第 3 名,在語意檢索的排名中是第 1 名。它的分數由兩個排名各貢獻一項相加,即 1/(60 + 3) + 1/(60 + 1),手算的結果為 0.03227,與 rrf_scores 給出的相同。

第二個商品 woolley vinyl single shower curtain,在 BM25 的排名中是第 2 名,在語意檢索的排名中是第 2 名,分數為 0.03226,因此排在第二。

混合執行結果的分數為 0.812:高於 BM25,低於單獨使用語意檢索。逐一查詢來看,它在 9 個查詢上勝過語意檢索,在 14 個查詢上落後,並在 7 個查詢上持平。它的 judged@10 為 0.980。

因此,融合在這裡並未優於語意檢索,但這並不等於讓搜尋變差。為何沒有幫助,是第 7 章要探討的問題;該章會在融合之前檢查每個檢索器,並在訓練查詢上調校混合比例。

如果你選的是高於語意檢索,這裡的教訓是:結合兩個排名並不保證有所提升。融合和其他任何改動一樣,必須加以衡量。

實用原則: 在融合之前,先單獨衡量每個檢索器。以 k = 60 的 RRF 作為未經調校的預設做法,只有當融合後的排名在保留的查詢上勝過最佳的單一檢索器時,才保留它。當某個檢索器明顯較弱時,請試著對排名加權,或在訓練查詢上調校 k,而不是接受等權的投票。第 7 章會在商店的資料上嘗試這兩種做法,而它在測試查詢上回報的調校後混合結果,沒有一個勝過語意檢索(在測試查詢上回報調校後的混合結果)。

以交叉編碼器重排前 50 名

最近一次變動於第 2026.11.10 版

到目前為止的每個階段都屬於檢索。檢索必須針對每個查詢為目錄中的每項商品評分,因此成本必須低。重排(reranking)則是用較慢的模型,重新排列檢索所得的短清單。由於模型只需處理幾十項商品,所以負擔得起同時檢視查詢與每項商品。

重排模型的一種是交叉編碼器(cross-encoder)。它把查詢與商品文字合為一個輸入一併讀取,並為這組配對輸出一個相關性分數,每組配對各自獨立評分(Nogueira and Cho, 2019)。嵌入模型只讀取商品本身,因此向量可以預先算好。交叉編碼器需要查詢才能運算,所以無法預先計算任何東西,對每組配對都執行一遍也會耗費時間:Sentence-BERT 比較中較慢的一方,就是 BERT 以交叉編碼器的方式讀取每一組句子配對。因此,交叉編碼器重排的是由較便宜的第一階段檢索出的短清單(Nogueira and Cho,第 2 節;Retrieve & Re-Rank)。換得的好處是,它能透過注意力(attention)將查詢的每個字與商品的每個字相互關聯;注意力是模型中讓輸入裡每個字都能參照其他字的部分。Reimers and Gurevych(2019,第 4.3 節)指出,BERT 能利用注意力直接比較兩個句子。2019 年,在 MS MARCO 段落基準上,以這種方式重排 BM25 的前 1,000 個段落,使 MRR@10(衡量第一個相關段落排得多前面的指標)比先前最佳結果相對提升 27%(Nogueira and Cho)。

交叉編碼器是語言模型,但不是大型語言模型(large language model,LLM)。LLM 是以大量文字訓練而成、能依提示產生文字的模型。後續各章會使用 LLM 來評判、改寫與重排。交叉編碼器不產生任何文字,它只輸出一個數字。

本店的交叉編碼器是開放模型 Alibaba-NLP/gte-reranker-modernbert-base,選用的理由與嵌入模型相同:它是開放的,採用 Apache 2.0 授權,任何人都能重新執行這份錄製檔。它的錄製檔 recordings/cross_encoder.py 將模型的分數一次儲存下來,涵蓋每個查詢在 BM25 與語意檢索各自前 100 名的聯集中的每項商品,因此混合結果前 50 名中的每項商品都有分數。重排步驟依這些分數重新排列混合執行結果(以 RRF 融合 BM25 與語意檢索)的前 50 名,其餘維持原位,再對結果評分,本書稱之為重排後的混合結果:

cross-encoder-rerank.py Python 3.12.15 · bm25s 0.3.16 · numpy 2.5.5 · pandas 3.1.1 · scipy 1.18.1 通過
"""Reorders the top 50 of the hybrid run by cross-encoder score, keeps the rest in place, and scores the result. The prose's point rests on the reranked nDCG@10 line, set beside the hybrid's score, and on the better and worse counts."""from lib.fusion import reciprocal_rank_fusionfrom lib.lexical import LexicalSearchfrom lib.metrics import evaluatefrom lib.recorded import cross_encoder_scores, manifestfrom lib.semantic import SemanticSearchfrom lib.store import load_products, load_queries, qrelsproducts, queries, judgments = load_products(), load_queries("test"), qrels("test")bm25 = LexicalSearch(products).run(queries, k=100)semantic = SemanticSearch().run(queries, k=100)hybrid = reciprocal_rank_fusion([bm25, semantic])        # the same function as in the hybrid sectionrecorded = cross_encoder_scores()                        # one score per (query, product) pairscore = {(query_id, product_id): value for query_id, product_id, value in recorded.itertuples(index=False)}print("cross-encoder model:", manifest("cross-encoder")["model"])depth = 50                                               # rerank the top 50reranked = {}for query_id, ranking in hybrid.items():    # the top 50 are reordered by score; equal scores keep the hybrid's order    top = sorted(ranking[:depth], key=lambda product_id: score[(query_id, product_id)], reverse=True)    # products below the top 50 keep their hybrid order    reranked[query_id] = top + ranking[depth:]before, after = evaluate(hybrid, judgments), evaluate(reranked, judgments)print(f"reranked ndcg@10: {after['ndcg@10'].mean():.3f}")print(f"reranked judged@10: {after['judged@10'].mean():.3f}")difference = (after["ndcg@10"] - before["ndcg@10"]).round(3)print("better than hybrid:", (difference > 0).sum())print("worse than hybrid:", (difference < 0).sum())
輸出python cross-encoder-rerank.py
cross-encoder model: Alibaba-NLP/gte-reranker-modernbert-base
reranked ndcg@10: 0.798
reranked judged@10: 0.980
better than hybrid: 11
worse than hybrid: 12

在這裡,重排器沒有幫助。它的得分為 0.798,而它所依據的混合結果為 0.812。它改善了 11 個查詢,並使 12 個查詢變差。以下是本章在測試查詢上的整體結果:

排序器 nDCG@10 judged@10
BM25 0.725 0.927
語意 0.830 0.990
混合(RRF,k = 60) 0.812 0.980
重排後的混合(前 50 名) 0.798 0.980

它使結果變差的其中一個查詢 refrigerator with ice an water in door,顯示了重排器如何失分:

rerank-one-query.py Python 3.12.15 · bm25s 0.3.16 · numpy 2.5.5 · pandas 3.1.1 · scipy 1.18.1 通過
"""Shows the hybrid and reranked top 5 for one query whose nDCG@10 the reranker lowered, with each product's cross-encoder score and judgment. The prose's point rests on the two rank lines and the two nDCG@10 lines."""import refrom lib.fusion import reciprocal_rank_fusionfrom lib.lexical import LexicalSearchfrom lib.metrics import ndcg_at_kfrom lib.recorded import cross_encoder_scoresfrom lib.semantic import SemanticSearchfrom lib.store import load_products, load_queries, product_text, qrelsLABELS = {2: "exact", 1: "partial", 0: "irrelevant"}products = load_products()query_id = 406query = load_queries().at[query_id, "query"]judgments = qrels()[query_id]recorded = cross_encoder_scores()score = recorded[recorded["query_id"] == query_id].set_index("product_id")["score"]bm25 = {query_id: list(LexicalSearch(products).search(query, k=100).index)}semantic = {query_id: list(SemanticSearch().search(query_id, k=100).index)}hybrid = reciprocal_rank_fusion([bm25, semantic])[query_id]# the top 50 are reordered by score; the rest stay in the hybrid's orderreranked = sorted(hybrid[:50], key=score.get, reverse=True) + hybrid[50:]print("query:", query)for name, ranking in [("hybrid", hybrid), ("reranked", reranked)]:    first_exact = next(rank for rank, product_id in enumerate(ranking, start=1) if judgments.get(product_id) == 2)    print()    print(f"{name} ndcg@10: {ndcg_at_k(ranking, judgments, 10):.3f}")    print(f"{name} rank of the first exact product: {first_exact}")    for product_id in ranking[:5]:        label = LABELS.get(judgments.get(product_id), "unjudged")        print(f"  {score[product_id]:.2f}  {label:<10}  {products.at[product_id, 'name']}")exact = next(product_id for product_id in hybrid if judgments.get(product_id) == 2)words = set(re.findall(r"\w+", product_text(products).at[exact].lower()))   # name, class, category, descriptionprint()print("hybrid's first exact product:", products.at[exact, "name"])print("its text says ice or water:", "yes" if {"ice", "water"} & words else "no")
輸出python rerank-one-query.py
query: refrigerator with ice an water in door

hybrid ndcg@10: 0.660
hybrid rank of the first exact product: 3
  1.95  partial     apec water icemaker-kit-38-14-ro refrigerator/icemaker installation accessory
  2.06  partial     36 '' french door 26.1 cu . ft. smart energy star refrigerator with interior ice maker
  1.44  exact       36 '' french door refrigerator 27 cu . ft. energy star refrigerator
  2.19  partial     36 '' energy star counter-depth french door 22.6 cu . ft. refrigerator with ice maker
  2.19  partial     800 series 21 cu . ft. smart energy star counter depth french door refrigerator with internal water dispenser

reranked ndcg@10: 0.533
reranked rank of the first exact product: 9
  2.20  partial     smeg classic 36 '' french door 22 cu . ft. refrigerator with ice maker
  2.19  partial     36 '' energy star counter-depth french door 22.6 cu . ft. refrigerator with ice maker
  2.19  partial     800 series 21 cu . ft. smart energy star counter depth french door refrigerator with internal water dispenser
  2.10  partial     28 '' energy star counter depth top freezer 12.56 cu . ft. refrigerator with auto ice maker
  2.06  partial     36 '' french door 26.1 cu . ft. smart energy star refrigerator with interior ice maker

hybrid's first exact product: 36 '' french door refrigerator 27 cu . ft. energy star refrigerator
its text says ice or water: no

交叉編碼器把第一項完全符合的商品埋沒了,將它從第 3 名移到第 9 名,使這個查詢的 nDCG@10 從 0.660 降到 0.533。重排後的前 5 名是名稱中寫有 ice maker 或 water dispenser 的冰箱,評審將它們標為部分相符:它們符合所要找的東西,也就是冰箱,卻不符合修飾條件,即門上的冰與水。混合結果中第一項完全符合的商品是一台法式對開門冰箱,在混合結果前 5 名中,它得到最低的交叉編碼器分數。

評審將那台冰箱標為完全符合,因此依資料集的定義,它完全符合該查詢。然而它的文字完全沒有提到冰或水,所以任何只讀文字的模型都無法依這項修飾條件為它排序。WANDS 的評審除了商品名稱,也看得到每項商品的圖片與網頁(標註準則),這大概讓他們看得出文字沒有說明的內容。通用的重排器從未見過這家店的評判。第 9 章會以訓練查詢的評判來訓練排序器。

實用原則: 只重排一份短清單(此處為 50 項商品),因為成本會隨每組配對增加。只有在重排器於保留的查詢上勝過最佳的單獨檢索時,才採用它。這個重排器沒有勝過混合結果,也沒有勝過語意檢索,所以本店目前為止最好的流程是單獨使用語意檢索。在放棄重排之前,可以試試不同的深度,或以自己的評判訓練的排序器。在第 9 章的測試查詢上,不同的深度與 LambdaMART 都沒有勝過語意檢索(依預算設定重排深度、在測試查詢上報告 LambdaMART)。

此部分尚未翻譯,因此以英文呈現。

What goes wrong in production

最近一次變動於第 2026.10.7 版

The tokenizer drops words without telling you. bm25s's default tokenizer keeps only tokens of two or more characters and doesn't stem words (bm25s tokenizer). Run it on a train query and a test query, and count the products it can't match:

tokenizer-drops-terms.py Python 3.12.15 · bm25s 0.3.16 · numpy 2.5.5 · pandas 3.1.1 · scipy 1.18.1 通過
"""Tokenizes one train query, one test query and the products' text with bm25s, and counts the products that have `chairs` or only `chair`. The prose's points rest on the printed token lists and on the two product counts."""import bm25sfrom lib.store import load_products, product_textdef tokenize(texts):    """The tokenizer lib.lexical uses for products and queries, with bm25s's default settings."""    # stopwords="en" removes English words such as "and"; no stemmer is set, so "chairs" and "chair" stay different tokens    return bm25s.tokenize(texts, stopwords="en", return_ids=False, show_progress=False)for query in ["ge top loading washer 4.5", "bistro table and chairs"]:    print(f"{query!r} -> {tokenize([query])[0]}")products = tokenize(product_text(load_products()).tolist())print()print("products with the token chairs:", sum("chairs" in tokens for tokens in products))print("products with chair but not chairs:", sum("chair" in tokens and "chairs" not in tokens for tokens in products))
輸出python tokenizer-drops-terms.py
'ge top loading washer 4.5' -> ['ge', 'top', 'loading', 'washer']
'bistro table and chairs' -> ['bistro', 'table', 'chairs']

products with the token chairs: 895
products with chair but not chairs: 359

The 4.5 in ge top loading washer 4.5 disappears, because 4 and 5 are one character each, so that number plays no part in the ranking. And chairs matches only products that say chairs: 895 products do, while 359 say chair and never chairs, and none of those can match that word of bistro table and chairs. Neither case raises an error. Chapter 5 shows both problems and adds stemming.

Unjudged results look like failures. A ranker that surfaces products nobody judged gets lower scores than it may deserve, and a ranker that surfaces the judged ones gets an advantage. Report judged@10 with every score, and have new products judged when it drops.

Scores from a sample are probably optimistic. The sample keeps 12,000 products, nearly all judged for some query. The full WANDS catalog has 42,994 (repository), and a store's catalog may be far larger, with more near misses competing for the top 10. Nothing in this chapter measures that, but expect lower scores on a full catalog. Don't compare the scores here with the WANDS paper's either: it ran the BM25 of Solr, a search server, over product titles and descriptions (Chen et al.).

Model outputs belong to one model version. Product and query embeddings must come from the same model at the same version, or their similarities aren't comparable. Change the model and you re-embed the whole catalog. Each folder in examples/data/recorded/ has a manifest.json that pins the model and its version.

Reranking costs time on every query. Product embeddings are computed once. A cross-encoder runs on every pair, for every query, while the shopper waits. In this chapter, a recording script computes the scores ahead of time on a GPU, which hides that cost. Chapter 9 counts what each rerank depth costs (Set the rerank depth from a budget).

Working rule: Check the tokens your queries become, track judged@10, and record the model version next to every vector you store.

本章做出的決定

最近一次變動於第 2026.11.8 版

這些預設值會一直沿用,直到後續章節以訓練查詢測試其中某一項為止。

  • 在測試集上報告,在訓練集上調校。 本章所有分數都是在 30 筆測試查詢上計算。本章不調校任何項目;後續章節只會在 50 筆訓練查詢上調校。
  • 主要指標:nDCG@10,並以 judged@10 作為對照。 評分等級當作增益,未經評判的商品視為等級 0;當這個假設影響重大時,judged@10 會顯示出來。
  • 基準線:採用 bm25s 預設值的 BM25, 即 k1=1.5 與 b=0.75,檢索範圍涵蓋每項商品的名稱、類別、分類與描述。調校要等到能以訓練查詢衡量時再進行。
  • 語意檢索使用事先記錄的嵌入向量, 來自單一開放模型,以單位長度向量的點積進行比較,因此每個範例都能離線執行,且每次輸出相同。模型之間的選擇是後續的問題。
  • 混合檢索:k = 60 且權重相等的 RRF,未經調校, 與其作者的用法相同。
  • 重新排序深度:混合檢索的前 50 筆, 由事先記錄的交叉編碼器評分,因此每項經過重新排序的商品都有一個已記錄的分數。在實際營運的商店中,深度既是品質問題,也是成本決策。
  • 要超越的排序器是單獨的語意檢索, 因為在這份資料上,融合與這個重新排序器都沒有勝過它。第 3 章會在評估框架中檢驗這類差距是否大於隨機誤差。
  • 本章所顯示的: 理應有幫助的階段,無論是融合或重新排序,在這份資料上都沒有勝過語意檢索,因此每一項變更在保留之前都要先經過衡量。

本章,於本版次中

版次 2026.11.10範例於 2026年11 月 19 通過測試本章 11 個範例 個範例中有 11 個通過

譯自英文書,版次 2026.11.10。

引用本節

格式

各種格式皆引用英文原書,本譯本即依循該原書。

每種格式都會標示本版。其連結會開啟目前版次,節標題會說明該節內文最後一次變動的時間。存取日期與擷取日期即為你開啟此頁的當日。

本頁內容

第 1 章,共 11 章: 搜尋的完整流程

  1. Load the store's data
  2. 以 BM25 對商品排序
  3. 對照判斷檢查結果
  4. Score one ranking with nDCG@10
  5. 為每個測試查詢評分
  6. Search by meaning with embeddings
  7. 以倒數排名融合法融合兩個排名
  8. 以交叉編碼器重排前 50 名
  9. What goes wrong in production
  10. 本章做出的決定

返回頂端所有章節