注意力
阅读约需 51分钟 分钟,10节中含8个示例。
学完本章,你将能够手工计算注意力:为每个词元的查询(query)与之前每个词元的键(key)的匹配程度打分,把分数转换为权重,再对这些权重所选出的值(value)取平均。你会加入因果掩码,使语言模型无法看到未来;加入缩放,使权重不会塌缩到某一个词元上;还会加入多个注意力头,让它们同时寻找不同的内容。随后,你将解读训练好的模型关注了什么,检验它复制了什么,并测量它能回看多远。
第 5 章的窗口模型根据前面恰好四个词元预测商店名称和描述中的下一个词元,这四个词元并排连接,每个位置都有各自的权重。这种设计有两个局限。一是它看不到往前数第五个词元,无论那个词元多么重要;二是它对第一个位置上的某个词元的学习,与对第三个位置上的同一个词元的学习是分开的。这家商店在线销售家具和家居用品,它的描述说明了第一个局限为何有害:描述常常会在几十个词元之后重复产品名称中的词,例如protect your outdoor patio furniture from harsh elements with this outdoor patio furniture cover。能够回看名称的模型,可以看出下一个词是什么;只能看到最后四个词元this outdoor patio furniture的模型,则只能在其他产品中跟在这些词之后的所有内容里猜测。
注意力(attention)让每个位置都能查看之前的每一个位置,并根据它们的内容决定各自的重要程度。它最初是为了让翻译模型能够回看它正在翻译的句子中的词而提出的(Bahdanau et al., 2015)。它是 transformer 的核心(Vaswani et al., 2017),也是本书在第 10 章中解读的开源模型所采用的设计。本章先单独构建注意力,再在本书的小型 transformer 中解读它。本书把这个模型称为商店的模型:它是在商店的文本上一次性训练出来的 transformer,由第 7 章逐部分组装。
这部分尚未翻译,因此以英文显示。
Look back with a weighted average
Attention replaces chapter 5's fixed slots with a weighted average. At each position, the model computes a new vector as an average of vectors from the positions it can see, with weights it chooses from their content: high for the positions that matter here, low for the rest. The weights come from three vectors that each token's embedding is turned into, by three learned matrices:
- a query, a vector for what this position is looking for,
- a key, a vector for what this position offers to the positions that look at it,
- a value, a vector for what this position passes on when it's looked at.
The score of position j for position i is the dot product of i's query with j's key, divided by the square root of d, the size of each query and key, meaning how many numbers it holds. Softmax turns each row of scores into weights that are positive and sum to one, and the output at position i is the weighted sum of the values. In matrix form, for all positions at once:
Attention(Q, K, V) = softmax(QKᵀ / √d) V
where Q, K and V hold every position's query, key and value as rows (Vaswani et al., 2017, section 3.2.1). The division by √d is the subject of a later section.
To watch the arithmetic on something small, take four tokens of the store's text, the start marker and sled coffee table, a table whose base is shaped like a sled's runners. Give each a random vector of 8 numbers, as an untrained embedding table would, and compute attention with random 8-by-4 query, key and value matrices:
1"""Computes one attention layer by hand for four tokens, with random matrices. The prose's point rests on the2printed weight matrix: each row says which positions one position draws from, and how much."""3import numpy as np45tokens = ["<s>", "sled</w>", "coffee</w>", "table</w>"]6rng = np.random.default_rng(2026) # a fixed seed, so the same numbers come out on every run7x = rng.normal(0, 1, (4, 8)) # one 8-number vector per token, as an embedding table gives8w_query, w_key, w_value = (rng.normal(0, 0.5, (8, 4)) for _ in range(3))910queries, keys, values = x @ w_query, x @ w_key, x @ w_value # what each token looks for, offers, and passes on11scores = queries @ keys.T / np.sqrt(4) # how well each query matches each key, scaled12weights = np.exp(scores - scores.max(1, keepdims=True)) # subtracting each row's largest score keeps exp from overflowing13weights /= weights.sum(1, keepdims=True) # softmax along each row: each row sums to 114output = weights @ values # each token: a weighted average of all the values1516np.set_printoptions(precision=2, suppress=True)17print("attention weights (rows attend to columns):")18for token, row in zip(tokens, weights):19 print(f" {token:<11}", row)20assert np.allclose(weights.sum(1), 1) # each row sums to 1, which the prose relies on21print("output shape:", output.shape)
python attention-by-hand.pyattention weights (rows attend to columns):
<s> [0.02 0.97 0.01 0. ]
sled</w> [0.21 0.52 0.16 0.11]
coffee</w> [0.04 0. 0.03 0.94]
table</w> [0.16 0. 0.03 0.81]
output shape: (4, 4)
Each row of the printed weights is one position's average: which positions it draws from, and how much. Every row sums to one, which the example asserts, and the output has one 4-number vector per position. The weights are random, because the matrices are, but they already show attention's freedom: coffee puts most of its weight on table, a token after it, and the start marker puts most of its weight on sled. Nothing in the arithmetic stops a position from looking ahead. For predicting the next token, that freedom is a bug, as the next section shows.
Three properties of this arithmetic matter for everything that follows. The weights depend on content, through queries and keys, so the same token can attend to different places in different texts. The number of positions doesn't matter: the same three matrices work for a name of 4 tokens or a description of 150, so the model learns one way of looking back, not one per slot. And a weighted average ignores the order of what it averages. Without a mask, attention gives bar the same output in bar stool as in stool bar. The causal mask of the next section lets a little order back in, because each token then sees a different number of earlier tokens, and Haviv and colleagues found that language models can infer positions from that alone (Haviv et al., 2022). Models still add explicit position information, as chapter 7 does.
Working rule: Write out attention's four lines once, as this example does, and print the weight matrix: its rows say exactly where each position looks. Keep queries, keys and values as separate matrices while you're learning. Libraries often keep them in one matrix, as PyTorch's MultiheadAttention does with its in_proj_weight (source), and split the result, which computes the same thing.
用因果掩码隐藏未来
在第一节中,coffee的平均值大部分来自后面的词元 table。语言模型不能这样做,因为它是根据前面的词元来预测每个词元的。如果位置 i 可以关注位置 i + 1,它就会看到答案:训练会奖励复制下一个词元,而模型学不到任何在生成文本时能用上的东西,因为生成时下一个词元还不存在。解决办法是因果掩码:在 softmax 之前,把所有指向后面位置的分数设为负无穷。负无穷的指数为零,因此这些位置的权重为零,每个位置只对自身及其之前的位置求平均。
掩码还能提高训练效率。有了它,对一段文本做一次前向计算,就能同时预测每个位置,每个位置只依据自己的过去;而第 5 章的窗口模型,每做一次预测都需要单独的一个窗口。
只有当coffee被允许向后看时,最后一个词元的变化才可能影响coffee。对同样四个词元的分数应用掩码,把table换成另一个向量,然后分别在没有掩码和有掩码的情况下,测量coffee处的输出变动了多少:
1"""Applies the causal mask to the same four tokens, then changes the last token and measures how far the output at2coffee moves, with and without the mask. The prose's point rests on the last printed line: with the mask, that change is zero."""3import numpy as np45tokens = ["<s>", "sled</w>", "coffee</w>", "table</w>"]6rng = np.random.default_rng(2026)7x = rng.normal(0, 1, (4, 8))8w_query, w_key, w_value = (rng.normal(0, 0.5, (8, 4)) for _ in range(3)) # the same draws as attention-by-hand9allowed = np.tril(np.ones((4, 4), dtype=bool)) # position i may look at positions 0 to i only101112def attend(x, mask):13 scores = (x @ w_query) @ (x @ w_key).T / np.sqrt(4)14 if mask:15 scores = np.where(allowed, scores, -np.inf) # a score of minus infinity becomes a weight of 016 weights = np.exp(scores - scores.max(1, keepdims=True))17 weights /= weights.sum(1, keepdims=True)18 return weights, weights @ (x @ w_value)192021weights, _ = attend(x, mask=True)22np.set_printoptions(precision=2, suppress=True)23print("weights with the causal mask:")24for token, row in zip(tokens, weights):25 print(f" {token:<11}", row)26print("weight on later tokens:", float(weights[~allowed].sum())) # ~allowed marks the positions above the diagonal2728changed = x.copy()29changed[3] = rng.normal(0, 1, 8) # a different last token in place of `table`30for mask in [False, True]:31 before, after = attend(x, mask)[1][2], attend(changed, mask)[1][2] # attend returns (weights, output); 2 is coffee32 change = float(np.abs(after - before).max())33 print(f"largest change in coffee's output, {'with' if mask else 'without'} the mask: {change:.2f}")34 assert (change == 0) == mask # the mask's job: with it, nothing after coffee reaches coffee, so the change is zero
python causal-mask.pyweights with the causal mask:
<s> [1. 0. 0. 0.]
sled</w> [0.29 0.71 0. 0. ]
coffee</w> [0.58 0.01 0.41 0. ]
table</w> [0.16 0. 0.03 0.81]
weight on later tokens: 0.0
largest change in coffee's output, without the mask: 4.69
largest change in coffee's output, with the mask: 0.00
权重矩阵现在是下三角矩阵,对角线以上全为零,因此落在后面词元上的总权重为 0.0。起始标记只能关注自身,所以它所在的行只有一个 1。sled把权重分给起始标记和它自己。coffee在加掩码之前几乎把全部权重放在后面的table上,现在则把权重分散到它被允许关注的位置上,因为 softmax 只在剩下的位置之间做除法。最后一行,即table,没有变化,因为它本来就没有后面的位置会被掩掉。
最后两行显示了掩码堵住的泄漏。没有掩码时,改变最后一个词元,会使coffee处的输出变动 4.69。有了掩码,coffee的输出不可能依赖之后出现的内容,示例也断言该变化为零。不加掩码训练的模型可以学会从下一个位置抄取每个答案,因此训练损失可能降到接近零,而它并没有学到任何可用于生成文本的东西。这一过程中不会出现任何崩溃,这正是错误的掩码难以被发现的原因。
工作准则: 把掩码应用在 softmax 之前的分数上,而不是之后的权重上,这样每一行的和仍为一。像本示例这样测试:检查后面位置上的权重恰好为零,并检查改变后面的词元不会改变前面的输出。如果模型的训练损失下降得比预期快得多,首先检查掩码。
缩放分数
注意力公式把每个分数都除以 √d,原因就在点积之中。两个向量的点积,是把对应位置的乘积逐项相加。如果查询和键中的数相互独立且随机,均值为 0、方差为 1,那么分数的方差为 d,因此查询和键的尺寸越大,分数越大(Vaswani et al., 2017,第 3.2.1 节,脚注 4)。softmax 会把分数之间的巨大差距变成极端的权重:一个位置几乎获得全部权重,其余位置几乎为零。Vaswani 等人推测,处于这种状态的模型学习缓慢,因为当某个权重接近 1 时,softmax 的梯度极小。除以 √d 可以把分数的方差拉回 1,与尺寸无关。
商店模型的例子可以让这一点更具体:它的注意力头使用 32 个数的查询和键。下面随机生成 4、32 和 256 三种尺寸的查询和键,把每个查询与 16 个键打分,分别在做除法和不做除法的情况下,记录每一行中最大的权重,并对 1,000 个查询取平均。先猜一猜:尺寸为 256 且不做除法时,一行中最大的权重平均有多大?
1"""Measures the largest softmax weight in each row, at three sizes of query and key, with and without dividing by the2square root of the size. The prose's point rests on the unscaled averages growing with the size, and the scaled ones staying level."""3import numpy as np45rng = np.random.default_rng(2026) # a fixed seed, so the printed averages are the same on every run6for size in [4, 32, 256]:7 # numbers with mean 0 and variance 1, the case the scaling argument assumes; 1,000 queries against 16 keys8 queries, keys = rng.normal(0, 1, (1000, size)), rng.normal(0, 1, (16, size))9 for divide in [False, True]:10 scores = queries @ keys.T / (np.sqrt(size) if divide else 1)11 weights = np.exp(scores - scores.max(1, keepdims=True))12 weights /= weights.sum(1, keepdims=True)13 label = "scaled" if divide else "unscaled"14 print(f"size {size}, {label}, average largest weight in a row: {weights.max(1).mean():.2f}")
python scaling-matters.pysize 4, unscaled, average largest weight in a row: 0.45
size 4, scaled, average largest weight in a row: 0.25
size 32, unscaled, average largest weight in a row: 0.78
size 32, scaled, average largest weight in a row: 0.24
size 256, unscaled, average largest weight in a row: 0.92
size 256, scaled, average largest weight in a row: 0.25
不做除法时,最大权重随尺寸增大而增大:4 个数时为 0.45,32 个数时为 0.78(这正是商店模型中注意力头的尺寸),256 个数时为 0.92。在较大的尺寸下,尚未经过任何训练,每个查询就已经主要关注某一个键。做了除法之后,无论尺寸多大,最大权重都保持在 0.25:权重分散在若干个键上,但并不均匀,因为 16 个相等的权重各约为 0.06。
训练应当从这样的状态出发。每个位置都关注其他若干个位置,梯度因此能传到所有这些位置,训练再在需要集中的地方把权重变得集中。训练好的注意力头仍然可以把大部分权重放在某一个位置上,商店模型的一些注意力头就是如此;缩放只是避免注意力头一开始就处于那种状态。
工作准则: 像 transformer 论文那样,把注意力分数除以查询尺寸的平方根。少数模型(例如 T5)省去这一除法,改为把这个因子折入查询权重的初始化方式中(来源),因此加载模型权重之前,先确认该模型采用哪一种做法。如果你改动了查询和键的计算或初始化方式,请打印未经训练的模型的平均最大权重,并预期它远低于 1。
多个头关注不同的内容
到目前为止,每次注意力计算只为每个位置给出一组权重,而这组权重只能寻找一类内容:比如前一个词元,或者商品的名称。模型通常需要同时关注好几类内容。多头注意力(multi-head attention)并行运行多个注意力计算,每个计算称为一个头(head)。每个头都有各自的查询、键和值矩阵,并且各自只处理向量中的一部分数值:若向量有 8 个数值、共 2 个头,则每个头处理其中 4 个。各头的输出并排拼接在一起,再由另一个可学习的矩阵,即输出投影(output projection),混合起来(Vaswani et al., 2017,第 3.2.2 节)。
拆分数值使总开销与一个头处理全部数值时大致相当,同时让每个头拥有自己的权重模式。商店所用的模型每层有 4 个头,每个头处理 32 个数值。
对四个位置的随机向量运行两个头,并使用因果掩码,然后看看最后一个位置在两个头中是否关注相同的位置:
1"""Runs two heads of causal attention over four positions, each head working in four of the eight numbers, then mixes2the heads with an output projection. The prose's point rests on the last position's weights differing between the heads,3and on the output keeping the input's shape."""4import numpy as np56rng = np.random.default_rng(2026) # a fixed seed, so the printed weights are the same on every run7positions, dims, heads = 4, 8, 28size = dims // heads # each head works in 4 of the 8 dimensions9x = rng.normal(0, 1, (positions, dims))10w_query, w_key, w_value, output_projection = (rng.normal(0, 0.5, (dims, dims)) for _ in range(4))111213def split(m): # (positions, dims) -> (heads, positions, size)14 return m.reshape(positions, heads, size).transpose(1, 0, 2) # head 1 takes the first four numbers of each vector151617queries, keys, values = split(x @ w_query), split(x @ w_key), split(x @ w_value)18scores = queries @ keys.transpose(0, 2, 1) / np.sqrt(size) # one score matrix per head19scores = np.where(np.tril(np.ones((positions, positions), dtype=bool)), scores, -np.inf) # the causal mask, for both heads20weights = np.exp(scores - scores.max(-1, keepdims=True))21weights /= weights.sum(-1, keepdims=True)22joined = (weights @ values).transpose(1, 0, 2).reshape(positions, dims) # the heads' outputs side by side23output = joined @ output_projection # mixed back together by one more matrix2425print("weights shape (heads, positions, positions):", weights.shape)26print("what the last token attends to, head 1:", np.round(weights[0, -1], 2)) # -1 is the last position, which sees all four27print("what the last token attends to, head 2:", np.round(weights[1, -1], 2))28print("output shape:", output.shape)
python attention-heads.pyweights shape (heads, positions, positions): (2, 4, 4)
what the last token attends to, head 1: [0.41 0.18 0.29 0.11]
what the last token attends to, head 2: [0.04 0.32 0.14 0.5 ]
output shape: (4, 8)
现在每个头各有一个权重矩阵,形状为 (2, 4, 4)。尽管两个头读取的是同一份输入,最后一个位置的各行在两个头之间并不相同:头 1 把最大的权重放在第一个位置上,头 2 则放在最后一个位置,也就是它自身。矩阵是随机的,所以这种差异目前没有任何意义,但它正是训练可以利用的空间:每个头可以学会不同的回看方式,输出投影再把它们的发现组合起来。输出再次具有与输入相同的形状 (4, 8),因此注意力层可以堆叠起来,第 7 章就是这样做的。
工作准则: 经验上,应让每个头至少有 32 个数值宽。在 Transformer 论文的实验中,16 个头、每头 32 的配置与 8 个头、每头 64 的配置效果相当,而 32 个头、每头 16 的配置效果更差,缩小键的维度也会损害效果(Vaswani et al., 2017,表 3)。每次修改模型时,都要检查权重张量的形状,即头数×位置数×位置数:头所在的轴放错位置,会在没有任何报错的情况下把各个头混在一起。
这部分尚未翻译,因此以英文显示。
Read what trained heads attend to
Random matrices show the arithmetic; a trained model shows what attention is for. The store's model has 4 layers, each with 4 heads of attention. It was trained once, in recordings/transformer.py, on the store's training names and descriptions, each product as one text: the start marker, the name, a separator token <sep>, the description and the end marker. Each layer's attention reads the output of the layer before, so later layers can attend on the basis of what earlier ones found. As in chapter 5, the examples read the saved weights and recompute the model in NumPy, with lib.transformer, which also returns every head's weights.
The question is where the model looks for cover in a validation product it never trained on, double chaise patio furniture cover, whose description repeats the end of its name. Stop at the description's second furniture, where the next token is cover, and print the model's probability for cover, and each head's three largest weights in the first and last layers. current: marks the position being predicted from, name: a token in the name, and description: an earlier token in the description:
1"""Runs the store's trained model on one validation product, up to the description's second furniture, and prints where2its heads attend there. The prose's point rests on the last layer's head with the most weight on the name's cover, and on3the probability it gives cover."""4import numpy as np56from lib import transformer7from lib.store import load_products89_, vocabulary, manifest = transformer.load() # forward reads the weights itself, so only the vocabulary and manifest are kept10settings = manifest["settings"]11index = {t: i for i, t in enumerate(vocabulary)}12product = load_products("validation").query("name == 'double chaise patio furniture cover'").iloc[0]13ids = transformer.product_ids(product["name"], product["description"], index, settings["longest"])14sep = ids.index(index[transformer.SEP])15position = ids.index(ids[sep - 1], sep) - 1 # just before the description first repeats the name's last token16logits, attention = transformer.forward(ids, keep=True)17probabilities = transformer.softmax(logits[position])181920def label(i):21 """Where a token sits: the current position, the name, the description, or a marker."""22 if i == position:23 return f"current:{vocabulary[ids[i]]}"24 if i == 0 or i == sep:25 return vocabulary[ids[i]]26 return f"{'name' if i < sep else 'description'}:{vocabulary[ids[i]]}"272829print("layers:", settings["layers"])30print("heads per layer:", settings["heads"])31print("dimensions per head:", settings["dimensions"] // settings["heads"])32print("name:", " ".join(vocabulary[i] for i in ids[1:sep]))33# after <sep>, up to and including the current position34print("description so far:", " ".join(vocabulary[i] for i in ids[sep + 1:position + 1]))35# position + 1 is the token that really comes next in the text36print(f"P(next token is {vocabulary[ids[position + 1]]}): {probabilities[ids[position + 1]]:.2f}")37for layer in [0, len(attention) - 1]:38 for head, weights in enumerate(attention[layer]):39 top = np.argsort(-weights[position], kind="stable")[:3]40 print(f"layer {layer + 1}, head {head + 1}: " + ", ".join(f"{label(i)} {weights[position, i]:.2f}" for i in top))4142name_cover = sep - 1 # the name's last token, `cover`43earlier = ids.index(ids[position], sep + 1) # the description's earlier `furniture`44last = attention[-1][:, position] # the last layer's heads, at the current position45head = int(last[:, name_cover].argmax())46print("last-layer head with the most weight on the name's cover:", f"head {head + 1}")47print("its weight on the name's cover:", f"{last[head, name_cover]:.2f}")48print(f"its weight on {vocabulary[ids[earlier + 1]].removesuffix('</w>')}, after the description's earlier furniture:", # earlier + 1: the token after the first furniture49 f"{last[head, earlier + 1]:.2f}")
python trained-attention.pylayers: 4
heads per layer: 4
dimensions per head: 32
name: double</w> chaise</w> patio</w> furniture</w> cover</w>
description so far: protect</w> your</w> outdoor</w> patio</w> furniture</w> from</w> har sh</w> elements</w> with</w> this</w> outdoor</w> patio</w> furniture</w>
P(next token is cover</w>): 0.97
layer 1, head 1: current:furniture</w> 0.80, description:furniture</w> 0.11, description:elements</w> 0.03
layer 1, head 2: current:furniture</w> 0.31, name:furniture</w> 0.13, description:furniture</w> 0.11
layer 1, head 3: current:furniture</w> 0.18, description:patio</w> 0.18, <sep> 0.12
layer 1, head 4: description:protect</w> 0.17, current:furniture</w> 0.14, description:patio</w> 0.11
layer 4, head 1: name:cover</w> 0.48, name:double</w> 0.18, name:furniture</w> 0.13
layer 4, head 2: name:cover</w> 0.37, <s> 0.13, name:chaise</w> 0.11
layer 4, head 3: description:patio</w> 0.27, current:furniture</w> 0.25, description:outdoor</w> 0.21
layer 4, head 4: description:with</w> 0.38, description:from</w> 0.21, description:elements</w> 0.06
last-layer head with the most weight on the name's cover: head 1
its weight on the name's cover: 0.48
its weight on from, after the description's earlier furniture: 0.00
The model gives cover a probability of 0.97, and the heads show where it may have looked. In the first layer, one head attends mostly to the current token, furniture, and the other three spread their weight over the description and the name so far. In the last layer, head 1 puts 0.48 of its weight on cover in the name, many tokens back: the token that followed furniture in the name. The description's own earlier furniture was followed by from, and that head gives from only 0.00. A second last-layer head also puts its largest weight on the name's cover.
That pattern has a name. Olsson and colleagues found pairs of heads that work this way in small models made only of attention layers, and heads with the same behaviour in large models. They call the later head of each pair an induction head: a head in an earlier layer copies each token's predecessor into its position, and the induction head uses that to find earlier positions that followed the current token, and attends to them (Olsson et al., 2022). The weights above are consistent with that behaviour, but weights show where a head drew its average from, not what the model used, so the next section tests it.
Working rule: Inspect a trained model by starting with specific tokens in texts you understand, where you know what the model should look at. Treat what the weights suggest as a hypothesis, and test it by changing the input, as the next section does.
检验模型复制了什么
如果模型是从名称中复制 cover 来预测它,那么改变名称就应当改变预测。如果模型预测 cover,是因为这类描述常常以此续写,那么名称就不应有太大影响。改变输入并观察输出的检验可以回答这个问题,而注意力权重做不到。
保持描述不变,改变名称:按原样出售的名称、去掉 cover 的名称,或以 set 结尾的名称。在同一位置上 cover 和 set 的概率,可以说明预测是否跟随名称。然后统计训练描述中 this outdoor patio furniture 之后出现的内容,也就是窗口模型能看到的那四个词元:
1"""Changes the name of one validation product and reads the model's probabilities for cover and set at the same position,2then counts what follows this outdoor patio furniture in the training descriptions. The prose's point rests on P(cover)3falling when cover leaves the name, and on how rarely this outdoor patio furniture is followed by cover in the training4descriptions, which is all the four tokens alone give."""5from lib import transformer6from lib.store import load_products78_, vocabulary, manifest = transformer.load() # forward reads the weights itself, so only the vocabulary and manifest are kept9index = {t: i for i, t in enumerate(vocabulary)}10product = load_products("validation").query("name == 'double chaise patio furniture cover'").iloc[0]11cover, other = index["cover</w>"], index["set</w>"]1213for name, label in [("double chaise patio furniture cover", "name as sold"),14 ("double chaise patio furniture", "name without cover"),15 ("double chaise patio furniture set", "name ending in set")]:16 ids = transformer.product_ids(name, product["description"], index, manifest["settings"]["longest"])17 sep = ids.index(index[transformer.SEP])18 furniture = [i for i in range(sep + 1, len(ids)) if ids[i] == index["furniture</w>"]]19 p = transformer.softmax(transformer.forward(ids)[furniture[1]]) # at the description's second `furniture`20 print(f"{label}, P(cover): {p[cover]:.2f}")21 print(f"{label}, P(set): {p[other]:.2f}")2223phrase = "this outdoor patio furniture " # the trailing space makes the phrase end at a word boundary24# the word after the phrase, in each description that has one; a phrase at the very end has no next word and is skipped25after = [d.split(phrase, 1)[1].split()[0] for d in load_products("train")["description"]26 if isinstance(d, str) and phrase in d and d.split(phrase, 1)[1].split()]27print("training descriptions with 'this outdoor patio furniture':", len(after))28print("followed by cover:", after.count("cover"))
python copy-from-the-name.pyname as sold, P(cover): 0.97
name as sold, P(set): 0.02
name without cover, P(cover): 0.00
name without cover, P(set): 0.29
name ending in set, P(cover): 0.00
name ending in set, P(set): 0.70
training descriptions with 'this outdoor patio furniture': 6
followed by cover: 2
名称决定了预测。名称按原样时,模型给 cover 的概率为 0.97;名称中去掉 cover 后,概率降至 0.00。如果改为以 set 结尾,模型给 set 的概率为 0.70,给 cover 的概率为 0.00。
仅凭这四个词元,只能微弱地指向 cover:在包含 this outdoor patio furniture 的 6 条训练描述中,有 2 条接着出现 cover。
这正是上一节的权重所暗示的行为,现在由其效果得到了证明:模型复制了产品名称中的一个词,而该词在许多词元之前,受限于四个词元窗口的模型做不到这一点。一个产品并不能衡量模型这样做的频率;接下来的两节会在许多产品上度量它的注意力头。
工作准则: 要检验关于模型使用了什么的论断,就改变该输入并度量输出,并且要包含一项本应不影响输出的改变。注意力权重用于提出假设,这类检验用于得出结论。
一些注意力头关注前一个词元
单看一段文本可能产生误导;对许多文本取平均,才能显示一个注意力头的习惯。一个值得度量的习惯是关注前一个词元。做到这一点的注意力头会把每个词元的前驱带到它自己的位置上,因此后面的层看到的是成对的词元,正如第 1 章中的二元组计数那样;前面几节中的复制也依赖于此:要找到之前跟在 furniture 后面的内容,注意力头需要每个较早的位置都知道自己的前驱。
对前 300 个验证集商品,取每个注意力头在前一个词元上的权重的平均值,并把在前一个词元上权重最大的注意力头与均匀分布作比较。均匀分布在位置 i 能看到的 i + 1 个词元上各放 1 / (i + 1) 的权重:
1"""Averages each head's weight on the previous token over 300 validation products, and sets the largest average beside2an even spread. The prose's point rests on the head with the most weight on the previous token, and on the even spread beside it."""3import numpy as np45from lib import transformer6from lib.products import product_texts78texts = product_texts("validation", 300) # the first 300 validation products, which the model never trained on9# with keep=True, forward's second value is every layer's weights; the head count is in the manifest's settings10layers, heads = len(transformer.forward(texts[0][0], keep=True)[1]), transformer.load()[2]["settings"]["heads"]11on_previous = np.zeros((layers, heads))12for ids, _ in texts:13 _, attention = transformer.forward(ids, keep=True)14 for layer, weights in enumerate(attention): # weights: (heads, positions, positions)15 on_previous[layer] += weights[:, np.arange(1, len(ids)), np.arange(len(ids) - 1)].mean(1) # position i's weight on i - 1, for i after the start16on_previous /= len(texts)17even = np.mean([np.mean(1 / np.arange(2, len(ids) + 1)) for ids, _ in texts]) # position i sees i + 1 tokens, so 1 / (i + 1) each1819print("average weight on the previous token (rows: layers, columns: heads):")20for layer, row in enumerate(on_previous, start=1):21 print(f" layer {layer}: " + " ".join(f"{w:.2f}" for w in row))22layer, head = np.unravel_index(on_previous.argmax(), on_previous.shape)23print("head with the most weight on the previous token:", f"layer {layer + 1}, head {head + 1}")24print("its weight on the previous token:", f"{on_previous.max():.2f}")25print("an even spread's weight on the previous token:", f"{even:.2f}")
python previous-token-heads.pyaverage weight on the previous token (rows: layers, columns: heads):
layer 1: 0.03 0.06 0.13 0.10
layer 2: 0.32 0.13 0.27 0.24
layer 3: 0.13 0.33 0.09 0.07
layer 4: 0.11 0.07 0.35 0.27
head with the most weight on the previous token: layer 4, head 3
its weight on the previous token: 0.35
an even spread's weight on the previous token: 0.07
在前一个词元上权重最大的注意力头是 layer 4, head 3,平均权重为 0.35。按同样的方法取平均,均匀分布在这里的权重为 0.07,所以该注意力头对前一个词元的偏好约为均匀分布的五倍。后面几层中有若干注意力头在这里放了四分之一或更多的权重,而第一层的注意力头放的权重要少得多:在训练后注意力示例中,其中一个头主要关注当前词元。
没有哪个注意力头是纯粹的前一词元头;每个头都把这一习惯与其他习惯混合在一起。一个头的平均值说明的是它最常做什么,而不是它做的全部,因此把一个头描述为只有一项职能是过度简化。
工作准则: 用测得的平均值来描述一个训练后的注意力头,例如它在前一个词元、当前词元或起始标记上的权重,并在几百段文本上取平均。把每个值与按同样方法取平均的均匀分布所给出的值作比较,这样 0.3 一类的数字才能判断是大还是小。
越过窗口
注意力的承诺是越过第 5 章的四词元窗口。那个窗口根据当前词元及其前三个词元进行预测。商店的模型是否做到了这一点,是可以测量的。窗口之外落了多少权重?对前 300 个验证集商品中的每个描述词元(不含结束标记),先把每个注意力头放在向前 4 个或更多位置的词元上的权重加起来,这是四词元窗口看不到的最近位置;再单独统计放在两个标记 <s> 和 <sep> 上的权重。然后对词元、注意力头和层取平均,并与在每个词元可见位置上的均匀分布作比较:
1"""Measures how much attention a description token puts on tokens four or more positions back, and on the two markers,2against an even spread, over 300 validation products. The prose's point rests on the all-layers share four or more back,3and on the count of descriptions that repeat a token of the name."""4import numpy as np56from lib import transformer7from lib.products import product_texts89_, vocabulary, manifest = transformer.load()10end = vocabulary.index(transformer.END)11texts = product_texts("validation", 300) # the first 300 validation products, which the model never trained on12layers = manifest["settings"]["layers"]13far, markers, even_far, even_markers, counted = np.zeros(layers), np.zeros(layers), 0.0, 0.0, 014for ids, sep in texts:15 # the description's tokens: after <sep>, so not the name, and without the end marker16 queries = np.array([i for i in range(sep + 1, len(ids)) if ids[i] != end])17 if len(queries) == 0:18 continue # a product with no description19 _, attention = transformer.forward(ids, keep=True)20 stacked = np.stack(attention)[:, :, queries] # (layers, heads, description tokens, positions)21 positions = np.arange(len(ids))22 is_marker = np.isin(positions, [0, sep]) # <s> and <sep>23 beyond = (queries[:, None] - positions[None, :] >= 4) & ~is_marker # 4 positions back is the nearest a four-token window can't see24 far += (stacked * beyond).sum(-1).mean(1).sum(-1)25 markers += (stacked * is_marker).sum(-1).mean(1).sum(-1)26 even_far += (beyond.sum(1) / (queries + 1)).sum() # an even spread over the positions each token can see27 even_markers += (2 / (queries + 1)).sum() # two markers among the query's i + 1 visible positions28 counted += len(queries)2930print("description tokens measured:", f"{counted:,}")31print("average attention weight of a description token, by layer:")32for layer in range(layers):33 print(f" layer {layer + 1}: 4 or more tokens back {far[layer] / counted:.2f}, on <s> and <sep> {markers[layer] / counted:.2f}")34print("all layers, weight 4 or more tokens back:", f"{far.mean() / counted:.2f}")35print("an even spread's weight 4 or more tokens back:", f"{even_far / counted:.2f}")36print("all layers, weight on <s> and <sep>:", f"{markers.mean() / counted:.2f}")37print("an even spread's weight on <s> and <sep>:", f"{even_markers / counted:.2f}")38# the name without <s>, and what follows <sep>; a product counts as described if anything but the end marker follows39described = [(ids[1:sep], ids[sep + 1:]) for ids, sep in texts if any(i != end for i in ids[sep + 1:])]40print("products with a description:", len(described))41print("descriptions that repeat a token of the name:", sum(bool(set(n) & set(d)) for n, d in described))
python attention-reaches-back.pydescription tokens measured: 20,571
average attention weight of a description token, by layer:
layer 1: 4 or more tokens back 0.42, on <s> and <sep> 0.12
layer 2: 4 or more tokens back 0.39, on <s> and <sep> 0.04
layer 3: 4 or more tokens back 0.58, on <s> and <sep> 0.13
layer 4: 4 or more tokens back 0.47, on <s> and <sep> 0.13
all layers, weight 4 or more tokens back: 0.46
an even spread's weight 4 or more tokens back: 0.85
all layers, weight on <s> and <sep>: 0.11
an even spread's weight on <s> and <sep>: 0.05
products with a description: 262
descriptions that repeat a token of the name: 244
正如预测下一个词时所预期的那样,各个头偏好邻近的词元,但仍有很大一部分注意力落在窗口之外,而本书中此前的任何模型都无法利用这部分信息。在所有层中,描述词元把 0.46 的注意力放在向前 4 个或更多位置的词元上,而均匀分布只会放 0.85,因为描述词元可见的内容大部分都在较远处。
这对这类文本很重要:在 262 个带描述的商品中,有 244 个的描述至少重复了名称中的一个词元。
标记获得了 0.11 的注意力,而均匀分布下为 0.05,约为随机水平的两倍,尽管它们本身并不预测任何词。一个可能的原因是在更大的模型中发现的一种模式。softmax 使每一行权重之和为 1,因此在某个位置上找不到有用内容的头,仍然必须把权重放到某处。Xiao 及其同事发现,语言模型会把很大的权重放在文本的最前面几个词元上,不论它们是什么,并称之为注意力汇聚点(attention sink,见 Xiao et al., 2024)。在商店的模型中,<s> 总是排在最前面,<sep> 出现在每段文本中,因此两者都可能充当汇聚点;<sep> 还标记着描述的起点,这本身就有用。本示例无法区分这两种情况。
工作准则: 在决定模型的上下文需要多长之前,先测量已训练模型向前注意多远,并与均匀分布作比较,使这个数字有意义。单独报告标记上的权重,以免标记权重被误当作长程注意力;为节省空间而截短模型的上下文时,要保留最前面的词元。
生产环境中会出什么问题
注意力是语言模型中成本随文本长度增长最快的部分,也是最常被误读的部分:
- 成本随长度的平方增长。 每个位置都要为之前的每个位置打分,因此长度翻倍的文本,注意力的成本是原来的四倍。要设定最大长度,就像商店的模型把上限设为 160 个词元那样,并检查有多少文本被这一上限截断。
- 错误的掩码会悄无声息地失效。 能看到未来的模型可以达到很低的训练损失,却依然无法用于生成文本。要在训练代码和生成代码中测试:后续位置上的权重为零,并且改动后面的词元不会影响前面的输出。
- 注意力权重不是解释。 高权重只说明某个注意力头的平均值取自哪里,并不说明模型为什么做出那样的预测。在同一个模型中,差异很大的注意力权重可以给出相同的预测(Jain and Wallace, 2019,针对循环和卷积分类器)。权重可以用来提出假设,再通过改变输入来检验假设,正如本章对产品名称所做的那样。
- 标记词元上的权重看起来像重要性。 第一个词元或分隔符上的权重很大,可能只是说明某个注意力头在那里无事可做。不要删除这些词元,也不要把它们的权重当作有意义的信号。
- softmax 不加小心就会溢出。 先从每一行中减去该行的最大分数,再做指数运算,
lib.transformer.softmax就是这样做的。要确保每一行至少保留一个未被掩码的位置,否则该行会变成 NaN:因果掩码总会为每个位置保留其自身,但遮蔽填充的掩码可能会把整行都遮住。
本章的决定
- **本书的模型使用因果、缩放的多头注意力,**与 Vaswani 等人的定义一致:分数除以每个头的查询与键的维度大小的平方根,在 softmax 之前把所有靠后位置设为负无穷,各个头分别处理数值的一部分,再由输出投影混合。
- **商店的模型有 4 层,每层 4 个头,每个头宽 32 个数,**在
recordings/transformer.py中用以<sep>标记连接的名称和描述一次性训练,并由lib.transformer用 NumPy 重新计算。 - **关于模型使用了什么的说法,要通过改变其输入来检验。**注意力权重只能提出假设;改变产品名称后可以看到,模型会从名称中复制
cover。 - **对训练后的头,用大量文本上的实测平均值来描述,**并与均匀分布比较:放在前一个词元上的权重、放在 4 个或更多位置之前的词元上的权重,以及放在标记上的权重,三者分别统计。
- **在商店的描述文本上,注意力的跨度远超第 5 章的窗口。**描述词元把 0.46 的注意力放在 4 个或更多位置之前的词元上,而且模型会从名称中复制内容,这是固定窗口做不到的。
- **注意力需要明确的位置信息。**加权平均不考虑顺序,因果掩码只能恢复其中很少的一部分,因此第 7 章加入了旋转位置编码,商店的模型也在每个头内部使用它。
本章,在本版次中
版次 2026.11.16示例于 2026年11月19日 通过本章 8个示例 个示例中有 8 个通过
译自英文版图书,版次 2026.11.16。