AI 数据治理
阅读约需 116分钟 分钟,12节中含10个示例。
本页内容
- Admit only reviews whose authors consented
- Replace personal data in the text, then measure what's left
- 用带密钥的哈希保持假名一致
- Find near-duplicates with MinHash
- Tune LSH for recall, then confirm every pair
- 为每条记录携带来源信息
- Generate the dataset card from the run
- 引用《欧盟人工智能法》关于训练数据的规定
- 生产环境中会出什么问题
- A learner withdraws after the build
- 其他故障
- What this chapter decided
读完本章,你将能够在与平台其他所有读取操作相同的规则下,构建 AI 助手学习所用的数据。你会只收录作者同意用于此用途的记录,用化名替换其中的个人数据并度量检测器的漏检情况,用 MinHash 找出并删除重复内容,让每条记录始终带有其来源,并让流水线在每次运行时生成并检查数据集卡片。你还将能够引述欧盟《人工智能法案》(EU AI Act)对训练数据的规定。
本书贯穿始终的在线学习平台希望有一个课程助手,能根据学习者在评价中写下的内容作答。平台计划先以这些文本为依据来约束助手的回答,之后再用同样的文本对其进行微调。review_corpus是第 4 章从中采集血缘的四条 SQL 流水线之一,它把learning.reviews的每一行都复制到ai.review_corpus,并附上每条评价所属的课程和评分。第 2 章发现ai.review_corpus没有所有者,也没有分类,而这份副本还有另外四个问题。第一,它包含所有评价,不论作者是否同意其文本用于训练 AI。第二,它的文本中含有姓名和电子邮件地址,这是第 3 章在评价中发现的。第三,其中大多数评价彼此重复。第四,其中没有任何信息说明记录来自何处,因此没有人能证明某条评价是被允许收录的,也无法在作者改变主意时重新找到它。
本章用本书共享的 AI 数据模块lib.ai_data重建这项作业,助手只从这个受治理的构建结果中学习:pipelines/review_corpus.sql仍保留为第 4 章中未经治理的副本,因此 CI会继续检查它的血缘。该模块通过第 6 章构建的执行层运行流水线自身的 SQL,替换文本中残留的个人数据,删除重复内容,并写出语料库、它的血缘和它的数据集卡片。模块的设置位于governance/ai_data.toml,像策略一样接受评审;代码所需的法律问题答案也在其中,每个答案都是一个占位项,由平台法务团队换成自己的答案和理由。
这部分尚未翻译,因此以英文显示。
Admit only reviews whose authors consented
On the platform, consent to one use doesn't cover another, because the question a learner answers names the use. Article 5(1)(b) of the EU's General Data Protection Regulation (GDPR) says that personal data shall be "collected for specified, explicit and legitimate purposes and not further processed in a manner that is incompatible with those purposes" (GDPR). The code reads the recorded answer to whether training an AI is covered from [legal.consent] in governance/ai_data.toml, and decides consent per record when the corpus is built, not from the account's sign-up and not once for the whole table.
crm.learners.ai_training_consent records whether a learner agreed to their text training the platform's AI. Chapter 5 made it an attribute the access policy reads, so a read of learning.reviews for the purpose ai-training keeps only the reviews whose author's answer is true, with contact details removed. lib.ai_data.read runs the SELECT of pipelines/review_corpus.sql through the enforcement layer as analyst-1, the data-science principal, stating the purpose. lib.ai_data.build refuses to run unless the layer filters on the column the consent answer names. Compare three builds: the plain pipeline, pipelines/review_corpus.sql run as written; the governed read; and a consent filter written by hand.
1"""Counts the reviews that three builds of the corpus would hold, and the personal data the plain copy would keep. The prose's point rests on the "share by learners never asked" line."""2from collections import Counter34from lib import ai_data, pii5from lib.catalog import open_catalog6from lib.enforce import Enforcer78platform = Enforcer(open_catalog(["crm.learners", "learning.courses", "learning.reviews"]))9layer = platform.connection # the enforcement layer's own access, for counting what each build would hold1011# The plain pipeline copies every review, with its text as written.12plain = layer.sql("select count(*) from learning.reviews").fetchall()[0][0]13print("reviews the plain pipeline copies:", f"{plain:,}")1415# The governed build: the same SQL, through the enforcement layer, as the job's principal, for its purpose.16# the job's own read: its principal and purpose come from the build settings17governed = ai_data.read(platform)18print(f"reviews the governed build reads, as {ai_data.BUILD['principal']} for {ai_data.BUILD['purpose']}:", f"{len(governed):,}")19# no purpose stated: the layer returns every review, with its text left out20without = ai_data.read(platform, purpose=None)21print("reviews read without the purpose:", f"{len(without):,}")22print("of them, with their text:", sum(r["body"] is not None for r in without))2324# The filter that looks right: leave out the learners who said no.25# every review except those whose learner said no; a null answer passes, since only "= false" is excluded26opt_out = layer.sql("""27 select count(*), count(*) filter (where l.ai_training_consent is null)28 from learning.reviews r join crm.learners l using (learner_id)29 where r.learner_id not in (select learner_id from crm.learners where ai_training_consent = false)""").fetchall()[0]30print("reviews a filter on refusals keeps:", f"{opt_out[0]:,}")31print("of them, by learners never asked:", f"{opt_out[1]:,}")32print("share by learners never asked:", f"{opt_out[1] / opt_out[0]:.0%}")33# the signup dates of the learners whose consent is still unknown34never = layer.sql("""select min(signup_at)::date, max(signup_at)::date from crm.learners35 where ai_training_consent is null""").fetchall()[0]36print("learners never asked signed up:", " to ".join(f"{day:%b} {day.day}, {day.year}" for day in never))3738# Personal data the plain pipeline's corpus would hold, from the labelled truth, and how often each email is written.39raw = dict(layer.sql("select review_id, body from learning.reviews").fetchall())40for entity, (placed, _) in ai_data.left(raw).items():41 print(f"plain pipeline, labelled {entity} values:", f"{placed:,}")42# each labelled email's text, counted by how many spans write it43emails = Counter(pii.text_of(s) for s in pii.truth() if s.source == "reviews" and s.entity == "EMAIL_ADDRESS")44print("plain pipeline, distinct labelled EMAIL_ADDRESS values:", len(emails))45print("of them, written once:", sum(n == 1 for n in emails.values()))
python corpus-consent-filter.pyreviews the plain pipeline copies: 3,000
reviews the governed build reads, as analyst-1 for ai-training: 436
reviews read without the purpose: 3,000
of them, with their text: 0
reviews a filter on refusals keeps: 2,045
of them, by learners never asked: 1,609
share by learners never asked: 79%
learners never asked signed up: Jan 1, 2024 to May 31, 2025
plain pipeline, labelled EMAIL_ADDRESS values: 120
plain pipeline, labelled PERSON values: 798
plain pipeline, distinct labelled EMAIL_ADDRESS values: 113
of them, written once: 106
Copied as written, the plain pipeline would put 120 labelled emails and 798 labelled names into the corpus, from 3,000 reviews.
The governed read keeps 436 reviews, the ones whose authors said yes. Without a purpose, the same principal gets 3,000 reviews, 0 of them with their text, so a job that forgets to state its purpose gets nothing it could train on.
Notice that the filter written by hand looks like respect for consent, which is why it's the one to recognize: it leaves out the learners who said no. It keeps 2,045 reviews, and 1,609 of them, 79%, are by learners who were never asked. The README of the platform's generated data says consent is unknown for sign-ups before Jun 1, 2025, and every learner with no answer signed up from Jan 1, 2024 to May 31, 2025. An unknown answer isn't a yes. A corpus built with that filter would be mostly text that no consent covers, and nothing about it would look wrong: no error, a plausible row count, and a condition on the consent column that a reviewer would read as correct. The enforcement layer's filter tests IS TRUE, so an unknown answer counts as no, which chapter 6 built for exactly this case.
On the platform, every review was written under one set of terms, so the card states one licence, read from [legal.licence]. A corpus that mixes sources, such as reviews and forum posts, needs a licence per record too, which is one reason every record keeps its source table.
Working rule: Decide consent per record, when the corpus is built, from the answer your legal team says covers the use, and admit only records whose answer is yes: never filter out the refusals and keep the rest. Build the corpus through the enforcement layer with its purpose stated, and have the build check that the layer filters on that answer. Keep the legal team's answers, with their reasons, in a reviewed file the code reads, and carry licences per record when a source mixes them.
这部分尚未翻译,因此以英文显示。
Replace personal data in the text, then measure what's left
Consent decides whose text may enter the corpus. It doesn't make the text free of personal data: a review names its instructor, or gives its author's email for questions. Before such text trains or grounds a model, the corpus job detects the personal data in it, replaces each value, and measures what's left, because a detector's output lists what it found and never what it missed. The truth, chapter 3's labelled spans, marks every piece of personal data the generator placed in the reviews. lib.ai_data.left counts how many labelled values are still in the text after a step, matching by value, since every replacement moves the offsets after it.
The reviews arrive from the enforcement layer with its redact_contacts mask applied, which replaces emails and phone numbers with [email] and [phone] (chapter 6). Chapter 3's detector, lib.pii.platform_detector, runs Presidio's pattern recognizers with no language model, and no pattern finds a name. So lib.ai_data.detector adds a Presidio deny list of the names the platform knows: every word of every instructor's name, which Presidio escapes and matches as a whole word, ignoring case unless told otherwise (Presidio's pattern recognizer). The list comes from crm.instructors, which analyst-1 may not read. So the platform team's own job, named in [names] in governance/ai_data.toml, builds it with the platform's own access, as the enforcement layer reads crm.learners for consent, and hands the corpus job the list, never the table. The card records which job built it and how many words it holds. lib.ai_data.known_names stands for that job. Each value found becomes a keyed pseudonym, which the next section builds. Measure each step on the reviews the corpus reads, then try a longer list, with every learner's name in it too:
1"""Counts the labelled names and emails still in the reviews after each step of the corpus job, and the false positives of the instructors' list and of a longer one. The prose's point rests on the "then the instructors' names, PERSON left" line and the false-positive lines."""2from lib import ai_data, pii3from lib.catalog import open_catalog4from lib.enforce import Enforcer56# The reviews the corpus job reads: consenting authors only, with emails and phone numbers already masked.7rows = ai_data.read(Enforcer(open_catalog(["crm.learners", "learning.courses", "learning.reviews"])))8texts = {row["review_id"]: row["body"] for row in rows} # keyed by review ID, to match the labelled values row by row910# What's left of the labelled personal data after each step, matched by value against the generator's truth.11detectors = {"then the pattern recognizers": pii.platform_detector(), "then the instructors' names": ai_data.detector()}12# the texts as the enforcement layer leaves them, before any detector runs13stages = {"masked by the enforcement layer": texts}14# each detector runs alone on the masked texts; pseudonymize returns the text first15stages |= {name: {i: ai_data.pseudonymize(engine, text)[0] for i, text in texts.items()} for name, engine in detectors.items()}16for stage, done in stages.items():17 for entity, (placed, still) in ai_data.left(done).items():18 print(f"{stage}, {entity} left: {still} of {placed}")1920# the instructors' words, from the governance team's list21names = ai_data.known_names()22print("words in the list of instructors' names:", len(names))23# short words are the ones ordinary text could contain24print("words in the list of 3 characters or fewer:", ", ".join(word for word in names if len(word) <= 3))25false_positives = sum(ai_data.unlabelled(ai_data.detector(), texts).values())26assert false_positives == 027print("false positives, instructors' names:", false_positives)2829# Every name the platform holds, learners' too, on the same reviews: in any case, then only as written.30# the union of the instructors' words and every learner's, which the table holds31longer = sorted(set(names) | set(ai_data.known_names("crm.learners")))32# matched in any case, as the platform's detector matches by default33everyone = ai_data.detector(longer)34wrong = ai_data.unlabelled(everyone, texts)35print("words in the list, with learners' names:", f"{len(longer):,}")36print("false positives, with learners' names:", sum(wrong.values()))37print("most common false positives:", ", ".join(f"{word} ({n})" for word, n in wrong.most_common(2)))38# review 24, which the longer list changes39print("a review after:", ai_data.pseudonymize(everyone, texts[24])[0])40# a word matches only as written, so a lowercase word no longer matches a capitalized name41cased = ai_data.detector(longer, match_case=True)42cased_wrong = sum(ai_data.unlabelled(cased, texts).values())43assert cased_wrong == 044print("false positives, with learners' names, matching case:", cased_wrong)45# pseudonymize with the cased detector, then count what is left by value46left = ai_data.left({i: ai_data.pseudonymize(cased, text)[0] for i, text in texts.items()})47assert left["PERSON"][1] == 048print("names left, with learners' names, matching case:", f"{left['PERSON'][1]} of {left['PERSON'][0]}")
python corpus-personal-data-left.pymasked by the enforcement layer, EMAIL_ADDRESS left: 0 of 19
masked by the enforcement layer, PERSON left: 120 of 120
then the pattern recognizers, EMAIL_ADDRESS left: 0 of 19
then the pattern recognizers, PERSON left: 120 of 120
then the instructors' names, EMAIL_ADDRESS left: 0 of 19
then the instructors' names, PERSON left: 0 of 120
words in the list of instructors' names: 106
words in the list of 3 characters or fewer: Ray, Roy, Ulf, Uss, de, la
false positives, instructors' names: 0
words in the list, with learners' names: 4,316
false positives, with learners' names: 147
most common false positives: best (80), date (67)
a review after: Some videos are out of <PERSON_8e2b63a6>.
false positives, with learners' names, matching case: 0
names left, with learners' names, matching case: 0 of 120
The enforcement layer's mask leaves 0 of 19 emails and 120 of 120 names, and the pattern recognizers leave 120 of 120 names, since no pattern finds a name.
Only the instructors' list removes names. With its names added, the detector leaves 0 of 120 names and makes no false positives, although its list holds short words that ordinary text could contain: Ray, Roy, Ulf, Uss, de, la.
The longer list, with every learner's name in it too, makes 147 false positives. The most common, best (80), date (67), are surnames that are also English words, and the list turns Some videos are out of date. into Some videos are out of <PERSON_8e2b63a6>. in the corpus.
Matching case avoids both. Matched only as written, the longer list makes no false positives and leaves no names, because every name in these reviews is capitalized. On real text it would miss the names typed in lowercase that chapter 3 found in tickets. The platform keeps the instructors' list, matched in any case: instructors are the people reviews talk about, and the corpus job then holds their names instead of every learner's.
The list's result holds for this data only. The labelled spans are generated, and the generated reviews name only instructors, so they can't contain the case a list of instructors misses, such as a classmate's name. For names nobody listed, the detector to try is a named-entity model. Chapter 3 compared one, OpenAI's Privacy Filter, from a recording of its detections, scored against the labelled spans with lib.pii.score. Score it the same way, on the reviews the corpus reads:
1"""Scores OpenAI's Privacy Filter, from its recorded detections, against the labelled spans in the reviews the corpus reads. The prose's point rests on the "names the model finds, exactly" and "names the model misses" lines."""2from collections import Counter34from lib import ai_data, pii5from lib.catalog import open_catalog6from lib.enforce import Enforcer78# Privacy Filter's eight labels, mapped to the labelled spans' entity types. A label with no type matches nothing.9MATCHES = {"private_person": {"PERSON"}, "private_email": {"EMAIL_ADDRESS"}, "private_phone": {"PHONE_NUMBER"},10 "private_address": {"LOCATION"}, "account_number": {"CREDIT_CARD", "IBAN_CODE"},11 "private_date": {"DATE_TIME"}, "private_url": set(), "secret": set()}1213# The reviews the corpus job reads, with the model's recorded detections and the labelled spans in them. The model read14# each review as written, before the enforcement layer's mask, which changes no name.15corpus = {row["review_id"] for row in ai_data.read(Enforcer(open_catalog(["crm.learners", "learning.courses",16 "learning.reviews"])))}17# the model's detections as recorded, not a live call to the model18model, manifest = pii.recorded("pii-entities")19found = [span for span in model if span.source == "reviews" and span.row_id in corpus]20labelled = [span for span in pii.truth() if span.source == "reviews" and span.row_id in corpus]21print("model:", manifest["model"])22print("reviews measured:", len(corpus))2324# per entity type: [labelled, found exactly, found overlapping], then per detector label: [found, exact, overlapping]25scores = pii.score(found, labelled, MATCHES)26# the overlap count is unused here: the prose counts exact matches only27placed, exact, _ = scores.recall["PERSON"]28print("labelled names:", placed)29print("names the model finds, exactly:", exact)30# private_person is the model's label for a name, so only that label counts as a match31missed = pii.missed([span for span in labelled if span.entity == "PERSON"], found, {"private_person"})32print("names the model misses:", len(missed))33often = Counter(map(pii.text_of, missed)).most_common(3)34print("names it misses most often:", ", ".join(f"{name} ({n})" for name, n in often))3536# What the model finds where no labelled span of the type its label maps to is: text it would replace for nothing.37wrong = sum(n - overlap for n, _, overlap in scores.precision.values())38print("the model's detections:", len(found))39print("of them, overlapping no labelled span of their type:", wrong)
python privacy-filter-on-the-corpus.pymodel: openai/privacy-filter
reviews measured: 436
labelled names: 120
names the model finds, exactly: 83
names the model misses: 37
names it misses most often: Marie (4), Liwia (4), Luigi (4)
the model's detections: 102
of them, overlapping no labelled span of their type: 0
On these reviews the model can't replace the list, and this data can't show what it adds, since every name in it is on the list. It finds 83 of the 120 labelled names exactly and misses 37, most often Marie (4), Liwia (4), Luigi (4), where the list misses none.
Of its 102 detections, 0 overlap no labelled span of their type. Where text names people no list holds, run both, measured this way first.
The text that comes out can still identify people. A detector misses spans, the text can identify someone without a name, and each record keeps a review ID that leads back to its author through learning.reviews. The GDPR's Recital 26 says: "Personal data which have undergone pseudonymisation, which could be attributed to a natural person by the use of additional information should be considered to be information on an identifiable natural person" (GDPR, Recital 26). So we call the corpus pseudonymized, never anonymized.
Working rule: Detect personal data in corpus text after the enforcement layer's masks, add the identifiers your platform holds about the people the text is about, and measure what's left against labelled data, by value, after every change to the detector. Prefer a short list of the names your text is about to every name you hold, since a long list matches ordinary words. Have a job that may read the names build the list, and record which list each corpus used. Measure a named-entity model the same way before you rely on it for names no list holds. Keep the list out of the corpus, and call the result pseudonymized.
用带密钥的哈希保持假名一致
检测到的姓名可以替换为占位符 <PERSON>,也可以替换为假名:假名是针对某一个姓名的标记,该姓名无论出现在何处,标记都相同。平台的评价只用名字称呼讲师,且没有两位讲师同名,因此这里一个标记代表一个人。假名保留了谁是谁的信息,所以两条称赞同一位讲师的评价仍然能体现这一点。
Presidio 的匿名器有一个 hash 运算符,自 2.2.361 版起,出于安全考虑,它默认为每次哈希随机加盐(Presidio anonymizer)。随机盐使每个哈希与下一个哈希无法关联,因此不能用作假名。要让相同的值得到相同的哈希,文档称"必须显式提供 salt 参数",其长度至少为 128 位(16 字节)。
语料库作业改用自己的带密钥哈希,通过 Presidio 的 custom 运算符实现,该运算符接受一个必须返回字符串的 lambda。lib.ai_data.pseudonym 在只有语料库作业持有的密钥下,对实体类型和经过大小写折叠的值计算 HMAC(使用 SHA-256),并保留八位十六进制数字。HMAC 是 RFC 2104 描述的为哈希加密钥的构造方法,其明确的目标之一是"基于对底层哈希函数的合理假设,对认证机制的强度给出一个易于理解的密码学分析"(RFC 2104)。Python 标准库中提供了它,即 hmac,所以作业没有理由用其他方式为哈希加密钥。
语料库中有两条评价提到了同一位讲师,用两种方式替换该姓名,可以看出哪种方式能保持两者的关联:
1"""Replaces the same instructor's name in two reviews by Presidio's hash, by the corpus's keyed hash and by the enforcement layer's mask. The prose's point rests on the Presidio hash lines and the keyed pseudonym line."""2from collections import Counter34from presidio_anonymizer import AnonymizerEngine5from presidio_anonymizer.entities import InvalidParamError, OperatorConfig6from sqlglot import exp78from lib import ai_data, enforce9from lib.catalog import open_catalog10from lib.enforce import Enforcer11from lib.platform import load1213# Two of the corpus's reviews that name the same instructor, with what the detector found in each.14layer = Enforcer(open_catalog(["crm.learners", "learning.courses", "learning.reviews"]))15engine = ai_data.detector()16found = [(row["body"], engine.analyze(row["body"], language="en")) for row in ai_data.read(layer)]17# the first two reviews whose only detected name is Patrick18named = [(text, spans) for text, spans in found if [text[s.start:s.end] for s in spans] == ["Patrick"]][:2]19print("reviews naming Patrick:", len(named))2021def replaced(operator: OperatorConfig) -> list[str]:22 """The value that replaced the name in each of the two reviews."""23 results = [AnonymizerEngine().anonymize(text, spans, {"PERSON": operator}) for text, spans in named]24 return [result.text[item.start:item.end] for result in results for item in result.items]2526# Presidio's hash salts each value at random, so the same name gets a different hash in each review27hashed = replaced(OperatorConfig("hash"))28assert hashed[0] != hashed[1]29print("Presidio's hash, the same name in two reviews gets one value:", hashed[0] == hashed[1])30# and hashing the same review again gives it a different hash too31rehashed = replaced(OperatorConfig("hash"))[0]32assert rehashed != hashed[0]33print("Presidio's hash, one review hashed twice gets one value:", rehashed == hashed[0])34# a fixed salt must be at least 16 bytes; this one is 10, so Presidio refuses it35try:36 replaced(OperatorConfig("hash", {"salt": "ten bytes!"}))37except InvalidParamError as error:38 print("a fixed salt of 10 bytes:", error)3940# the custom operator's lambda returns the keyed pseudonym as the replacement41keyed = replaced(OperatorConfig("custom", {"lambda": lambda value: ai_data.pseudonym("PERSON", value)}))42print("keyed pseudonyms:", ", ".join(keyed))43print("the name typed in lowercase:", ai_data.pseudonym("PERSON", "patrick"))44# the first word of each instructor's full name45first_names = Counter(name.split()[0] for name in load("crm.instructors").column("full_name").to_pylist())46shared = sum(n for n in first_names.values() if n > 1)47assert shared == 048print("instructors who share a first name with another:", shared)4950# What the enforcement layer's pseudonymize mask makes of the same name, run by the layer under its own key.51mask = enforce.masked("pseudonymize", exp.Literal.string("Patrick"), "VARCHAR").sql(enforce.DIALECT)52print("the enforcement layer's pseudonymize mask, on the same name:", layer.run(f"select {mask}").fetchall()[0][0])53# functions whose name mentions HMAC: the prose says DuckDB has none54hmac = layer.connection.sql("select count(*) from duckdb_functions() where function_name ilike '%hmac%'").fetchall()[0][0]55assert hmac == 056print("HMAC functions in DuckDB:", hmac)
python keyed-pseudonyms.pyreviews naming Patrick: 2
Presidio's hash, the same name in two reviews gets one value: False
Presidio's hash, one review hashed twice gets one value: False
a fixed salt of 10 bytes: Salt must be at least 16 bytes (128 bits). Provided salt is 10 bytes.
keyed pseudonyms: <PERSON_d14171be>, <PERSON_d14171be>
the name typed in lowercase: <PERSON_d14171be>
instructors who share a first name with another: 0
the enforcement layer's pseudonymize mask, on the same name: 23b3d8671d01840655296f752121aecc4f7991694260e5e1431d2b57c5e3d53f
HMAC functions in DuckDB: 0
Presidio 的 hash 在两条评价中给同一个姓名不同的值,并且同一条评价每次哈希也得到不同的值。如文档所述,Presidio 还会拒绝过短的固定盐,这里是 10 字节,并给出 Salt must be at least 16 bytes (128 bits). Provided salt is 10 bytes.。带密钥的哈希给两条评价都得到 <PERSON_d14171be>, <PERSON_d14171be>,小写输入的姓名也得到 <PERSON_d14171be>。
执行层的 pseudonymize 掩码由该层对同一姓名运行,得到 23b3d8671d01840655296f752121aecc4f7991694260e5e1431d2b57c5e3d53f。正如第 6 章所写,它是对该层密钥后接值计算的 SHA-256,而不是 HMAC,因为该层的掩码是 SQL,而 DuckDB 2.0.0 没有 HMAC 函数。两者是不同密钥下的不同构造,因此语料库的假名无法与该层展示给分析人员的假名连接。
一致性有两项代价。持有密钥和一份姓名列表的人可以算出每一个假名,正如第 6 章对电子邮件所展示的那样,所以密钥应保存在密钥管理器中,绝不放入语料库、代码仓库或日志。带密钥的假名还会把占位符会合并的评价区分开:96 个互不相同的文本,对比 21,下一节会对此进行度量。当文本的含义取决于区分不同的人时,例如工单中同时提到客服人员和学习者,我们建议使用带密钥的假名;当下游不需要区分时,使用占位符。
工作准则: 用只有作业持有的密钥通过 HMAC 生成假名,并为假名不得相互连接的每个语料库或受众各设一个单独的密钥。切勿在不加盐的情况下用 Presidio 的 hash 运算符生成假名,因为在其固定版本中每个哈希都是随机的。当两个人可能共用姓名中的某个词时,应对人员 ID 而不是姓名中的词做假名化;当下游不需要区分人员时,使用普通占位符。
这部分尚未翻译,因此以英文显示。
Find near-duplicates with MinHash
Once its names are pseudonymized, the corpus's next problem is repetition: templates, copies, a review pasted twice, the same sentence from many people. Repetition is a privacy problem as well as a quality one. Kandpal, Wallace and Raffel found that "a sequence that is present 10 times in the training data is on average generated ~1000 times more often than a sequence that is present only once" (Kandpal et al., 2022).
Near-copies need a measure of similarity. Broder defined the resemblance of two documents as the Jaccard similarity of their sets of shingles, the overlapping runs of characters or words in a text. He showed that sampling the smallest shingles under a random permutation gives an unbiased estimate of that resemblance (Broder, 1997). Taking the smallest shingle under each of many permutations gives a MinHash signature, which estimates the similarity of two texts without comparing their sets (Mining of Massive Datasets, section 3.3).
lib.ai_data.deduplicate removes exact copies first, by SHA-256 of the case-folded text with runs of white space made one. It compares what's left as sets of 5-character shingles with datasketch's MinHash and MinHashLSH, which returns candidate pairs, and confirms each candidate by its exact Jaccard similarity. A confirmed pair at 0.8 or more is a near-duplicate, unless it differs only in a name, and of each group of duplicates the record with the lowest review ID stays. The duplicate step runs on the pseudonymized reviews, and each stage reports what it removed:
1"""Runs the corpus's duplicate step on the pseudonymized reviews: exact copies, then near-duplicates by MinHash LSH, each confirmed exactly, and the records kept. The prose's point rests on the "of them, differing only in a name" and "near-duplicates removed" lines."""2from collections import Counter34from lib import ai_data5from lib.catalog import open_catalog6from lib.enforce import Enforcer78# The reviews the corpus job reads, pseudonymized, then deduplicated: exact copies, then near-duplicates.9# the reviews read, each text pseudonymized; the second value counts the replacements10rows, _ = ai_data.read_pseudonymized(Enforcer(open_catalog(["crm.learners", "learning.courses", "learning.reviews"])))11# groups of reviews that stand for one record, and how many each step removed12groups, removed = ai_data.duplicate_groups(rows)13print("reviews read:", len(rows))14# one review per distinct normalized text: the one with the lowest review ID15print("distinct texts:", len(ai_data.exact_distinct(rows)))16for step, count in removed.items():17 print(f"{step}:", count)18print("records kept:", len(groups))19print("share of the reviews read that duplicate another:", f"{1 - len(groups) / len(rows):.0%}")2021# identical texts only, before normalization22text, copies = Counter(row["body"] for row in rows).most_common(1)[0]23print("the most repeated text:", text)24print("its copies:", copies)2526# A pair MinHash finds and the exact comparison confirms, and whether it differs only in a name.27distinct = [row["body"] for row in ai_data.exact_distinct(rows)]28# the first confirmed pair, by position among the distinct texts29i, j = sorted(ai_data.near_duplicates(distinct).confirmed)[0]30print("a near-duplicate pair:", distinct[i], "|", distinct[j])31print("its Jaccard similarity:", f"{ai_data.jaccard(distinct[i], distinct[j]):.2f}")32# equal once each pseudonym is a placeholder: the pair differs only in which names it holds33print("it differs only in a name:", ai_data.placeholders(distinct[i]) == ai_data.placeholders(distinct[j]))3435# Each record kept stands for its group's reviews, under the review ID and course of the first.36# a group whose reviews come from more than one course37several = sum(len({row["course_id"] for row in group}) > 1 for group in groups)38print("records that stand for reviews of several courses:", several)
python corpus-duplicates.pyreviews read: 436
distinct texts: 96
exact duplicates removed: 340
near-duplicate pairs from LSH: 381
near-duplicate pairs confirmed: 8
of them, differing only in a name: 8
near-duplicates removed: 0
records kept: 96
share of the reviews read that duplicate another: 78%
the most repeated text: Clear explanations and good exercises.
its copies: 56
a near-duplicate pair: Some videos are out of date. <PERSON_85783bac> explains things well. Happy to talk about it, write to me at [email]. | Some videos are out of date. <PERSON_967f9ae8> explains things well. Happy to talk about it, write to me at [email].
its Jaccard similarity: 0.81
it differs only in a name: True
records that stand for reviews of several courses: 23
Most of the corpus repeats itself. Of the 436 reviews read, 78% repeat another: 340 are exact copies, and only 96 texts are distinct. That's mostly the generator's doing, since it builds reviews from a handful of fixed sentences, so measure the share on your own text before training on it.
The most repeated text, "Clear explanations and good exercises.", alone appears 56 times.
MinHash then confirms 8 pairs at 0.8 or more, such as two reviews at a similarity of 0.81 that differ only in their instructor's pseudonym. Of those pairs, 8 differ only in a name, which the platform keeps apart, so near-duplicates remove 0.
Why pairs that differ only in a name stay apart comes down to what one threshold means at different lengths. Compare the corpus's reviews with the names as written, with keyed pseudonyms and with plain placeholders, then split every pair of distinct texts by whether it differs only in a name:
1"""Counts near-duplicate pairs of the corpus's reviews with names as written, with keyed pseudonyms and with placeholders, then splits the pairs that differ only in a name by length. The prose's point rests on the pair counts and the length ranges."""2from itertools import combinations34from lib import ai_data5from lib.catalog import open_catalog6from lib.enforce import Enforcer78# The reviews the corpus job reads, three ways: names as written, keyed pseudonyms, and plain placeholders.9layer = Enforcer(open_catalog(["crm.learners", "learning.courses", "learning.reviews"]))10keyed, _ = ai_data.read_pseudonymized(layer)11# the same reviews with each pseudonym replaced by a plain placeholder, as the third way12ways = {"names as written": ai_data.read(layer), "keyed pseudonyms": keyed,13 "placeholders": [row | {"body": ai_data.placeholders(row["body"])} for row in keyed]}14# the similarity at which a pair is a near-duplicate15threshold = ai_data.DUPLICATES["threshold"]16for way, rows in ways.items():17 texts = [row["body"] for row in ai_data.exact_distinct(rows)]18 print(f"{way}, distinct texts:", len(texts))19 print(f"{way}, pairs at {threshold} or more:", len(ai_data.near_duplicates(texts).confirmed))2021# Every pair of distinct pseudonymized texts, split by whether the pair differs only in a name.22# distinct texts only, so each text appears once in the pairs23texts = [row["body"] for row in ai_data.exact_distinct(keyed)]24name_only, other = [], []25for a, b in combinations(texts, 2):26 # similarity comes first, so min and max over these tuples order the pairs by it27 pair = (ai_data.jaccard(a, b), len(a), a, b)28 (name_only if ai_data.placeholders(a) == ai_data.placeholders(b) else other).append(pair)29print("pairs that differ only in a name:", len(name_only))30# the lowest and highest similarity among the name-only pairs31print("their Jaccard similarity:", f"{min(name_only)[0]:.2f} to {max(name_only)[0]:.2f}")32for label, lengths in [(f"below {threshold}", [n for s, n, _, _ in name_only if s < threshold]),33 (f"at {threshold} or more", [n for s, n, _, _ in name_only if s >= threshold])]:34 print(f"their length in characters, {label}:", f"{min(lengths)} to {max(lengths)}")35# the most similar pair among those that differ in more than a name36similarity, _, a, b = max(other)37print("the most similar pair that differs in more than a name:", f"{similarity:.2f}")38print("that pair:", a, "|", b)
python names-and-near-duplicates.pynames as written, distinct texts: 96
names as written, pairs at 0.8 or more: 10
keyed pseudonyms, distinct texts: 96
keyed pseudonyms, pairs at 0.8 or more: 8
placeholders, distinct texts: 21
placeholders, pairs at 0.8 or more: 0
pairs that differ only in a name: 459
their Jaccard similarity: 0.68 to 0.83
their length in characters, below 0.8: 66 to 78
their length in characters, at 0.8 or more: 116 to 119
the most similar pair that differs in more than a name: 0.66
that pair: Clear explanations and good exercises. Happy to talk about it, write to me at [email]. | Clear explanations and good exercises. <PERSON_3026fb67> explains things well. Happy to talk about it, write to me at [email].
Which pairs reach 0.8 depends on what stands in for each name. With the names as written, 10 pairs reach 0.8, and with keyed pseudonyms 8, since a pseudonym is longer than most first names and so a larger share of a short review. With plain placeholders, the 21 distinct texts hold 0.
Among the pseudonymized texts, 459 pairs differ only in which pseudonym they hold, at similarities from 0.68 to 0.83, and the threshold splits them by length alone: those below 0.8 are 66 to 78 characters long, and those at 0.8 or more are 116 to 119 characters long.
So the task, not the threshold, decides whether such reviews are duplicates: for a model that learns how learners describe courses, they are, and for one that learns what learners say about each instructor, they aren't. The platform's assistant answers about each course and its instructor, so name_only_pairs_are_duplicates = false in governance/ai_data.toml, and deduplicate never merges two texts that are the same once their pseudonyms are placeholders. The threshold decides the other pairs. The most similar of those, at 0.66, is a review with and without its sentence about the instructor: two different reviews. The platform keeps 0.8, above that pair with a margin.
Deduplicating on text folds reviews of different courses into one record, under the review ID and course of its first review, so the corpus keeps 96 records, and 23 of them stand for reviews of several courses. For training that's the point, since the model learns from the text. For grounding an answer about one course it costs recall, which the provenance section takes up.
Working rule: Deduplicate training text in two steps, exact copies by a hash of the normalized text and then near-duplicates by MinHash, each confirmed by its exact similarity, and report how much each step removed. Pseudonymize first, so that no name reaches the comparison or the sketches and the comparison can tell which pairs differ only in a name. Keep those pairs apart when the text's meaning depends on whom it names, and compare on placeholders when it doesn't. Choose the threshold by reading the pairs close above and below it in your own text.
这部分尚未翻译,因此以英文显示。
Tune LSH for recall, then confirm every pair
MinHash signatures make each comparison cheap; locality-sensitive hashing (LSH) avoids most comparisons. LSH cuts each signature into b bands of r rows, and two texts become a candidate pair when all the rows of any one band match. Leskovec, Rajaraman and Ullman derive the chance that a pair with Jaccard similarity s becomes a candidate pair (Mining of Massive Datasets, chapter 3):
P(candidate) = 1 − (1 − sʳ)ᵇ
It's an S-shaped curve: a pair well below the threshold is rarely a candidate, a pair well above it nearly always, and near the threshold it's close to a coin toss. The curve's approximate threshold is (1/b)^(1/r). datasketch's documentation warns of both kinds of error: "There will be false positives - sets that do not satisfy your threshold but returned, and false negatives - qualifying sets that are not returned" (datasketch, MinHash LSH).
With 96 distinct texts, the platform can afford to compare every pair exactly, 4,560 pairs, and that gives the exact pairs to measure LSH against. LSH sees only similarity, so it must return every pair at 0.8, whatever the name rule then decides. Score LSH against the exact comparison, tuned for 0.8 and tuned lower, each under datasketch's default hashing and under the hashing it used before its 2.0.0 release:
1"""Measures the pairs LSH misses against every pair compared exactly, under two tunings and two hashing schemes, and shows what datasketch does with sketches of two schemes. The prose's point rests on the "missed" lines."""2from itertools import combinations34from datasketch import MinHashLSH56from lib import ai_data7from lib.catalog import open_catalog8from lib.enforce import Enforcer910rows, _ = ai_data.read_pseudonymized(Enforcer(open_catalog(["crm.learners", "learning.courses", "learning.reviews"])))11texts = [row["body"] for row in ai_data.exact_distinct(rows)]12# the similarity at which a pair is a near-duplicate13threshold = ai_data.DUPLICATES["threshold"]1415# Every pair compared exactly: possible at this size, and the truth LSH is measured against.16pairs = {(i, j) for i, j in combinations(range(len(texts)), 2) if ai_data.jaccard(texts[i], texts[j]) >= threshold}17print("pairs compared exactly:", f"{len(texts) * (len(texts) - 1) // 2:,}")18print(f"pairs at a Jaccard similarity of {threshold} or more:", len(pairs))1920found = {}21for scheme in ["affine32", "legacy"]:22 # LSH tuned for the threshold, and for the lower candidate threshold23 for tuned_for in [threshold, ai_data.DUPLICATES["candidate_threshold"]]:24 # the bands and rows LSH chose, its candidates, and the candidates the exact check confirms25 found[scheme, tuned_for] = result = ai_data.near_duplicates(texts, threshold, tuned_for, scheme)26 setting = f"{scheme}, LSH tuned for {tuned_for}"27 print(f"{setting}, bands:", result.bands)28 print(f"{setting}, rows per band:", result.rows)29 print(f"{setting}, candidate pairs:", len(result.candidates))30 print(f"{setting}, confirmed:", len(result.confirmed))31 # pairs the exact comparison found that LSH never proposed as candidates32 missed = len(pairs - result.candidates)33 print(f"{setting}, missed:", missed)34 if tuned_for != threshold:35 assert missed == 0 # tuned lower, LSH finds every pair under either scheme3637# The chance that a pair of similarity s becomes a candidate, with b bands of r rows: 1 - (1 - s^r)^b.38# the bands and rows LSH chose for the threshold, as the formula uses them39b, r = found["affine32", threshold].bands, found["affine32", threshold].rows40for s in [0.8, 0.9]:41 print(f"chance of a candidate at similarity {s}, LSH tuned for {threshold}:", f"{1 - (1 - s ** r) ** b:.2f}")42# the S-curve's approximate threshold, which the prose compares with the threshold asked for43print("the approximate threshold, (1/b)^(1/r):", f"{(1 / b) ** (1 / r):.2f}")4445# What datasketch does with a sketch made under the other scheme.46# an index takes sketches of one scheme, so the legacy sketch below is refused47index = MinHashLSH(threshold=threshold, num_perm=ai_data.DUPLICATES["num_perm"])48index.insert(0, ai_data.minhash(texts[0], "affine32"))49try:50 index.insert(1, ai_data.minhash(texts[1], "legacy"))51except ValueError as error:52 print("a legacy sketch in an index of affine32 sketches:", error)
python lsh-misses-near-duplicates.pypairs compared exactly: 4,560
pairs at a Jaccard similarity of 0.8 or more: 8
affine32, LSH tuned for 0.8, bands: 9
affine32, LSH tuned for 0.8, rows per band: 13
affine32, LSH tuned for 0.8, candidate pairs: 36
affine32, LSH tuned for 0.8, confirmed: 1
affine32, LSH tuned for 0.8, missed: 7
affine32, LSH tuned for 0.6, bands: 18
affine32, LSH tuned for 0.6, rows per band: 7
affine32, LSH tuned for 0.6, candidate pairs: 381
affine32, LSH tuned for 0.6, confirmed: 8
affine32, LSH tuned for 0.6, missed: 0
legacy, LSH tuned for 0.8, bands: 9
legacy, LSH tuned for 0.8, rows per band: 13
legacy, LSH tuned for 0.8, candidate pairs: 52
legacy, LSH tuned for 0.8, confirmed: 6
legacy, LSH tuned for 0.8, missed: 2
legacy, LSH tuned for 0.6, bands: 18
legacy, LSH tuned for 0.6, rows per band: 7
legacy, LSH tuned for 0.6, candidate pairs: 406
legacy, LSH tuned for 0.6, confirmed: 8
legacy, LSH tuned for 0.6, missed: 0
chance of a candidate at similarity 0.8, LSH tuned for 0.8: 0.40
chance of a candidate at similarity 0.9, LSH tuned for 0.8: 0.93
the approximate threshold, (1/b)^(1/r): 0.84
a legacy sketch in an index of affine32 sketches: MinHash scheme 'legacy' does not match scheme 'affine32' of the MinHash previously given to this index
Tuned for 0.8, datasketch's default LSH returns 36 candidate pairs, misses 7 of the 8 pairs that reach 0.8, and so confirms only 1 of them.
The formula says why. With the 9 bands of 13 rows datasketch chose, a pair at similarity 0.8 becomes a candidate with a probability of 0.40, one at 0.9 with 0.93, and the approximate threshold is 0.84, above the threshold asked for.
Under the default scheme, tuned for 0.6, LSH finds every pair and returns 381 candidates. Each candidate costs one exact comparison, against 4,560 for comparing every pair, and the exact comparison discards every pair below 0.8. So the platform tunes LSH for 0.6 and confirms at 0.8, both settings in governance/ai_data.toml.
datasketch's 2.0.0 release made affine32 the default permutation scheme of MinHash, and its release notes say that hash values differ from earlier versions, so they advise rebuilding persisted sketches and LSH indexes, or passing MinHash(..., scheme="legacy"). With the legacy scheme, LSH tuned for 0.8 returns 52 candidates and confirms 6, so a count recorded under one scheme isn't reproduced under the other.
Tuned for 0.6, both schemes find all 8 pairs. With high recall and an exact check, the result depends on the similarity, not on the hash functions.
datasketch refuses to mix schemes in one index, with MinHash scheme 'legacy' does not match scheme 'affine32' of the MinHash previously given to this index, but it won't stop you comparing old counts with new ones. The book's examples pin datasketch at 2.0.0 and write scheme = "affine32" into the settings, so an upgrade can't change a recorded count without a reviewed change.
Working rule: Tune LSH below your similarity threshold, so that it misses few pairs, and confirm every candidate by its exact similarity. Measure LSH's misses against an exhaustive comparison on a sample of your corpus whenever its settings, its library or the text change. Pin the MinHash library, write its hashing scheme into your settings, and never compare counts or stored sketches made under different schemes.
为每条记录携带来源信息
无法追溯到来源的记录,就无法被检查、更正或删除。记录层面的来源信息回答三个问题:它来自哪张表,来自哪一行,以及由哪个作业的哪一次运行把它放进了语料库。第 4 章为平台提供了最后一部分:作业的每次运行都会发送 OpenLineage 事件,其中的运行 ID 由运行时间和作业名称派生,列级血缘则从作业的 SQL 中解析得到。
lib.ai_data.build 写入每条记录时,会在评论自身的列旁边附上它的来源信息:source_table、review_id、job 和 run_id。它通过 lib.lineage.record 发送该作业的运行事件。这里有一个选择很重要:血缘是从执行层实际运行的查询中解析的,其中包含同意过滤条件和掩码,而不是从 pipelines/review_corpus.sql 中的 SQL 解析的。构建语料库,然后沿着一条记录追溯到它的那次运行:
1"""Builds the corpus and follows one record back to its source row, its job and the run that wrote it, then checks that run's lineage against the query as written. The prose's point rests on the "rows filtered by" and "body" lines."""2from lib import ai_data, lineage, pipelines3from lib.catalog import open_catalog45catalog = open_catalog(["crm.learners", "learning.courses", "learning.reviews"])6built = ai_data.build(catalog)7# the second record, whose text holds a pseudonym8record = built.records.to_pylist()[1]9print("a record's text:", record["body"])10for column in ["source_table", "review_id", "job"]:11 print(f"its {column}:", record[column])1213# The run that wrote it, from the lineage events: the query as the enforcement layer ran it.14# the job's COMPLETE event, which carries the run's lineage15complete = [e for e in lineage.read_events(built.folder / "events.jsonl") if e["eventType"] == "COMPLETE"][0]16same_run = record["run_id"] == complete["run"]["runId"]17assert same_run18print("its run_id is the COMPLETE event's run:", same_run)19print("the run's inputs:", ", ".join(dataset["name"] for dataset in complete["inputs"]))20# column lineage: the inputs that filter the rows, and how each field is transformed21facet = complete["outputs"][0]["facets"]["columnLineage"]22# a FILTER transformation marks an input whose values decide which rows the output holds23filters = [f"{i['name']}.{i['field']}" for i in facet["dataset"] if any(t["subtype"] == "FILTER" for t in i["transformations"])]24print("rows filtered by, as run:", ", ".join(filters))25# the lineage of the review text, from the reviews table26text = [i for i in facet["fields"]["body"]["inputFields"] if (i["name"], i["field"]) == ("learning.reviews", "body")]27# IDENTITY means copied unchanged; TRANSFORMATION means the value was changed on its way28print("body, as run:", text[0]["transformations"][0]["subtype"])29# the same lineage parsed from the SQL as written, with no enforcement rewrite30fields, rows = lineage.column_lineage(pipelines.sql(ai_data.BUILD["job"]), lineage.schema(catalog))31print("rows filtered by, as written:", ", ".join(i.source for i in rows if i.subtype == "FILTER") or "nothing")32print("body, as written:", fields["body"][0].subtype)3334# Back from every record to its review's author, with the platform's own access, against today's answers.35# each record's review ID, checked against the author's consent today, not at build time36ids = built.records.column("review_id").to_pylist()37print("records whose author consented:", f"{ai_data.consenting(catalog, ids)} of {len(ids)}")
python corpus-provenance.pya record's text: Too fast in the second half. <PERSON_85783bac> explains things well.
its source_table: learning.reviews
its review_id: 18
its job: platform/review_corpus
its run_id is the COMPLETE event's run: True
the run's inputs: crm.learners, learning.courses, learning.reviews
rows filtered by, as run: crm.learners.ai_training_consent, learning.reviews.learner_id
body, as run: TRANSFORMATION
rows filtered by, as written: nothing
body, as written: IDENTITY
records whose author consented: 96 of 96
追溯一条记录,可以看出血缘必须记录哪些内容。记录“Too fast in the second half. <PERSON_85783bac> explains things well.”来自 learning.reviews,是评论 18,由作业 platform/review_corpus 写入,它的 run_id 就是该作业 COMPLETE 事件所对应的那次运行。
那次运行读取了 crm.learners, learning.courses, learning.reviews,其血缘表明,这些行是由 crm.learners.ai_training_consent, learning.reviews.learner_id 过滤的,这是决定哪些行存在的INDIRECT输入,并且 body 是评论文本的 TRANSFORMATION。
如果按流水线中原样书写的 SQL 来解析,同一作业的行则由 nothing 过滤,body 是一份 IDENTITY 副本。这会表明个人文本被原样复制,且不涉及任何同意列。实际运行的查询才是真正发生的事情,所以血缘记录的是它。
来源信息使语料库在事后可以被检查。将每条记录通过 learning.reviews 连接回其作者,可以找出 96 of 96 条作者已同意的记录,平台可以对任何旧的构建版本,按今天的答案运行这项检查。
来源信息还使检索能够尊重每位读者的权限。OWASP 要求基于语言模型的应用“实施细粒度的访问控制,以及具备权限感知能力的向量和嵌入存储”(OWASP LLM08:2025)。平台根据这个语料库构建的索引来回答某个人的提问,索引会将每条记录的 review_id 与其向量一并保存。在检索到的记录送达模型之前,检索层会以被回答者的身份询问执行层,这些评论中哪些是此人可以阅读的,方式与第 6 章运行助手查询的方式相同,并丢弃其余的记录。这样,讲师只会得到来自自己课程的记录。一条代表多门课程评论的记录携带的是第一条评论的 ID,因此该检查会以失败即拒绝的方式处理:对于这类记录中的每一条(共 23 条),它可能会隐藏此人本可阅读的文本,但绝不会显示此人无权阅读的文本。
**工作准则:**为每条记录提供其来源表、所在行的 ID,以及写入它的作业和运行,并按查询实际运行时的样子记录作业的血缘,包括其中的过滤条件和掩码。通过这些来源信息,对照今天的同意状态检查旧的构建版本。在模型看到任何记录之前,按读者本人对每条记录来源行的权限判定过滤每一次检索。
这部分尚未翻译,因此以英文显示。
Generate the dataset card from the run
A dataset card tells whoever trains or audits a model what's in its data, where it came from and what was done to it. Croissant is MLCommons' metadata format for machine-learning datasets (Akhtar et al., 2024). Its 1.1 specification makes a card checkable: "Croissant datasets must declare that they conform to the versioned schema, e.g. http://mlcommons.org/croissant/1.1", and every dataset must give a description, licence, name, URL, creator and publication date. For provenance, the specification recommends the W3C PROV Ontology (PROV-O) (Croissant 1.1). Its responsible-AI vocabulary adds properties such as rai:dataCollection and rai:personalSensitiveInformation (Croissant RAI 1.0). A card written by hand describes the data someone meant to build; a card generated by the run describes the data it built.
lib.ai_data.card generates the platform's card with mlcroissant 1.1.1, MLCommons' Python library for Croissant. The run supplies the counts, the corpus file's SHA-256, a field for every column, and three PROV-O properties: the tables the pipeline's own SQL reads, which the values come from, as prov:wasDerivedFrom; the table the enforcement layer's rewrite adds, which only decides which rows exist, as prov:wasInfluencedBy; and the run, as prov:wasGeneratedBy. mlcroissant writes them through its extra_properties argument but adds no prov prefix to the card's context, so the module adds it. The creator is the owner of ai.review_corpus in governance/metadata.toml. The licence, the name, URL, description and use cases come from governance/ai_data.toml, and lib.ai_data.card_problems is the platform's own check of the result. Generate the card, first with mlcroissant's default conformsTo:
1"""Generates the corpus's Croissant card twice, first with mlcroissant's default conformsTo and then as the platform writes it, and runs the platform's own check on it. The prose's point rests on the "conformsTo" lines and the "problems" line."""2from lib import ai_data3from lib.catalog import open_catalog45built = ai_data.build(open_catalog(["crm.learners", "learning.courses", "learning.reviews"]))67# mlcroissant's default, then the card the pipeline writes, with Croissant 1.1 written out.8# an empty list leaves conformsTo to mlcroissant, which writes its own default9default, _ = ai_data.card(built, conforms_to=[])10print("conformsTo, mlcroissant's default:", default["conformsTo"])11# the card as the platform writes it: Croissant 1.1 and the RAI vocabulary, written out12card, warnings = ai_data.card(built)13ai_data.write_card(card, built.folder)14print("conformsTo, written out:", ", ".join(card["conformsTo"]))15print("datePublished, the run's:", card["datePublished"])16# the build's date as the major number: a nightly build may remove records17print("version, the run's:", card["version"])18print("the sources the card names:", ", ".join(source["@id"] for source in card["prov:wasDerivedFrom"]))19print("the tables that decided which rows, as the card names them:",20 ", ".join(table["@id"] for table in card["prov:wasInfluencedBy"]))21print("the activity the card names:", card["prov:wasGeneratedBy"]["prov:label"])22# the card names every column of the corpus as a field23print("fields in the record set:", ", ".join(f["name"] for f in card["recordSet"][0]["field"]))24for line in card["rai:dataPreprocessingProtocol"] + card["rai:personalSensitiveInformation"]:25 print("-", line)2627# What mlcroissant reports, and what the platform's own check of the card reports.28# the message after mlcroissant's severity tag29print("mlcroissant's warnings:", len(warnings))30for warning in warnings:31 print("- mlcroissant:", warning.split("] ", 1)[1])32# the platform's own check: the library's validator doesn't report every requirement33problems = ai_data.card_problems(card, built.folder)34print("problems the book's check finds:", len(problems))35for problem in problems:36 print("-", problem)
python corpus-dataset-card.pyconformsTo, mlcroissant's default: http://mlcommons.org/croissant/1.0
conformsTo, written out: http://mlcommons.org/croissant/1.1, http://mlcommons.org/croissant/RAI/1.0
datePublished, the run's: 2026-10-01T02:00:00Z
version, the run's: 20261001.0.0
the sources the card names: iceberg://platform/learning.courses, iceberg://platform/learning.reviews
the tables that decided which rows, as the card names them: iceberg://platform/crm.learners
the activity the card names: platform/review_corpus
fields in the record set: review_id, course_id, course_title, rating, body, source_table, job, run_id
- Read 436 of 3,000 reviews, filtered on crm.learners.ai_training_consent, learning.reviews.learner_id.
- Replaced personal data with keyed pseudonyms (PERSON 120).
- Removed 340 exact duplicates and 0 near-duplicates (MinHash LSH, Jaccard similarity of 0.8 or more, confirmed exactly; 8 pairs that differ only in a name kept apart).
- Kept 96 records, each with its source table, review ID, job and run.
- Values replaced by keyed pseudonyms: PERSON 120.
- Emails and phone numbers replaced by [email] and [phone] by the enforcement layer's redact_contacts mask.
- Names found with a list of 106 words from crm.instructors, built by the governance team's job governance/instructor_names; the list isn't part of the corpus.
- Measured against labelled spans: 0 of 139 labelled values left in the text.
mlcroissant's warnings: 1
- mlcroissant: Property "http://mlcommons.org/croissant/citeAs" is recommended, but does not exist.
problems the book's check finds: 1
- creator: missing
Left to its default, mlcroissant writes http://mlcommons.org/croissant/1.0, although its own LATEST_VERSION is 1.1 (mlcroissant source), so a card would declare a version it wasn't written for. The platform writes http://mlcommons.org/croissant/1.1, http://mlcommons.org/croissant/RAI/1.0.
The card's datePublished, 2026-10-01T02:00:00Z, is the build's time, because the platform publishes a card only with the build it describes. Its version, 20261001.0.0, takes the build's date as its major number. Croissant recommends MAJOR.MINOR.PATCH, where a new major version means "the existing data has been changed (edited, removed or shuffled records across splits)", and any nightly build may remove records.
The card names the sources iceberg://platform/learning.courses, iceberg://platform/learning.reviews, the influence iceberg://platform/crm.learners and the activity platform/review_corpus, and lists preprocessing steps with the run's counts.
The warnings mlcroissant reports, 1 in all, are about recommended properties, here citeAs, not about the problem the platform's check finds: creator is missing. ai.review_corpus has no owner in the metadata file, the gap chapter 2 found, so the card stays unpublished until the table has an owner. Chapter 10 reports the same gap on every run, as a finding already on main.
mlcroissant can also read a card back, with mlc.Dataset, but at 1.1.1, with rdflib 7.6.0 and pandas 3.1.1, reading emits a DeprecationWarning from rdflib and a Pandas4Warning, and returns text fields as bytes. So the book's examples only write cards, and check them with the platform's own code.
Working rule: Generate the card in the run that builds the data, with every count, hash, source and version taken from that run, and write the format's version out. Check the card with your own code for every property the specification requires, since the library's validator doesn't report them all, and don't publish or train on a corpus whose card fails.
引用《欧盟人工智能法》关于训练数据的规定
欧盟《人工智能法》(Regulation (EU) 2024/1689)在多处涉及训练数据。本节引用其中两处,即第 10 条和第 53(1)(d) 条,我们对二者均不作解释。
对于高风险人工智能系统,第 10(2) 条规定,训练、验证和测试数据集“应遵循数据治理和管理实践”,并列出了这些实践所涉及的内容。其中包括数据收集过程和数据来源(涉及个人数据时还包括收集的原始目的),以及标注、标记、清洗、更新、充实和聚合等数据准备工作(AI Act)。
第 10(3) 条要求数据集“相关,具有足够的代表性,并在最大可能范围内,鉴于预期目的而无错误且完整”(AI Act)。
Regulation (EU) 2026/1744,即关于人工智能的《数字综合法案》(Digital Omnibus on AI),于 2026 年 7 月 24 日公布,保留上述所引条文不变。该法案替换了第 10 条的第 1 款和第 6 款,删除了第 5 款,并插入了第 4a 条“为偏差检测和纠正而处理特殊类别个人数据”。它还规定了第三章第 1 至 3 节(其中包括第 10 条)的适用时间:对于依据第 6(2) 条和附件 III 归类的高风险人工智能系统,自 2027 年 12 月 2 日起适用;对于依据第 6(1) 条和附件 I 归类的系统,自 2028 年 8 月 2 日起适用(Regulation (EU) 2026/1744)。
对于通用人工智能模型的提供者,第 53(1)(d) 条要求其“按照人工智能办公室提供的模板,编制并公开发布一份关于通用人工智能模型训练所用内容的足够详细的摘要”。欧盟委员会已将该模板以 C(2025) 8311 的编号公布(Commission template,PDF)。
其说明性通知指出,依据通知中所列的欧盟委员会通用人工智能模型指南,对已投放欧盟市场的模型进行修改的下游实体,可能成为所生成模型的提供者。在这种情况下,该修改实体只需报告用于修改的训练内容。
该法的附件 III 题为“第 6(2) 条所指的高风险人工智能系统”,其第 3 点“教育和职业培训”包括“(b) 拟用于评估学习成果的人工智能系统,包括这些成果被用于引导各级教育和职业培训机构中自然人的学习过程的情形”(AI Act)。我们引用这一条,是因为该平台是教育平台,仅此而已,不再展开。该平台的任何系统是否属于第 6 条所称的高风险系统,是另一个问题,本章的代码均不依赖于这一问题的答案。
**工作准则:**引用经修订后的法律文本,来源为《官方公报》,并附上链接和你阅读的日期,因为《综合法案》同时改动了条文和日期。不要把法律写进流水线的逻辑中:卡片上的事实应由运行结果生成,并由负责解答法律问题的人来决定适用哪些条文。
生产环境中会出什么问题
语料库不会发出任何预警。模型完成训练,模型卡随之发布,问题则在之后才浮现:有的出现在模型的输出里,有的出现在一个无人能够回答的请求中。
这部分尚未翻译,因此以英文显示。
A learner withdraws after the build
Like marketing's list in chapter 7, a corpus records who had consented on the day it was built. Take the corpus that lib.ai_data.build writes in the nightly run of Oct 1, the Oct 1 build, and withdraw the consent of the author of one of its records. Then rebuild the corpus in the same folder the next night, without generating its card again, and check each build against today's consent, as a training run does before it starts:
1"""Withdraws one author's consent after a build, rebuilds the corpus in the same folder, and checks both builds against today's consent and the card against the file. The prose's point rests on the "records whose author consents today" and "problems" lines."""2from datetime import timedelta34import pyarrow.compute as pc5import pyarrow.parquet as pq67from lib import ai_data, lineage8from lib.catalog import open_catalog9from lib.platform import load1011catalog = open_catalog(["crm.learners", "learning.courses", "learning.reviews"])12first = ai_data.build(catalog) # the nightly run of Oct 1, and its card13card, _ = ai_data.card(first)14ai_data.write_card(card, first.folder)1516# The next day, the author of one of the corpus's records withdraws their consent to AI training.17# review ID to the learner who wrote it, from the reviews table18authors = dict(zip(*load("learning.reviews").select(["review_id", "learner_id"]).to_pydict().values()))19# the author of the Oct 1 build's second record20learner = authors[first.records.column("review_id")[1].as_py()]21learners = catalog.load_table("crm.learners")22rows = learners.scan().to_arrow()23# that learner's answer becomes False; every other row keeps its answer24consent = pc.if_else(pc.equal(rows["learner_id"], learner), False, rows["ai_training_consent"])25learners.overwrite(rows.set_column(rows.schema.get_field_index("ai_training_consent"), "ai_training_consent", consent))2627# The next nightly run rebuilds the corpus in the same folder, and nobody generates the card again. A training run28# checks its build against today's consent before it starts.29# the same folder: the corpus file is replaced, so the first card no longer matches it30second = ai_data.build(catalog, first.folder, lineage.NIGHTLY + timedelta(days=1))31for name, built in [("Oct 1 build", first), ("Oct 2 build", second)]:32 ids = built.records.column("review_id").to_pylist()33 # records in each build whose author is the learner who withdrew34 print(f"{name}, records from the learner who withdrew:", sum(authors[i] == learner for i in ids))35 print(f"{name}, records whose author consents today:", f"{ai_data.consenting(catalog, ids)} of {len(ids)}")36different_runs = first.run_id != second.run_id37assert different_runs38print("the two builds are different runs:", different_runs)39# the legal team's placeholder answer, from governance/ai_data.toml40print("a model made from the Oct 1 build, the legal team's answer:", ai_data.LEGAL["withdrawal"]["models_made_before"])4142# the card's last preprocessing step, which states the Oct 1 counts43print("the Oct 1 card says:", card["rai:dataPreprocessingProtocol"][-1])44print("records in the corpus file now:", pq.read_metadata(first.folder / ai_data.CORPUS_FILE).num_rows)45# the platform's check of the Oct 1 card against the corpus file as it is now46problems = ai_data.card_problems(card, first.folder)47print("problems with the Oct 1 card, checked against the corpus now:", len(problems))48for problem in problems:49 print("-", problem)
python consent-withdrawn-after-build.pyOct 1 build, records from the learner who withdrew: 1
Oct 1 build, records whose author consents today: 95 of 96
Oct 2 build, records from the learner who withdrew: 0
Oct 2 build, records whose author consents today: 95 of 95
the two builds are different runs: True
a model made from the Oct 1 build, the legal team's answer: undecided
the Oct 1 card says: Kept 96 records, each with its source table, review ID, job and run.
records in the corpus file now: 95
problems with the Oct 1 card, checked against the corpus now: 2
- creator: missing
- review_corpus.parquet: its SHA-256 isn't the one the card states
The Oct 1 build holds 1 record by that learner, and the Oct 2 build holds 0, so the next corpus leaves the learner out. The two builds are different runs, with their own run IDs. The next training run has to leave the learner out too, and so does the next index.
Checked through provenance against today's answers, the Oct 1 build has 95 of 96 records whose author consents, so a training run that makes the check refuses it. The Oct 2 build passes with 95 of 95.
What was made from the Oct 1 build before the withdrawal keeps the review: a copy of its file, an index built from it, a model fine-tuned on it. The European Data Protection Board considers that "AI models trained with personal data cannot, in all cases, be considered anonymous" (EDPB Opinion 28/2024).
Article 7(3) of the GDPR gives the right to withdraw consent at any time. It says that withdrawing doesn't affect the lawfulness of processing based on consent before the withdrawal, that the person must be told of this right before giving consent, and that withdrawing must be no harder than giving it (GDPR). Chapter 7 quotes the first sentences of the same article and says what a withdrawal changes in the platform's code.
The code's part is to find, through provenance, every build that holds the record, and, by run ID, every model and index made from those builds. It then reads what happens to them from [legal.withdrawal] in governance/ai_data.toml. The legal team's answer for a model made from the Oct 1 build is undecided, a placeholder under which the job lists such models for the legal team and changes nothing. If the answer is that a model must not keep the review, the one way to remove it that we can vouch for is to train the model again on a build without it.
The card failed too, without a sound. Its last preprocessing step still reads "Kept 96 records, each with its source table, review ID, job and run." The file holds 95 records, and the platform's check finds 2 problems: the missing creator, and a file whose SHA-256 isn't the one the card states. A card that isn't generated again with its data describes another dataset.
Working rule: Record which build, by run ID, every model and index was made from. Before every training run, check the build's records against today's consent and the file's hash against its card, and refuse a build that fails either. When a learner withdraws, find the builds that hold their records and the models made from them, and let your legal team's answer in config decide what happens to those.
其他故障
- 保留未知答复的拒绝过滤器。 用它构建时,语料中只有 2,045 条评论,而不是 436 条,其中大多来自从未被询问过的学习者。发现的办法是通过来源信息,按同意答复统计语料中的记录数:除“是”以外的任何答复都属于缺陷。
- 模型从其文本中记住的个人数据。 Carlini 等人从一个语言模型中提取出了训练文本,其中包括姓名、电话号码和电子邮件地址等公开的个人信息,尽管每个这样的序列只在一份文档中出现过一次(Carlini et al., 2021)。普通流水线的语料中有 120 个已标注的电子邮件,其中 113 个互不相同,106 个只写过一次;而重复会让检测器漏掉的内容在每一份副本中一再出现。请对照你自己文本的已标注样本,衡量检测之后残留的内容,并在训练前去重。
- 泄露的假名密钥。 有了语料的密钥和一份讲师姓名清单,任何人都能算出语料中的每一个假名。密钥可能已经泄露时,应轮换密钥,重建所有用它生成的语料,并像处理撤回请求那样,按运行 ID 找出由旧密钥下的构建所生成的副本和模型。
这部分尚未翻译,因此以英文显示。
What this chapter decided
- The corpus is built through the enforcement layer, as
analyst-1forai-training: only reviews whose author'sai_training_consentis true, contacts masked, an unknown consent counted as no. - Every legal answer the code needs is a placeholder for the legal team in
[legal]ingovernance/ai_data.toml: the consent that admits a record, the card's licence, and what happens to models made before a withdrawal. - Personal data is detected after the enforcement layer's masks, with the pattern recognizers and the platform team's list of instructors' names, and replaced by keyed pseudonyms. Privacy Filter is measured, not added. The corpus is pseudonymized, never anonymized.
- Pseudonyms are HMAC-SHA256 under the corpus job's own key, not the enforcement layer's SHA-256 over key and value, and never Presidio's randomly salted
hash. - Duplicates are removed in two steps, exact copies by SHA-256, then near-duplicates from MinHash LSH tuned for 0.6 and confirmed at a Jaccard similarity of 0.8, never merging texts that differ only in a name. datasketch and its scheme,
affine32, are pinned. - Every record keeps its provenance,
source_table,review_id,jobandrun_id, and a training run checks its build against today's consent first. - The dataset card is Croissant 1.1 with RAI 1.0, generated by the run, checked by the platform's own code, and unpublished until
ai.review_corpushas an owner. - Answers for one person come from an index of the corpus, filtered by that person's decision on each record's source review.
- The EU AI Act is quoted as amended and linked, never interpreted. A corpus is only as clean as what was admitted into it. Each record is decided when the corpus is built, each record keeps its source, so a decision can be traced back to the review it came from, and the card says what was done with the rest.
本章,在本版次中
版次 2026.11.31示例于 2026年11月19日 通过本章 10个示例 个示例中有 10 个通过
译自英文版图书,版次 2026.11.31。