AI를 위한 데이터 거버넌스
읽는 데 약 78분분 걸린다, 12절에 10개 예제이(가) 있음.
AI로 번역함. 영어 원문을 사람이 검증함. 우리가 책을 만드는 방식
영어판 2026.11.31판을 번역한 것이다. 원문 읽기
이 페이지의 내용
- Admit only reviews whose authors consented
- Replace personal data in the text, then measure what's left
- 키 기반 해시로 가명을 일관되게 유지하기
- Find near-duplicates with MinHash
- Tune LSH for recall, then confirm every pair
- 모든 레코드에 출처 정보를 담는다
- Generate the dataset card from the run
- EU AI Act가 학습 데이터에 대해 말하는 내용을 인용한다
- 운영 환경에서 무엇이 잘못되는가
- A learner withdraws after the build
- Other failures
- What this chapter decided
이 장을 마치면 AI 어시스턴트가 학습하는 데이터를 플랫폼의 다른 모든 읽기 작업과 같은 규칙 아래에서 구축할 수 있다. 저자가 그러한 용도에 동의한 레코드만 받아들이고, 개인 데이터를 가명으로 바꾸고 탐지기가 놓친 것을 측정하며, MinHash로 찾은 중복을 제거하고, 모든 레코드에 그 출처를 함께 보관하며, 파이프라인이 실행될 때마다 데이터셋 카드를 작성하고 검사하게 한다. 또한 학습 데이터에 관해 EU AI Act가 무엇을 말하는지 인용할 수 있게 된다.
이 책이 따라가는 온라인 학습 플랫폼은 학습자가 리뷰에 쓴 글을 바탕으로 답하는 강좌 어시스턴트를 원한다. 플랫폼은 먼저 그 글에 근거해 어시스턴트가 답하게 하고, 나중에 같은 글로 어시스턴트를 파인튜닝할 계획이다. 4장에서 계보를 수집한 네 SQL 파이프라인 중 하나인 review_corpus은 learning.reviews의 모든 행을 각 리뷰의 강좌와 평점과 함께 ai.review_corpus로 복사한다. 2장에서 ai.review_corpus에 소유자도 분류도 없다는 것을 확인했는데, 이 복사본에는 문제가 네 가지 더 있다. 첫째, 저자가 자기 글이 AI를 학습시키는 데 쓰이는 것에 동의했는지와 관계없이 모든 리뷰를 담고 있다. 둘째, 텍스트에 이름과 이메일이 들어 있으며, 이는 3장에서 리뷰 안에서 찾아낸 것이다. 셋째, 리뷰 대부분이 서로를 반복한다. 넷째, 레코드가 어디서 왔는지 알려 주는 정보가 없어서, 어떤 리뷰가 허용되어 들어왔음을 입증할 수도, 저자가 마음을 바꿨을 때 그 리뷰를 다시 찾을 수도 없다.
이 장은 이 작업을 책의 AI 데이터 공용 모듈 lib.ai_data로 다시 구성하며, 어시스턴트는 이렇게 거버넌스를 거친 빌드로만 학습한다. pipelines/review_corpus.sql는 4장의 거버넌스 없는 복사본으로 그대로 두므로 CI는 그 계보를 계속 검사한다. 이 모듈은 파이프라인 자체의 SQL을 6장에서 만든 집행 계층을 통해 실행하고, 텍스트에 남은 개인 데이터를 바꾸고, 중복을 제거하고, 코퍼스와 그 계보와 데이터셋 카드를 작성한다. 모듈의 설정은 governance/ai_data.toml에 있으며 정책처럼 검토된다. 코드에 필요한 법적 질문의 답도 마찬가지로 거기에 있는데, 각 답은 플랫폼의 법무팀이 자신의 답과 이유로 바꿔 넣을 자리 표시자이다.
이 부분은 아직 번역되지 않았으므로 영어로 표시된다.
Admit only reviews whose authors consented
On the platform, consent to one use doesn't cover another, because the question a learner answers names the use. Article 5(1)(b) of the EU's General Data Protection Regulation (GDPR) says that personal data shall be "collected for specified, explicit and legitimate purposes and not further processed in a manner that is incompatible with those purposes" (GDPR). The code reads the recorded answer to whether training an AI is covered from [legal.consent] in governance/ai_data.toml, and decides consent per record when the corpus is built, not from the account's sign-up and not once for the whole table.
crm.learners.ai_training_consent records whether a learner agreed to their text training the platform's AI. Chapter 5 made it an attribute the access policy reads, so a read of learning.reviews for the purpose ai-training keeps only the reviews whose author's answer is true, with contact details removed. lib.ai_data.read runs the SELECT of pipelines/review_corpus.sql through the enforcement layer as analyst-1, the data-science principal, stating the purpose. lib.ai_data.build refuses to run unless the layer filters on the column the consent answer names. Compare three builds: the plain pipeline, pipelines/review_corpus.sql run as written; the governed read; and a consent filter written by hand.
1"""Counts the reviews that three builds of the corpus would hold, and the personal data the plain copy would keep. The prose's point rests on the "share by learners never asked" line."""2from collections import Counter34from lib import ai_data, pii5from lib.catalog import open_catalog6from lib.enforce import Enforcer78platform = Enforcer(open_catalog(["crm.learners", "learning.courses", "learning.reviews"]))9layer = platform.connection # the enforcement layer's own access, for counting what each build would hold1011# The plain pipeline copies every review, with its text as written.12plain = layer.sql("select count(*) from learning.reviews").fetchall()[0][0]13print("reviews the plain pipeline copies:", f"{plain:,}")1415# The governed build: the same SQL, through the enforcement layer, as the job's principal, for its purpose.16# the job's own read: its principal and purpose come from the build settings17governed = ai_data.read(platform)18print(f"reviews the governed build reads, as {ai_data.BUILD['principal']} for {ai_data.BUILD['purpose']}:", f"{len(governed):,}")19# no purpose stated: the layer returns every review, with its text left out20without = ai_data.read(platform, purpose=None)21print("reviews read without the purpose:", f"{len(without):,}")22print("of them, with their text:", sum(r["body"] is not None for r in without))2324# The filter that looks right: leave out the learners who said no.25# every review except those whose learner said no; a null answer passes, since only "= false" is excluded26opt_out = layer.sql("""27 select count(*), count(*) filter (where l.ai_training_consent is null)28 from learning.reviews r join crm.learners l using (learner_id)29 where r.learner_id not in (select learner_id from crm.learners where ai_training_consent = false)""").fetchall()[0]30print("reviews a filter on refusals keeps:", f"{opt_out[0]:,}")31print("of them, by learners never asked:", f"{opt_out[1]:,}")32print("share by learners never asked:", f"{opt_out[1] / opt_out[0]:.0%}")33# the signup dates of the learners whose consent is still unknown34never = layer.sql("""select min(signup_at)::date, max(signup_at)::date from crm.learners35 where ai_training_consent is null""").fetchall()[0]36print("learners never asked signed up:", " to ".join(f"{day:%b} {day.day}, {day.year}" for day in never))3738# Personal data the plain pipeline's corpus would hold, from the labelled truth, and how often each email is written.39raw = dict(layer.sql("select review_id, body from learning.reviews").fetchall())40for entity, (placed, _) in ai_data.left(raw).items():41 print(f"plain pipeline, labelled {entity} values:", f"{placed:,}")42# each labelled email's text, counted by how many spans write it43emails = Counter(pii.text_of(s) for s in pii.truth() if s.source == "reviews" and s.entity == "EMAIL_ADDRESS")44print("plain pipeline, distinct labelled EMAIL_ADDRESS values:", len(emails))45print("of them, written once:", sum(n == 1 for n in emails.values()))
python corpus-consent-filter.pyreviews the plain pipeline copies: 3,000
reviews the governed build reads, as analyst-1 for ai-training: 436
reviews read without the purpose: 3,000
of them, with their text: 0
reviews a filter on refusals keeps: 2,045
of them, by learners never asked: 1,609
share by learners never asked: 79%
learners never asked signed up: Jan 1, 2024 to May 31, 2025
plain pipeline, labelled EMAIL_ADDRESS values: 120
plain pipeline, labelled PERSON values: 798
plain pipeline, distinct labelled EMAIL_ADDRESS values: 113
of them, written once: 106
Copied as written, the plain pipeline would put 120 labelled emails and 798 labelled names into the corpus, from 3,000 reviews.
The governed read keeps 436 reviews, the ones whose authors said yes. Without a purpose, the same principal gets 3,000 reviews, 0 of them with their text, so a job that forgets to state its purpose gets nothing it could train on.
Notice that the filter written by hand looks like respect for consent, which is why it's the one to recognize: it leaves out the learners who said no. It keeps 2,045 reviews, and 1,609 of them, 79%, are by learners who were never asked. The README of the platform's generated data says consent is unknown for sign-ups before Jun 1, 2025, and every learner with no answer signed up from Jan 1, 2024 to May 31, 2025. An unknown answer isn't a yes. A corpus built with that filter would be mostly text that no consent covers, and nothing about it would look wrong: no error, a plausible row count, and a condition on the consent column that a reviewer would read as correct. The enforcement layer's filter tests IS TRUE, so an unknown answer counts as no, which chapter 6 built for exactly this case.
On the platform, every review was written under one set of terms, so the card states one licence, read from [legal.licence]. A corpus that mixes sources, such as reviews and forum posts, needs a licence per record too, which is one reason every record keeps its source table.
Working rule: Decide consent per record, when the corpus is built, from the answer your legal team says covers the use, and admit only records whose answer is yes: never filter out the refusals and keep the rest. Build the corpus through the enforcement layer with its purpose stated, and have the build check that the layer filters on that answer. Keep the legal team's answers, with their reasons, in a reviewed file the code reads, and carry licences per record when a source mixes them.
이 부분은 아직 번역되지 않았으므로 영어로 표시된다.
Replace personal data in the text, then measure what's left
Consent decides whose text may enter the corpus. It doesn't make the text free of personal data: a review names its instructor, or gives its author's email for questions. Before such text trains or grounds a model, the corpus job detects the personal data in it, replaces each value, and measures what's left, because a detector's output lists what it found and never what it missed. The truth, chapter 3's labelled spans, marks every piece of personal data the generator placed in the reviews. lib.ai_data.left counts how many labelled values are still in the text after a step, matching by value, since every replacement moves the offsets after it.
The reviews arrive from the enforcement layer with its redact_contacts mask applied, which replaces emails and phone numbers with [email] and [phone] (chapter 6). Chapter 3's detector, lib.pii.platform_detector, runs Presidio's pattern recognizers with no language model, and no pattern finds a name. So lib.ai_data.detector adds a Presidio deny list of the names the platform knows: every word of every instructor's name, which Presidio escapes and matches as a whole word, ignoring case unless told otherwise (Presidio's pattern recognizer). The list comes from crm.instructors, which analyst-1 may not read. So the platform team's own job, named in [names] in governance/ai_data.toml, builds it with the platform's own access, as the enforcement layer reads crm.learners for consent, and hands the corpus job the list, never the table. The card records which job built it and how many words it holds. lib.ai_data.known_names stands for that job. Each value found becomes a keyed pseudonym, which the next section builds. Measure each step on the reviews the corpus reads, then try a longer list, with every learner's name in it too:
1"""Counts the labelled names and emails still in the reviews after each step of the corpus job, and the false positives of the instructors' list and of a longer one. The prose's point rests on the "then the instructors' names, PERSON left" line and the false-positive lines."""2from lib import ai_data, pii3from lib.catalog import open_catalog4from lib.enforce import Enforcer56# The reviews the corpus job reads: consenting authors only, with emails and phone numbers already masked.7rows = ai_data.read(Enforcer(open_catalog(["crm.learners", "learning.courses", "learning.reviews"])))8texts = {row["review_id"]: row["body"] for row in rows} # keyed by review ID, to match the labelled values row by row910# What's left of the labelled personal data after each step, matched by value against the generator's truth.11detectors = {"then the pattern recognizers": pii.platform_detector(), "then the instructors' names": ai_data.detector()}12# the texts as the enforcement layer leaves them, before any detector runs13stages = {"masked by the enforcement layer": texts}14# each detector runs alone on the masked texts; pseudonymize returns the text first15stages |= {name: {i: ai_data.pseudonymize(engine, text)[0] for i, text in texts.items()} for name, engine in detectors.items()}16for stage, done in stages.items():17 for entity, (placed, still) in ai_data.left(done).items():18 print(f"{stage}, {entity} left: {still} of {placed}")1920# the instructors' words, from the governance team's list21names = ai_data.known_names()22print("words in the list of instructors' names:", len(names))23# short words are the ones ordinary text could contain24print("words in the list of 3 characters or fewer:", ", ".join(word for word in names if len(word) <= 3))25false_positives = sum(ai_data.unlabelled(ai_data.detector(), texts).values())26assert false_positives == 027print("false positives, instructors' names:", false_positives)2829# Every name the platform holds, learners' too, on the same reviews: in any case, then only as written.30# the union of the instructors' words and every learner's, which the table holds31longer = sorted(set(names) | set(ai_data.known_names("crm.learners")))32# matched in any case, as the platform's detector matches by default33everyone = ai_data.detector(longer)34wrong = ai_data.unlabelled(everyone, texts)35print("words in the list, with learners' names:", f"{len(longer):,}")36print("false positives, with learners' names:", sum(wrong.values()))37print("most common false positives:", ", ".join(f"{word} ({n})" for word, n in wrong.most_common(2)))38# review 24, which the longer list changes39print("a review after:", ai_data.pseudonymize(everyone, texts[24])[0])40# a word matches only as written, so a lowercase word no longer matches a capitalized name41cased = ai_data.detector(longer, match_case=True)42cased_wrong = sum(ai_data.unlabelled(cased, texts).values())43assert cased_wrong == 044print("false positives, with learners' names, matching case:", cased_wrong)45# pseudonymize with the cased detector, then count what is left by value46left = ai_data.left({i: ai_data.pseudonymize(cased, text)[0] for i, text in texts.items()})47assert left["PERSON"][1] == 048print("names left, with learners' names, matching case:", f"{left['PERSON'][1]} of {left['PERSON'][0]}")
python corpus-personal-data-left.pymasked by the enforcement layer, EMAIL_ADDRESS left: 0 of 19
masked by the enforcement layer, PERSON left: 120 of 120
then the pattern recognizers, EMAIL_ADDRESS left: 0 of 19
then the pattern recognizers, PERSON left: 120 of 120
then the instructors' names, EMAIL_ADDRESS left: 0 of 19
then the instructors' names, PERSON left: 0 of 120
words in the list of instructors' names: 106
words in the list of 3 characters or fewer: Ray, Roy, Ulf, Uss, de, la
false positives, instructors' names: 0
words in the list, with learners' names: 4,316
false positives, with learners' names: 147
most common false positives: best (80), date (67)
a review after: Some videos are out of <PERSON_8e2b63a6>.
false positives, with learners' names, matching case: 0
names left, with learners' names, matching case: 0 of 120
The enforcement layer's mask leaves 0 of 19 emails and 120 of 120 names, and the pattern recognizers leave 120 of 120 names, since no pattern finds a name.
Only the instructors' list removes names. With its names added, the detector leaves 0 of 120 names and makes no false positives, although its list holds short words that ordinary text could contain: Ray, Roy, Ulf, Uss, de, la.
The longer list, with every learner's name in it too, makes 147 false positives. The most common, best (80), date (67), are surnames that are also English words, and the list turns Some videos are out of date. into Some videos are out of <PERSON_8e2b63a6>. in the corpus.
Matching case avoids both. Matched only as written, the longer list makes no false positives and leaves no names, because every name in these reviews is capitalized. On real text it would miss the names typed in lowercase that chapter 3 found in tickets. The platform keeps the instructors' list, matched in any case: instructors are the people reviews talk about, and the corpus job then holds their names instead of every learner's.
The list's result holds for this data only. The labelled spans are generated, and the generated reviews name only instructors, so they can't contain the case a list of instructors misses, such as a classmate's name. For names nobody listed, the detector to try is a named-entity model. Chapter 3 compared one, OpenAI's Privacy Filter, from a recording of its detections, scored against the labelled spans with lib.pii.score. Score it the same way, on the reviews the corpus reads:
1"""Scores OpenAI's Privacy Filter, from its recorded detections, against the labelled spans in the reviews the corpus reads. The prose's point rests on the "names the model finds, exactly" and "names the model misses" lines."""2from collections import Counter34from lib import ai_data, pii5from lib.catalog import open_catalog6from lib.enforce import Enforcer78# Privacy Filter's eight labels, mapped to the labelled spans' entity types. A label with no type matches nothing.9MATCHES = {"private_person": {"PERSON"}, "private_email": {"EMAIL_ADDRESS"}, "private_phone": {"PHONE_NUMBER"},10 "private_address": {"LOCATION"}, "account_number": {"CREDIT_CARD", "IBAN_CODE"},11 "private_date": {"DATE_TIME"}, "private_url": set(), "secret": set()}1213# The reviews the corpus job reads, with the model's recorded detections and the labelled spans in them. The model read14# each review as written, before the enforcement layer's mask, which changes no name.15corpus = {row["review_id"] for row in ai_data.read(Enforcer(open_catalog(["crm.learners", "learning.courses",16 "learning.reviews"])))}17# the model's detections as recorded, not a live call to the model18model, manifest = pii.recorded("pii-entities")19found = [span for span in model if span.source == "reviews" and span.row_id in corpus]20labelled = [span for span in pii.truth() if span.source == "reviews" and span.row_id in corpus]21print("model:", manifest["model"])22print("reviews measured:", len(corpus))2324# per entity type: [labelled, found exactly, found overlapping], then per detector label: [found, exact, overlapping]25scores = pii.score(found, labelled, MATCHES)26# the overlap count is unused here: the prose counts exact matches only27placed, exact, _ = scores.recall["PERSON"]28print("labelled names:", placed)29print("names the model finds, exactly:", exact)30# private_person is the model's label for a name, so only that label counts as a match31missed = pii.missed([span for span in labelled if span.entity == "PERSON"], found, {"private_person"})32print("names the model misses:", len(missed))33often = Counter(map(pii.text_of, missed)).most_common(3)34print("names it misses most often:", ", ".join(f"{name} ({n})" for name, n in often))3536# What the model finds where no labelled span of the type its label maps to is: text it would replace for nothing.37wrong = sum(n - overlap for n, _, overlap in scores.precision.values())38print("the model's detections:", len(found))39print("of them, overlapping no labelled span of their type:", wrong)
python privacy-filter-on-the-corpus.pymodel: openai/privacy-filter
reviews measured: 436
labelled names: 120
names the model finds, exactly: 83
names the model misses: 37
names it misses most often: Marie (4), Liwia (4), Luigi (4)
the model's detections: 102
of them, overlapping no labelled span of their type: 0
On these reviews the model can't replace the list, and this data can't show what it adds, since every name in it is on the list. It finds 83 of the 120 labelled names exactly and misses 37, most often Marie (4), Liwia (4), Luigi (4), where the list misses none.
Of its 102 detections, 0 overlap no labelled span of their type. Where text names people no list holds, run both, measured this way first.
The text that comes out can still identify people. A detector misses spans, the text can identify someone without a name, and each record keeps a review ID that leads back to its author through learning.reviews. The GDPR's Recital 26 says: "Personal data which have undergone pseudonymisation, which could be attributed to a natural person by the use of additional information should be considered to be information on an identifiable natural person" (GDPR, Recital 26). So we call the corpus pseudonymized, never anonymized.
Working rule: Detect personal data in corpus text after the enforcement layer's masks, add the identifiers your platform holds about the people the text is about, and measure what's left against labelled data, by value, after every change to the detector. Prefer a short list of the names your text is about to every name you hold, since a long list matches ordinary words. Have a job that may read the names build the list, and record which list each corpus used. Measure a named-entity model the same way before you rely on it for names no list holds. Keep the list out of the corpus, and call the result pseudonymized.
키 기반 해시로 가명을 일관되게 유지하기
탐지된 이름은 자리표시자 <PERSON>로 바꿀 수도 있고, 가명으로 바꿀 수도 있다. 가명은 이름 하나를 나타내는 토큰이며, 그 이름이 나타나는 모든 곳에서 같다. 이 플랫폼의 후기는 강사를 이름만으로 부르고 이름이 같은 강사는 없으므로, 여기서는 토큰 하나가 한 사람을 가리킨다. 가명은 누가 누구인지를 보존하므로, 같은 강사를 칭찬하는 두 후기는 가명을 통해서도 그 사실을 드러낸다.
Presidio의 익명화 도구에는 hash 연산자가 있으며, 버전 2.2.361부터는 보안을 위해 기본적으로 해시마다 무작위 솔트를 적용한다(Presidio anonymizer). 무작위 솔트는 각 해시를 다음 해시와 연결할 수 없게 만들므로 가명으로는 쓸모가 없다. 같은 값에서 같은 해시를 얻으려면, 문서가 말하듯 "반드시 salt 매개변수를 명시적으로 제공해야" 하며, 그 길이는 128비트(16바이트) 이상이어야 한다.
코퍼스 작업은 대신 자체 키 기반 해시를 사용하며, Presidio의 custom 연산자를 통해 적용한다. 이 연산자는 문자열을 반환해야 하는 람다를 받는다. lib.ai_data.pseudonym는 엔티티 유형과 대소문자를 통일한 값에 대해, 코퍼스 작업만 가진 키로 SHA-256 기반 HMAC을 계산하고 16진수 여덟 자리를 남긴다. HMAC은 RFC 2104가 해시에 키를 적용하는 방식으로 기술한 구성이며, 이 문서가 밝힌 목표 중 하나는 "기반 해시 함수에 대한 합리적인 가정 위에서 인증 메커니즘의 강도에 대한 잘 이해된 암호학적 분석을 갖추는 것"이다(RFC 2104). Python 표준 라이브러리에는 hmac이 있으므로, 이 작업이 해시에 키를 적용하는 다른 방법을 쓸 이유는 없다.
코퍼스의 두 후기가 같은 강사를 언급한다고 하자. 그 이름을 두 방식으로 모두 바꿔 보면 어느 쪽이 두 후기의 연결을 유지하는지 알 수 있다.
1"""Replaces the same instructor's name in two reviews by Presidio's hash, by the corpus's keyed hash and by the enforcement layer's mask. The prose's point rests on the Presidio hash lines and the keyed pseudonym line."""2from collections import Counter34from presidio_anonymizer import AnonymizerEngine5from presidio_anonymizer.entities import InvalidParamError, OperatorConfig6from sqlglot import exp78from lib import ai_data, enforce9from lib.catalog import open_catalog10from lib.enforce import Enforcer11from lib.platform import load1213# Two of the corpus's reviews that name the same instructor, with what the detector found in each.14layer = Enforcer(open_catalog(["crm.learners", "learning.courses", "learning.reviews"]))15engine = ai_data.detector()16found = [(row["body"], engine.analyze(row["body"], language="en")) for row in ai_data.read(layer)]17# the first two reviews whose only detected name is Patrick18named = [(text, spans) for text, spans in found if [text[s.start:s.end] for s in spans] == ["Patrick"]][:2]19print("reviews naming Patrick:", len(named))2021def replaced(operator: OperatorConfig) -> list[str]:22 """The value that replaced the name in each of the two reviews."""23 results = [AnonymizerEngine().anonymize(text, spans, {"PERSON": operator}) for text, spans in named]24 return [result.text[item.start:item.end] for result in results for item in result.items]2526# Presidio's hash salts each value at random, so the same name gets a different hash in each review27hashed = replaced(OperatorConfig("hash"))28assert hashed[0] != hashed[1]29print("Presidio's hash, the same name in two reviews gets one value:", hashed[0] == hashed[1])30# and hashing the same review again gives it a different hash too31rehashed = replaced(OperatorConfig("hash"))[0]32assert rehashed != hashed[0]33print("Presidio's hash, one review hashed twice gets one value:", rehashed == hashed[0])34# a fixed salt must be at least 16 bytes; this one is 10, so Presidio refuses it35try:36 replaced(OperatorConfig("hash", {"salt": "ten bytes!"}))37except InvalidParamError as error:38 print("a fixed salt of 10 bytes:", error)3940# the custom operator's lambda returns the keyed pseudonym as the replacement41keyed = replaced(OperatorConfig("custom", {"lambda": lambda value: ai_data.pseudonym("PERSON", value)}))42print("keyed pseudonyms:", ", ".join(keyed))43print("the name typed in lowercase:", ai_data.pseudonym("PERSON", "patrick"))44# the first word of each instructor's full name45first_names = Counter(name.split()[0] for name in load("crm.instructors").column("full_name").to_pylist())46shared = sum(n for n in first_names.values() if n > 1)47assert shared == 048print("instructors who share a first name with another:", shared)4950# What the enforcement layer's pseudonymize mask makes of the same name, run by the layer under its own key.51mask = enforce.masked("pseudonymize", exp.Literal.string("Patrick"), "VARCHAR").sql(enforce.DIALECT)52print("the enforcement layer's pseudonymize mask, on the same name:", layer.run(f"select {mask}").fetchall()[0][0])53# functions whose name mentions HMAC: the prose says DuckDB has none54hmac = layer.connection.sql("select count(*) from duckdb_functions() where function_name ilike '%hmac%'").fetchall()[0][0]55assert hmac == 056print("HMAC functions in DuckDB:", hmac)
python keyed-pseudonyms.pyreviews naming Patrick: 2
Presidio's hash, the same name in two reviews gets one value: False
Presidio's hash, one review hashed twice gets one value: False
a fixed salt of 10 bytes: Salt must be at least 16 bytes (128 bits). Provided salt is 10 bytes.
keyed pseudonyms: <PERSON_d14171be>, <PERSON_d14171be>
the name typed in lowercase: <PERSON_d14171be>
instructors who share a first name with another: 0
the enforcement layer's pseudonymize mask, on the same name: 23b3d8671d01840655296f752121aecc4f7991694260e5e1431d2b57c5e3d53f
HMAC functions in DuckDB: 0
Presidio의 hash는 두 후기에서 같은 이름에 서로 다른 값을 주고, 한 후기도 해시할 때마다 다른 값을 준다. 또한 문서가 말하듯 Presidio는 너무 짧은 고정 솔트, 여기서는 10바이트를 Salt must be at least 16 bytes (128 bits). Provided salt is 10 bytes.로 거부한다. 키 기반 해시는 두 후기 모두에 <PERSON_d14171be>, <PERSON_d14171be>를 주며, 소문자로 입력한 이름에도 <PERSON_d14171be>를 준다.
시행 계층이 같은 이름에 대해 실행하는 pseudonymize 마스크는 23b3d8671d01840655296f752121aecc4f7991694260e5e1431d2b57c5e3d53f를 준다. 이는 6장에 적힌 대로 계층의 키 뒤에 값을 이어 붙인 SHA-256이며 HMAC이 아니다. 계층의 마스크는 SQL이고 DuckDB 2.0.0에는 HMAC 함수가 없기 때문이다. 두 방식은 서로 다른 키를 쓰는 서로 다른 구성이므로, 코퍼스의 가명을 계층이 분석가에게 보여 주는 가명과 조인할 수 없다.
일관성에는 두 가지 비용이 따른다. 키와 이름 목록을 가진 사람은 누구나 모든 가명을 계산할 수 있다. 6장이 이메일에 대해 보였듯이 그렇다. 따라서 키는 비밀 관리자에 두고, 코퍼스나 저장소나 로그에는 절대 두지 않는다. 또한 키 기반 가명은 자리표시자라면 하나로 합쳐질 후기들을 따로 유지한다. 서로 다른 텍스트는 96개인 반면 자리표시자에서는 21개이며, 이는 다음 절에서 측정한다. 우리는 상담원과 학습자를 언급하는 티켓처럼 텍스트의 의미가 사람을 구별하는 데 달려 있을 때는 키 기반 가명을, 후속 과정에서 그럴 필요가 없을 때는 자리표시자를 권한다.
작업 규칙: 가명은 작업만 가진 키로 HMAC을 사용해 만들고, 가명이 서로 조인되어서는 안 되는 코퍼스나 대상마다 별도의 키를 부여한다. 가명에는 솔트 없는 Presidio hash 연산자를 쓰지 않는다. 고정된 버전에서는 해시마다 값이 무작위이기 때문이다. 두 사람이 이름의 한 단어를 공유할 수 있다면 이름의 단어 대신 사람의 ID를 가명화하고, 후속 과정에서 사람을 구별할 필요가 없다면 단순한 자리표시자를 사용한다.
이 부분은 아직 번역되지 않았으므로 영어로 표시된다.
Find near-duplicates with MinHash
Once its names are pseudonymized, the corpus's next problem is repetition: templates, copies, a review pasted twice, the same sentence from many people. Repetition is a privacy problem as well as a quality one. Kandpal, Wallace and Raffel found that "a sequence that is present 10 times in the training data is on average generated ~1000 times more often than a sequence that is present only once" (Kandpal et al., 2022).
Near-copies need a measure of similarity. Broder defined the resemblance of two documents as the Jaccard similarity of their sets of shingles, the overlapping runs of characters or words in a text. He showed that sampling the smallest shingles under a random permutation gives an unbiased estimate of that resemblance (Broder, 1997). Taking the smallest shingle under each of many permutations gives a MinHash signature, which estimates the similarity of two texts without comparing their sets (Mining of Massive Datasets, section 3.3).
lib.ai_data.deduplicate removes exact copies first, by SHA-256 of the case-folded text with runs of white space made one. It compares what's left as sets of 5-character shingles with datasketch's MinHash and MinHashLSH, which returns candidate pairs, and confirms each candidate by its exact Jaccard similarity. A confirmed pair at 0.8 or more is a near-duplicate, unless it differs only in a name, and of each group of duplicates the record with the lowest review ID stays. The duplicate step runs on the pseudonymized reviews, and each stage reports what it removed:
1"""Runs the corpus's duplicate step on the pseudonymized reviews: exact copies, then near-duplicates by MinHash LSH, each confirmed exactly, and the records kept. The prose's point rests on the "of them, differing only in a name" and "near-duplicates removed" lines."""2from collections import Counter34from lib import ai_data5from lib.catalog import open_catalog6from lib.enforce import Enforcer78# The reviews the corpus job reads, pseudonymized, then deduplicated: exact copies, then near-duplicates.9# the reviews read, each text pseudonymized; the second value counts the replacements10rows, _ = ai_data.read_pseudonymized(Enforcer(open_catalog(["crm.learners", "learning.courses", "learning.reviews"])))11# groups of reviews that stand for one record, and how many each step removed12groups, removed = ai_data.duplicate_groups(rows)13print("reviews read:", len(rows))14# one review per distinct normalized text: the one with the lowest review ID15print("distinct texts:", len(ai_data.exact_distinct(rows)))16for step, count in removed.items():17 print(f"{step}:", count)18print("records kept:", len(groups))19print("share of the reviews read that duplicate another:", f"{1 - len(groups) / len(rows):.0%}")2021# identical texts only, before normalization22text, copies = Counter(row["body"] for row in rows).most_common(1)[0]23print("the most repeated text:", text)24print("its copies:", copies)2526# A pair MinHash finds and the exact comparison confirms, and whether it differs only in a name.27distinct = [row["body"] for row in ai_data.exact_distinct(rows)]28# the first confirmed pair, by position among the distinct texts29i, j = sorted(ai_data.near_duplicates(distinct).confirmed)[0]30print("a near-duplicate pair:", distinct[i], "|", distinct[j])31print("its Jaccard similarity:", f"{ai_data.jaccard(distinct[i], distinct[j]):.2f}")32# equal once each pseudonym is a placeholder: the pair differs only in which names it holds33print("it differs only in a name:", ai_data.placeholders(distinct[i]) == ai_data.placeholders(distinct[j]))3435# Each record kept stands for its group's reviews, under the review ID and course of the first.36# a group whose reviews come from more than one course37several = sum(len({row["course_id"] for row in group}) > 1 for group in groups)38print("records that stand for reviews of several courses:", several)
python corpus-duplicates.pyreviews read: 436
distinct texts: 96
exact duplicates removed: 340
near-duplicate pairs from LSH: 381
near-duplicate pairs confirmed: 8
of them, differing only in a name: 8
near-duplicates removed: 0
records kept: 96
share of the reviews read that duplicate another: 78%
the most repeated text: Clear explanations and good exercises.
its copies: 56
a near-duplicate pair: Some videos are out of date. <PERSON_85783bac> explains things well. Happy to talk about it, write to me at [email]. | Some videos are out of date. <PERSON_967f9ae8> explains things well. Happy to talk about it, write to me at [email].
its Jaccard similarity: 0.81
it differs only in a name: True
records that stand for reviews of several courses: 23
Most of the corpus repeats itself. Of the 436 reviews read, 78% repeat another: 340 are exact copies, and only 96 texts are distinct. That's mostly the generator's doing, since it builds reviews from a handful of fixed sentences, so measure the share on your own text before training on it.
The most repeated text, "Clear explanations and good exercises.", alone appears 56 times.
MinHash then confirms 8 pairs at 0.8 or more, such as two reviews at a similarity of 0.81 that differ only in their instructor's pseudonym. Of those pairs, 8 differ only in a name, which the platform keeps apart, so near-duplicates remove 0.
Why pairs that differ only in a name stay apart comes down to what one threshold means at different lengths. Compare the corpus's reviews with the names as written, with keyed pseudonyms and with plain placeholders, then split every pair of distinct texts by whether it differs only in a name:
1"""Counts near-duplicate pairs of the corpus's reviews with names as written, with keyed pseudonyms and with placeholders, then splits the pairs that differ only in a name by length. The prose's point rests on the pair counts and the length ranges."""2from itertools import combinations34from lib import ai_data5from lib.catalog import open_catalog6from lib.enforce import Enforcer78# The reviews the corpus job reads, three ways: names as written, keyed pseudonyms, and plain placeholders.9layer = Enforcer(open_catalog(["crm.learners", "learning.courses", "learning.reviews"]))10keyed, _ = ai_data.read_pseudonymized(layer)11# the same reviews with each pseudonym replaced by a plain placeholder, as the third way12ways = {"names as written": ai_data.read(layer), "keyed pseudonyms": keyed,13 "placeholders": [row | {"body": ai_data.placeholders(row["body"])} for row in keyed]}14# the similarity at which a pair is a near-duplicate15threshold = ai_data.DUPLICATES["threshold"]16for way, rows in ways.items():17 texts = [row["body"] for row in ai_data.exact_distinct(rows)]18 print(f"{way}, distinct texts:", len(texts))19 print(f"{way}, pairs at {threshold} or more:", len(ai_data.near_duplicates(texts).confirmed))2021# Every pair of distinct pseudonymized texts, split by whether the pair differs only in a name.22# distinct texts only, so each text appears once in the pairs23texts = [row["body"] for row in ai_data.exact_distinct(keyed)]24name_only, other = [], []25for a, b in combinations(texts, 2):26 # similarity comes first, so min and max over these tuples order the pairs by it27 pair = (ai_data.jaccard(a, b), len(a), a, b)28 (name_only if ai_data.placeholders(a) == ai_data.placeholders(b) else other).append(pair)29print("pairs that differ only in a name:", len(name_only))30# the lowest and highest similarity among the name-only pairs31print("their Jaccard similarity:", f"{min(name_only)[0]:.2f} to {max(name_only)[0]:.2f}")32for label, lengths in [(f"below {threshold}", [n for s, n, _, _ in name_only if s < threshold]),33 (f"at {threshold} or more", [n for s, n, _, _ in name_only if s >= threshold])]:34 print(f"their length in characters, {label}:", f"{min(lengths)} to {max(lengths)}")35# the most similar pair among those that differ in more than a name36similarity, _, a, b = max(other)37print("the most similar pair that differs in more than a name:", f"{similarity:.2f}")38print("that pair:", a, "|", b)
python names-and-near-duplicates.pynames as written, distinct texts: 96
names as written, pairs at 0.8 or more: 10
keyed pseudonyms, distinct texts: 96
keyed pseudonyms, pairs at 0.8 or more: 8
placeholders, distinct texts: 21
placeholders, pairs at 0.8 or more: 0
pairs that differ only in a name: 459
their Jaccard similarity: 0.68 to 0.83
their length in characters, below 0.8: 66 to 78
their length in characters, at 0.8 or more: 116 to 119
the most similar pair that differs in more than a name: 0.66
that pair: Clear explanations and good exercises. Happy to talk about it, write to me at [email]. | Clear explanations and good exercises. <PERSON_3026fb67> explains things well. Happy to talk about it, write to me at [email].
Which pairs reach 0.8 depends on what stands in for each name. With the names as written, 10 pairs reach 0.8, and with keyed pseudonyms 8, since a pseudonym is longer than most first names and so a larger share of a short review. With plain placeholders, the 21 distinct texts hold 0.
Among the pseudonymized texts, 459 pairs differ only in which pseudonym they hold, at similarities from 0.68 to 0.83, and the threshold splits them by length alone: those below 0.8 are 66 to 78 characters long, and those at 0.8 or more are 116 to 119 characters long.
So the task, not the threshold, decides whether such reviews are duplicates: for a model that learns how learners describe courses, they are, and for one that learns what learners say about each instructor, they aren't. The platform's assistant answers about each course and its instructor, so name_only_pairs_are_duplicates = false in governance/ai_data.toml, and deduplicate never merges two texts that are the same once their pseudonyms are placeholders. The threshold decides the other pairs. The most similar of those, at 0.66, is a review with and without its sentence about the instructor: two different reviews. The platform keeps 0.8, above that pair with a margin.
Deduplicating on text folds reviews of different courses into one record, under the review ID and course of its first review, so the corpus keeps 96 records, and 23 of them stand for reviews of several courses. For training that's the point, since the model learns from the text. For grounding an answer about one course it costs recall, which the provenance section takes up.
Working rule: Deduplicate training text in two steps, exact copies by a hash of the normalized text and then near-duplicates by MinHash, each confirmed by its exact similarity, and report how much each step removed. Pseudonymize first, so that no name reaches the comparison or the sketches and the comparison can tell which pairs differ only in a name. Keep those pairs apart when the text's meaning depends on whom it names, and compare on placeholders when it doesn't. Choose the threshold by reading the pairs close above and below it in your own text.
이 부분은 아직 번역되지 않았으므로 영어로 표시된다.
Tune LSH for recall, then confirm every pair
MinHash signatures make each comparison cheap; locality-sensitive hashing (LSH) avoids most comparisons. LSH cuts each signature into b bands of r rows, and two texts become a candidate pair when all the rows of any one band match. Leskovec, Rajaraman and Ullman derive the chance that a pair with Jaccard similarity s becomes a candidate pair (Mining of Massive Datasets, chapter 3):
P(candidate) = 1 − (1 − sʳ)ᵇ
It's an S-shaped curve: a pair well below the threshold is rarely a candidate, a pair well above it nearly always, and near the threshold it's close to a coin toss. The curve's approximate threshold is (1/b)^(1/r). datasketch's documentation warns of both kinds of error: "There will be false positives - sets that do not satisfy your threshold but returned, and false negatives - qualifying sets that are not returned" (datasketch, MinHash LSH).
With 96 distinct texts, the platform can afford to compare every pair exactly, 4,560 pairs, and that gives the exact pairs to measure LSH against. LSH sees only similarity, so it must return every pair at 0.8, whatever the name rule then decides. Score LSH against the exact comparison, tuned for 0.8 and tuned lower, each under datasketch's default hashing and under the hashing it used before its 2.0.0 release:
1"""Measures the pairs LSH misses against every pair compared exactly, under two tunings and two hashing schemes, and shows what datasketch does with sketches of two schemes. The prose's point rests on the "missed" lines."""2from itertools import combinations34from datasketch import MinHashLSH56from lib import ai_data7from lib.catalog import open_catalog8from lib.enforce import Enforcer910rows, _ = ai_data.read_pseudonymized(Enforcer(open_catalog(["crm.learners", "learning.courses", "learning.reviews"])))11texts = [row["body"] for row in ai_data.exact_distinct(rows)]12# the similarity at which a pair is a near-duplicate13threshold = ai_data.DUPLICATES["threshold"]1415# Every pair compared exactly: possible at this size, and the truth LSH is measured against.16pairs = {(i, j) for i, j in combinations(range(len(texts)), 2) if ai_data.jaccard(texts[i], texts[j]) >= threshold}17print("pairs compared exactly:", f"{len(texts) * (len(texts) - 1) // 2:,}")18print(f"pairs at a Jaccard similarity of {threshold} or more:", len(pairs))1920found = {}21for scheme in ["affine32", "legacy"]:22 # LSH tuned for the threshold, and for the lower candidate threshold23 for tuned_for in [threshold, ai_data.DUPLICATES["candidate_threshold"]]:24 # the bands and rows LSH chose, its candidates, and the candidates the exact check confirms25 found[scheme, tuned_for] = result = ai_data.near_duplicates(texts, threshold, tuned_for, scheme)26 setting = f"{scheme}, LSH tuned for {tuned_for}"27 print(f"{setting}, bands:", result.bands)28 print(f"{setting}, rows per band:", result.rows)29 print(f"{setting}, candidate pairs:", len(result.candidates))30 print(f"{setting}, confirmed:", len(result.confirmed))31 # pairs the exact comparison found that LSH never proposed as candidates32 missed = len(pairs - result.candidates)33 print(f"{setting}, missed:", missed)34 if tuned_for != threshold:35 assert missed == 0 # tuned lower, LSH finds every pair under either scheme3637# The chance that a pair of similarity s becomes a candidate, with b bands of r rows: 1 - (1 - s^r)^b.38# the bands and rows LSH chose for the threshold, as the formula uses them39b, r = found["affine32", threshold].bands, found["affine32", threshold].rows40for s in [0.8, 0.9]:41 print(f"chance of a candidate at similarity {s}, LSH tuned for {threshold}:", f"{1 - (1 - s ** r) ** b:.2f}")42# the S-curve's approximate threshold, which the prose compares with the threshold asked for43print("the approximate threshold, (1/b)^(1/r):", f"{(1 / b) ** (1 / r):.2f}")4445# What datasketch does with a sketch made under the other scheme.46# an index takes sketches of one scheme, so the legacy sketch below is refused47index = MinHashLSH(threshold=threshold, num_perm=ai_data.DUPLICATES["num_perm"])48index.insert(0, ai_data.minhash(texts[0], "affine32"))49try:50 index.insert(1, ai_data.minhash(texts[1], "legacy"))51except ValueError as error:52 print("a legacy sketch in an index of affine32 sketches:", error)
python lsh-misses-near-duplicates.pypairs compared exactly: 4,560
pairs at a Jaccard similarity of 0.8 or more: 8
affine32, LSH tuned for 0.8, bands: 9
affine32, LSH tuned for 0.8, rows per band: 13
affine32, LSH tuned for 0.8, candidate pairs: 36
affine32, LSH tuned for 0.8, confirmed: 1
affine32, LSH tuned for 0.8, missed: 7
affine32, LSH tuned for 0.6, bands: 18
affine32, LSH tuned for 0.6, rows per band: 7
affine32, LSH tuned for 0.6, candidate pairs: 381
affine32, LSH tuned for 0.6, confirmed: 8
affine32, LSH tuned for 0.6, missed: 0
legacy, LSH tuned for 0.8, bands: 9
legacy, LSH tuned for 0.8, rows per band: 13
legacy, LSH tuned for 0.8, candidate pairs: 52
legacy, LSH tuned for 0.8, confirmed: 6
legacy, LSH tuned for 0.8, missed: 2
legacy, LSH tuned for 0.6, bands: 18
legacy, LSH tuned for 0.6, rows per band: 7
legacy, LSH tuned for 0.6, candidate pairs: 406
legacy, LSH tuned for 0.6, confirmed: 8
legacy, LSH tuned for 0.6, missed: 0
chance of a candidate at similarity 0.8, LSH tuned for 0.8: 0.40
chance of a candidate at similarity 0.9, LSH tuned for 0.8: 0.93
the approximate threshold, (1/b)^(1/r): 0.84
a legacy sketch in an index of affine32 sketches: MinHash scheme 'legacy' does not match scheme 'affine32' of the MinHash previously given to this index
Tuned for 0.8, datasketch's default LSH returns 36 candidate pairs, misses 7 of the 8 pairs that reach 0.8, and so confirms only 1 of them.
The formula says why. With the 9 bands of 13 rows datasketch chose, a pair at similarity 0.8 becomes a candidate with a probability of 0.40, one at 0.9 with 0.93, and the approximate threshold is 0.84, above the threshold asked for.
Under the default scheme, tuned for 0.6, LSH finds every pair and returns 381 candidates. Each candidate costs one exact comparison, against 4,560 for comparing every pair, and the exact comparison discards every pair below 0.8. So the platform tunes LSH for 0.6 and confirms at 0.8, both settings in governance/ai_data.toml.
datasketch's 2.0.0 release made affine32 the default permutation scheme of MinHash, and its release notes say that hash values differ from earlier versions, so they advise rebuilding persisted sketches and LSH indexes, or passing MinHash(..., scheme="legacy"). With the legacy scheme, LSH tuned for 0.8 returns 52 candidates and confirms 6, so a count recorded under one scheme isn't reproduced under the other.
Tuned for 0.6, both schemes find all 8 pairs. With high recall and an exact check, the result depends on the similarity, not on the hash functions.
datasketch refuses to mix schemes in one index, with MinHash scheme 'legacy' does not match scheme 'affine32' of the MinHash previously given to this index, but it won't stop you comparing old counts with new ones. The book's examples pin datasketch at 2.0.0 and write scheme = "affine32" into the settings, so an upgrade can't change a recorded count without a reviewed change.
Working rule: Tune LSH below your similarity threshold, so that it misses few pairs, and confirm every candidate by its exact similarity. Measure LSH's misses against an exhaustive comparison on a sample of your corpus whenever its settings, its library or the text change. Pin the MinHash library, write its hashing scheme into your settings, and never compare counts or stored sketches made under different schemes.
모든 레코드에 출처 정보를 담는다
출처를 추적할 수 없는 레코드는 검증할 수도, 바로잡을 수도, 제거할 수도 없다. 레코드 수준의 출처 정보는 세 가지 질문에 답한다. 어느 테이블에서 왔는지, 어느 행인지, 어떤 작업의 어느 실행이 그 레코드를 코퍼스에 넣었는지이다. 4장은 플랫폼에 마지막 부분을 제공한다. 작업이 실행될 때마다 OpenLineage 이벤트를 보내며, 실행 ID는 실행 시각과 작업 이름에서 만들어지고, 컬럼 리니지는 작업의 SQL을 파싱해서 얻는다.
lib.ai_data.build는 리뷰 자체의 컬럼 옆에 출처 정보를 붙여 모든 레코드를 기록한다. 그 컬럼은 source_table, review_id, job, run_id이다. 작업의 실행 이벤트는 lib.lineage.record로 보낸다. 여기에는 중요한 선택이 하나 있다. 리니지는 pipelines/review_corpus.sql에 있는 SQL이 아니라, 시행 계층이 실제로 실행한 쿼리, 즉 동의 필터와 마스크가 들어간 쿼리를 파싱해서 얻는다. 코퍼스를 구축한 다음, 레코드 하나를 그 실행까지 거슬러 따라가 보자.
1"""Builds the corpus and follows one record back to its source row, its job and the run that wrote it, then checks that run's lineage against the query as written. The prose's point rests on the "rows filtered by" and "body" lines."""2from lib import ai_data, lineage, pipelines3from lib.catalog import open_catalog45catalog = open_catalog(["crm.learners", "learning.courses", "learning.reviews"])6built = ai_data.build(catalog)7# the second record, whose text holds a pseudonym8record = built.records.to_pylist()[1]9print("a record's text:", record["body"])10for column in ["source_table", "review_id", "job"]:11 print(f"its {column}:", record[column])1213# The run that wrote it, from the lineage events: the query as the enforcement layer ran it.14# the job's COMPLETE event, which carries the run's lineage15complete = [e for e in lineage.read_events(built.folder / "events.jsonl") if e["eventType"] == "COMPLETE"][0]16same_run = record["run_id"] == complete["run"]["runId"]17assert same_run18print("its run_id is the COMPLETE event's run:", same_run)19print("the run's inputs:", ", ".join(dataset["name"] for dataset in complete["inputs"]))20# column lineage: the inputs that filter the rows, and how each field is transformed21facet = complete["outputs"][0]["facets"]["columnLineage"]22# a FILTER transformation marks an input whose values decide which rows the output holds23filters = [f"{i['name']}.{i['field']}" for i in facet["dataset"] if any(t["subtype"] == "FILTER" for t in i["transformations"])]24print("rows filtered by, as run:", ", ".join(filters))25# the lineage of the review text, from the reviews table26text = [i for i in facet["fields"]["body"]["inputFields"] if (i["name"], i["field"]) == ("learning.reviews", "body")]27# IDENTITY means copied unchanged; TRANSFORMATION means the value was changed on its way28print("body, as run:", text[0]["transformations"][0]["subtype"])29# the same lineage parsed from the SQL as written, with no enforcement rewrite30fields, rows = lineage.column_lineage(pipelines.sql(ai_data.BUILD["job"]), lineage.schema(catalog))31print("rows filtered by, as written:", ", ".join(i.source for i in rows if i.subtype == "FILTER") or "nothing")32print("body, as written:", fields["body"][0].subtype)3334# Back from every record to its review's author, with the platform's own access, against today's answers.35# each record's review ID, checked against the author's consent today, not at build time36ids = built.records.column("review_id").to_pylist()37print("records whose author consented:", f"{ai_data.consenting(catalog, ids)} of {len(ids)}")
python corpus-provenance.pya record's text: Too fast in the second half. <PERSON_85783bac> explains things well.
its source_table: learning.reviews
its review_id: 18
its job: platform/review_corpus
its run_id is the COMPLETE event's run: True
the run's inputs: crm.learners, learning.courses, learning.reviews
rows filtered by, as run: crm.learners.ai_training_consent, learning.reviews.learner_id
body, as run: TRANSFORMATION
rows filtered by, as written: nothing
body, as written: IDENTITY
records whose author consented: 96 of 96
레코드를 거슬러 따라가 보면 리니지가 무엇을 기록해야 하는지 드러난다. 레코드 "Too fast in the second half. <PERSON_85783bac> explains things well."는 learning.reviews에서 왔고, 리뷰는 18이며, 작업 platform/review_corpus이 기록했다. 그 run_id는 해당 작업의 COMPLETE 이벤트가 나타내는 실행의 ID이다.
그 실행은 crm.learners, learning.courses, learning.reviews을 읽었다. 리니지에 따르면 행은 crm.learners.ai_training_consent, learning.reviews.learner_id로 필터링되었다. 이는 어떤 행이 존재할지를 결정하는 INDIRECT 입력이다. 또한 body은 리뷰 텍스트의 TRANSFORMATION이다.
파이프라인의 SQL을 작성된 그대로 파싱하면, 같은 작업의 행은 nothing로 필터링되고 body은 IDENTITY 복사본이 된다. 그렇다면 동의 컬럼이 전혀 관여하지 않은 채 개인 텍스트가 그대로 복사되었다는 뜻이 된다. 실제로 일어난 일은 실행된 쿼리이므로, 리니지가 기록하는 것도 실행된 쿼리이다.
출처 정보가 있으면 코퍼스를 사후에 검증할 수 있다. 모든 레코드를 learning.reviews을 거쳐 작성자까지 조인하면 작성자가 동의한 레코드 96 of 96개를 찾을 수 있다. 플랫폼은 이 검사를 어떤 오래된 빌드에도 오늘의 답을 기준으로 실행할 수 있다.
출처 정보는 검색이 각 독자의 권한을 존중하게 해 주기도 한다. OWASP는 언어 모델 기반 애플리케이션에 "세분화된 접근 제어와 권한을 인식하는 벡터 및 임베딩 저장소를 구현하라"고 요구한다(OWASP LLM08:2025). 플랫폼은 이 코퍼스로 만든 인덱스에서 한 사람에게 답하는데, 이 인덱스는 각 레코드의 review_id를 벡터와 함께 보관한다. 검색된 레코드가 모델에 도달하기 전에 검색 계층은 답을 받을 그 사람의 자격으로 시행 계층에 그 리뷰 중 어느 것을 읽을 수 있는지 묻고, 나머지는 버린다. 6장이 어시스턴트의 쿼리를 실행하는 방식과 같다. 그러면 강사는 자신의 강좌에서 나온 레코드만 받는다. 여러 강좌의 리뷰를 대표하는 레코드는 첫 번째 리뷰의 ID를 달고 있으므로, 검사는 실패 시 차단하는 방향으로 동작한다. 그런 레코드 23개 각각에 대해, 그 사람이 읽을 수 있었던 텍스트를 숨길 수는 있어도 읽을 수 없는 텍스트를 보여 주는 일은 없다.
작업 규칙: 모든 레코드에 원본 테이블, 그 행의 ID, 그리고 그것을 기록한 작업과 실행을 부여하고, 작업의 리니지는 필터와 마스크를 포함해 실제로 실행된 쿼리에서 기록한다. 이 출처 정보를 통해 오래된 빌드를 오늘의 동의 기준으로 검사한다. 모델이 보기 전에, 각 레코드의 원본 행에 대한 독자 본인의 결정으로 모든 검색을 필터링한다.
이 부분은 아직 번역되지 않았으므로 영어로 표시된다.
Generate the dataset card from the run
A dataset card tells whoever trains or audits a model what's in its data, where it came from and what was done to it. Croissant is MLCommons' metadata format for machine-learning datasets (Akhtar et al., 2024). Its 1.1 specification makes a card checkable: "Croissant datasets must declare that they conform to the versioned schema, e.g. http://mlcommons.org/croissant/1.1", and every dataset must give a description, licence, name, URL, creator and publication date. For provenance, the specification recommends the W3C PROV Ontology (PROV-O) (Croissant 1.1). Its responsible-AI vocabulary adds properties such as rai:dataCollection and rai:personalSensitiveInformation (Croissant RAI 1.0). A card written by hand describes the data someone meant to build; a card generated by the run describes the data it built.
lib.ai_data.card generates the platform's card with mlcroissant 1.1.1, MLCommons' Python library for Croissant. The run supplies the counts, the corpus file's SHA-256, a field for every column, and three PROV-O properties: the tables the pipeline's own SQL reads, which the values come from, as prov:wasDerivedFrom; the table the enforcement layer's rewrite adds, which only decides which rows exist, as prov:wasInfluencedBy; and the run, as prov:wasGeneratedBy. mlcroissant writes them through its extra_properties argument but adds no prov prefix to the card's context, so the module adds it. The creator is the owner of ai.review_corpus in governance/metadata.toml. The licence, the name, URL, description and use cases come from governance/ai_data.toml, and lib.ai_data.card_problems is the platform's own check of the result. Generate the card, first with mlcroissant's default conformsTo:
1"""Generates the corpus's Croissant card twice, first with mlcroissant's default conformsTo and then as the platform writes it, and runs the platform's own check on it. The prose's point rests on the "conformsTo" lines and the "problems" line."""2from lib import ai_data3from lib.catalog import open_catalog45built = ai_data.build(open_catalog(["crm.learners", "learning.courses", "learning.reviews"]))67# mlcroissant's default, then the card the pipeline writes, with Croissant 1.1 written out.8# an empty list leaves conformsTo to mlcroissant, which writes its own default9default, _ = ai_data.card(built, conforms_to=[])10print("conformsTo, mlcroissant's default:", default["conformsTo"])11# the card as the platform writes it: Croissant 1.1 and the RAI vocabulary, written out12card, warnings = ai_data.card(built)13ai_data.write_card(card, built.folder)14print("conformsTo, written out:", ", ".join(card["conformsTo"]))15print("datePublished, the run's:", card["datePublished"])16# the build's date as the major number: a nightly build may remove records17print("version, the run's:", card["version"])18print("the sources the card names:", ", ".join(source["@id"] for source in card["prov:wasDerivedFrom"]))19print("the tables that decided which rows, as the card names them:",20 ", ".join(table["@id"] for table in card["prov:wasInfluencedBy"]))21print("the activity the card names:", card["prov:wasGeneratedBy"]["prov:label"])22# the card names every column of the corpus as a field23print("fields in the record set:", ", ".join(f["name"] for f in card["recordSet"][0]["field"]))24for line in card["rai:dataPreprocessingProtocol"] + card["rai:personalSensitiveInformation"]:25 print("-", line)2627# What mlcroissant reports, and what the platform's own check of the card reports.28# the message after mlcroissant's severity tag29print("mlcroissant's warnings:", len(warnings))30for warning in warnings:31 print("- mlcroissant:", warning.split("] ", 1)[1])32# the platform's own check: the library's validator doesn't report every requirement33problems = ai_data.card_problems(card, built.folder)34print("problems the book's check finds:", len(problems))35for problem in problems:36 print("-", problem)
python corpus-dataset-card.pyconformsTo, mlcroissant's default: http://mlcommons.org/croissant/1.0
conformsTo, written out: http://mlcommons.org/croissant/1.1, http://mlcommons.org/croissant/RAI/1.0
datePublished, the run's: 2026-10-01T02:00:00Z
version, the run's: 20261001.0.0
the sources the card names: iceberg://platform/learning.courses, iceberg://platform/learning.reviews
the tables that decided which rows, as the card names them: iceberg://platform/crm.learners
the activity the card names: platform/review_corpus
fields in the record set: review_id, course_id, course_title, rating, body, source_table, job, run_id
- Read 436 of 3,000 reviews, filtered on crm.learners.ai_training_consent, learning.reviews.learner_id.
- Replaced personal data with keyed pseudonyms (PERSON 120).
- Removed 340 exact duplicates and 0 near-duplicates (MinHash LSH, Jaccard similarity of 0.8 or more, confirmed exactly; 8 pairs that differ only in a name kept apart).
- Kept 96 records, each with its source table, review ID, job and run.
- Values replaced by keyed pseudonyms: PERSON 120.
- Emails and phone numbers replaced by [email] and [phone] by the enforcement layer's redact_contacts mask.
- Names found with a list of 106 words from crm.instructors, built by the governance team's job governance/instructor_names; the list isn't part of the corpus.
- Measured against labelled spans: 0 of 139 labelled values left in the text.
mlcroissant's warnings: 1
- mlcroissant: Property "http://mlcommons.org/croissant/citeAs" is recommended, but does not exist.
problems the book's check finds: 1
- creator: missing
Left to its default, mlcroissant writes http://mlcommons.org/croissant/1.0, although its own LATEST_VERSION is 1.1 (mlcroissant source), so a card would declare a version it wasn't written for. The platform writes http://mlcommons.org/croissant/1.1, http://mlcommons.org/croissant/RAI/1.0.
The card's datePublished, 2026-10-01T02:00:00Z, is the build's time, because the platform publishes a card only with the build it describes. Its version, 20261001.0.0, takes the build's date as its major number. Croissant recommends MAJOR.MINOR.PATCH, where a new major version means "the existing data has been changed (edited, removed or shuffled records across splits)", and any nightly build may remove records.
The card names the sources iceberg://platform/learning.courses, iceberg://platform/learning.reviews, the influence iceberg://platform/crm.learners and the activity platform/review_corpus, and lists preprocessing steps with the run's counts.
The warnings mlcroissant reports, 1 in all, are about recommended properties, here citeAs, not about the problem the platform's check finds: creator is missing. ai.review_corpus has no owner in the metadata file, the gap chapter 2 found, so the card stays unpublished until the table has an owner. Chapter 10 reports the same gap on every run, as a finding already on main.
mlcroissant can also read a card back, with mlc.Dataset, but at 1.1.1, with rdflib 7.6.0 and pandas 3.1.1, reading emits a DeprecationWarning from rdflib and a Pandas4Warning, and returns text fields as bytes. So the book's examples only write cards, and check them with the platform's own code.
Working rule: Generate the card in the run that builds the data, with every count, hash, source and version taken from that run, and write the format's version out. Check the card with your own code for every property the specification requires, since the library's validator doesn't report them all, and don't publish or train on a corpus whose card fails.
EU AI Act가 학습 데이터에 대해 말하는 내용을 인용한다
EU의 인공지능법(Artificial Intelligence Act), 즉 Regulation (EU) 2024/1689는 여러 곳에서 학습 데이터를 다룬다. 이 절은 그중 두 곳, 곧 Article 10과 Article 53(1)(d)를 인용하며, 어느 쪽도 해석하지 않는다.
고위험 AI 시스템에 대해 Article 10(2)는 학습, 검증, 시험 데이터 세트가 "데이터 거버넌스 및 관리 관행의 적용을 받아야 한다"고 규정하고, 그 관행이 다루는 사항을 열거한다. 여기에는 데이터 수집 과정과 데이터의 출처, 개인 데이터의 경우 수집의 원래 목적, 그리고 주석 달기, 레이블링, 정제, 갱신, 보강, 집계 같은 데이터 준비가 포함된다(AI Act).
Article 10(3)은 "의도된 목적에 비추어 관련성이 있고, 충분히 대표성이 있으며, 가능한 한 최대한 오류가 없고 완전한" 데이터 세트를 요구한다(AI Act).
2026년 7월 24일에 공표된 Regulation (EU) 2026/1744, 곧 AI에 관한 디지털 옴니버스(Digital Omnibus on AI)는 위 구절을 인용한 그대로 둔다. 이 규정은 Article 10의 1항과 6항을 대체하고, 5항을 삭제하며, "편향 탐지 및 시정을 위한 특별 범주 개인정보의 처리"라는 제목의 Article 4a를 신설한다. 또한 Article 10을 포함하는 제III장 제1절부터 제3절까지가 적용되는 시기를 정한다. Article 6(2)와 Annex III에 따라 분류되는 고위험 AI 시스템에는 2027년 12월 2일부터, Article 6(1)과 Annex I에 따라 분류되는 시스템에는 2028년 8월 2일부터 적용된다(Regulation (EU) 2026/1744).
범용 AI 모델의 제공자에 대해 Article 53(1)(d)는 "AI Office가 제공하는 양식에 따라, 범용 AI 모델의 학습에 사용된 콘텐츠에 관한 충분히 상세한 요약을 작성하여 공개해야 한다"고 요구한다. 유럽연합 집행위원회는 이 양식을 C(2025) 8311로 공표했다(집행위원회 양식, PDF).
설명 고지에 따르면, 이미 연합 시장에 출시된 모델을 수정하는 하위 단계의 주체는 그 고지가 명시한 범용 AI 모델에 관한 집행위원회 지침에 따라 결과 모델의 제공자가 될 수 있다. 이 경우 수정한 주체는 수정에 사용한 학습 콘텐츠만 보고한다.
이 법의 Annex III의 제목은 "Article 6(2)에서 말하는 고위험 AI 시스템"이며, 그 3호 "교육 및 직업 훈련"에는 "(b) 모든 수준의 교육 및 직업 훈련 기관에서 학습 성과를 평가하기 위한 AI 시스템. 그 성과가 자연인의 학습 과정을 조정하는 데 사용되는 경우를 포함한다"가 들어 있다(AI Act). 이 플랫폼이 교육 플랫폼이므로 인용하며, 여기까지만 다룬다. 이 플랫폼의 어떤 시스템이 Article 6에 따라 고위험인지는 별개의 문제이고, 이 장의 코드는 어느 것도 그 답에 의존하지 않는다.
작업 규칙: 법은 개정된 상태로 관보에서 인용하고, 링크와 읽은 날짜를 함께 적는다. 옴니버스가 조문과 날짜를 모두 바꾸었기 때문이다. 법을 파이프라인의 로직 밖에 둔다. 카드의 사실은 실행 결과에서 생성하고, 어떤 조문이 적용되는지는 법적 질문에 답하는 사람들이 결정하게 한다.
운영 환경에서 무엇이 잘못되는가
코퍼스는 아무런 경고를 주지 않는다. 모델은 학습되고, 카드는 공개되며, 문제는 나중에 모델이 내놓는 말이나 아무도 답할 수 없는 요청을 통해 드러난다.
이 부분은 아직 번역되지 않았으므로 영어로 표시된다.
A learner withdraws after the build
Like marketing's list in chapter 7, a corpus records who had consented on the day it was built. Take the corpus that lib.ai_data.build writes in the nightly run of Oct 1, the Oct 1 build, and withdraw the consent of the author of one of its records. Then rebuild the corpus in the same folder the next night, without generating its card again, and check each build against today's consent, as a training run does before it starts:
1"""Withdraws one author's consent after a build, rebuilds the corpus in the same folder, and checks both builds against today's consent and the card against the file. The prose's point rests on the "records whose author consents today" and "problems" lines."""2from datetime import timedelta34import pyarrow.compute as pc5import pyarrow.parquet as pq67from lib import ai_data, lineage8from lib.catalog import open_catalog9from lib.platform import load1011catalog = open_catalog(["crm.learners", "learning.courses", "learning.reviews"])12first = ai_data.build(catalog) # the nightly run of Oct 1, and its card13card, _ = ai_data.card(first)14ai_data.write_card(card, first.folder)1516# The next day, the author of one of the corpus's records withdraws their consent to AI training.17# review ID to the learner who wrote it, from the reviews table18authors = dict(zip(*load("learning.reviews").select(["review_id", "learner_id"]).to_pydict().values()))19# the author of the Oct 1 build's second record20learner = authors[first.records.column("review_id")[1].as_py()]21learners = catalog.load_table("crm.learners")22rows = learners.scan().to_arrow()23# that learner's answer becomes False; every other row keeps its answer24consent = pc.if_else(pc.equal(rows["learner_id"], learner), False, rows["ai_training_consent"])25learners.overwrite(rows.set_column(rows.schema.get_field_index("ai_training_consent"), "ai_training_consent", consent))2627# The next nightly run rebuilds the corpus in the same folder, and nobody generates the card again. A training run28# checks its build against today's consent before it starts.29# the same folder: the corpus file is replaced, so the first card no longer matches it30second = ai_data.build(catalog, first.folder, lineage.NIGHTLY + timedelta(days=1))31for name, built in [("Oct 1 build", first), ("Oct 2 build", second)]:32 ids = built.records.column("review_id").to_pylist()33 # records in each build whose author is the learner who withdrew34 print(f"{name}, records from the learner who withdrew:", sum(authors[i] == learner for i in ids))35 print(f"{name}, records whose author consents today:", f"{ai_data.consenting(catalog, ids)} of {len(ids)}")36different_runs = first.run_id != second.run_id37assert different_runs38print("the two builds are different runs:", different_runs)39# the legal team's placeholder answer, from governance/ai_data.toml40print("a model made from the Oct 1 build, the legal team's answer:", ai_data.LEGAL["withdrawal"]["models_made_before"])4142# the card's last preprocessing step, which states the Oct 1 counts43print("the Oct 1 card says:", card["rai:dataPreprocessingProtocol"][-1])44print("records in the corpus file now:", pq.read_metadata(first.folder / ai_data.CORPUS_FILE).num_rows)45# the platform's check of the Oct 1 card against the corpus file as it is now46problems = ai_data.card_problems(card, first.folder)47print("problems with the Oct 1 card, checked against the corpus now:", len(problems))48for problem in problems:49 print("-", problem)
python consent-withdrawn-after-build.pyOct 1 build, records from the learner who withdrew: 1
Oct 1 build, records whose author consents today: 95 of 96
Oct 2 build, records from the learner who withdrew: 0
Oct 2 build, records whose author consents today: 95 of 95
the two builds are different runs: True
a model made from the Oct 1 build, the legal team's answer: undecided
the Oct 1 card says: Kept 96 records, each with its source table, review ID, job and run.
records in the corpus file now: 95
problems with the Oct 1 card, checked against the corpus now: 2
- creator: missing
- review_corpus.parquet: its SHA-256 isn't the one the card states
The Oct 1 build holds 1 record by that learner, and the Oct 2 build holds 0, so the next corpus leaves the learner out. The two builds are different runs, with their own run IDs. The next training run has to leave the learner out too, and so does the next index.
Checked through provenance against today's answers, the Oct 1 build has 95 of 96 records whose author consents, so a training run that makes the check refuses it. The Oct 2 build passes with 95 of 95.
What was made from the Oct 1 build before the withdrawal keeps the review: a copy of its file, an index built from it, a model fine-tuned on it. The European Data Protection Board considers that "AI models trained with personal data cannot, in all cases, be considered anonymous" (EDPB Opinion 28/2024).
Article 7(3) of the GDPR gives the right to withdraw consent at any time. It says that withdrawing doesn't affect the lawfulness of processing based on consent before the withdrawal, that the person must be told of this right before giving consent, and that withdrawing must be no harder than giving it (GDPR). Chapter 7 quotes the first sentences of the same article and says what a withdrawal changes in the platform's code.
The code's part is to find, through provenance, every build that holds the record, and, by run ID, every model and index made from those builds. It then reads what happens to them from [legal.withdrawal] in governance/ai_data.toml. The legal team's answer for a model made from the Oct 1 build is undecided, a placeholder under which the job lists such models for the legal team and changes nothing. If the answer is that a model must not keep the review, the one way to remove it that we can vouch for is to train the model again on a build without it.
The card failed too, without a sound. Its last preprocessing step still reads "Kept 96 records, each with its source table, review ID, job and run." The file holds 95 records, and the platform's check finds 2 problems: the missing creator, and a file whose SHA-256 isn't the one the card states. A card that isn't generated again with its data describes another dataset.
Working rule: Record which build, by run ID, every model and index was made from. Before every training run, check the build's records against today's consent and the file's hash against its card, and refuse a build that fails either. When a learner withdraws, find the builds that hold their records and the models made from them, and let your legal team's answer in config decide what happens to those.
이 부분은 아직 번역되지 않았으므로 영어로 표시된다.
Other failures
- A filter on refusals that keeps the unknowns. Built with it, the corpus held 2,045 reviews instead of 436, mostly by learners never asked. Notice it by counting the corpus's records per consent answer, through their provenance: any answer but yes is a defect.
- Personal data a model memorizes from its text. Carlini and colleagues extracted training text from a language model, including public personal information such as names, phone numbers and email addresses, even though each such sequence appeared in a single document (Carlini et al., 2021). The plain pipeline's corpus holds 120 labelled emails, 113 of them distinct and 106 written only once, and duplicates repeat whatever a detector misses in every copy. Measure what's left against labelled samples of your own text, and deduplicate before training.
- A pseudonym key that leaks. With the corpus's key and a list of instructors' names, anyone can compute every pseudonym in the corpus. When the key may have leaked, rotate it, rebuild every corpus made with it, and find by run ID the copies and models made from builds under the old key, as for a withdrawal.
이 부분은 아직 번역되지 않았으므로 영어로 표시된다.
What this chapter decided
- The corpus is built through the enforcement layer, as
analyst-1forai-training: only reviews whose author'sai_training_consentis true, contacts masked, an unknown consent counted as no. - Every legal answer the code needs is a placeholder for the legal team in
[legal]ingovernance/ai_data.toml: the consent that admits a record, the card's licence, and what happens to models made before a withdrawal. - Personal data is detected after the enforcement layer's masks, with the pattern recognizers and the platform team's list of instructors' names, and replaced by keyed pseudonyms. Privacy Filter is measured, not added. The corpus is pseudonymized, never anonymized.
- Pseudonyms are HMAC-SHA256 under the corpus job's own key, not the enforcement layer's SHA-256 over key and value, and never Presidio's randomly salted
hash. - Duplicates are removed in two steps, exact copies by SHA-256, then near-duplicates from MinHash LSH tuned for 0.6 and confirmed at a Jaccard similarity of 0.8, never merging texts that differ only in a name. datasketch and its scheme,
affine32, are pinned. - Every record keeps its provenance,
source_table,review_id,jobandrun_id, and a training run checks its build against today's consent first. - The dataset card is Croissant 1.1 with RAI 1.0, generated by the run, checked by the platform's own code, and unpublished until
ai.review_corpushas an owner. - Answers for one person come from an index of the corpus, filtered by that person's decision on each record's source review.
- The EU AI Act is quoted as amended and linked, never interpreted. A corpus is only as clean as what was admitted into it. Each record is decided when the corpus is built, each record keeps its source, so a decision can be traced back to the review it came from, and the card says what was done with the rest.
이 판의 이 장
2026.11.31판예제 통과 2026년 11월 19이 장에서 10개 예제개 중 10개가 통과한다
영어판 2026.11.31판을 번역한 것이다.