找出個人資料

閱讀約需 112 分鐘 分鐘,13 節 中共有 11 個範例。

版次 2026.11.31 範例於 2026年11 月 19 通過測試 108 個範例中有 108 個通過 變更內容

由 AI 翻譯。經人工以英文驗證。 我們如何製作書籍

譯自英文書,版次 2026.11.31。 閱讀原文

讀完本章後,你能在資料平台上自動找出個人資料,並說明漏掉了多少。你會依欄位的名稱、型別,以及可重現的值樣本,為每個欄位提出分類建議;在自由文字中逐個片段找出個人資料;並在任何人依賴偵測器之前,以標註資料衡量每個偵測器。過程中,你會離線執行 Presidio 的樣式辨識器,並以衡量結果決定門檻值;將它們與一個專為找出個人資料而訓練的模型所記錄的偵測結果比較;而且只針對結構變更新增的欄位重新分類。

平台上的每個欄位都已有分類。第 2 章已在 governance/metadata.toml 中為每個欄位填入分類,這是一份供人審查的檔案,而審查過的檔案只記錄審查者認為某個欄位存放什麼內容。在線上學習平台(本書所治理的資料倉儲就屬於它)中,有三項這類認定從未被檢驗。billing.payments.card_number 是舊有欄位,其說明寫著它存放完整的卡號,但沒有人查證過。crm.learners.notes 是客服人員輸入的自由文字,因此它的名稱完全無法顯示內容是什麼。學習者在客服單與課程評論中想寫什麼就寫什麼。對整個欄位所做的分類,無法保護這些文字。要先遮蔽它們,平台必須逐個片段找出其中的每個姓名、電子郵件與電話號碼,如同第 9 章在課程助理學習評論之前所做的處理。

本章為這項工作建立本書共用的模組 lib.pii:針對欄位名稱的規則、以固定種子取得的欄位值樣本、設定為離線執行的 Presidio 偵測器,以及能依標註資料為任何偵測器評分的程式碼。本章的每個偵測器都以同一個檔案衡量,我們稱之為基準真值:data/platform/pii_spans.parquet。平台的資料產生器會把它與客服單和評論一併寫出,其中記錄了產生器放進文字裡的每一項個人資料,以及其確切的起點與終點。發票編號、訂單編號與問題發生的日期維持不標註,因為平台不把它們視為個人資料。

本書中的電話號碼、卡號與 IBAN 都是隨機產生的,如資料卡所述,而且沒有任何範例完整印出其中一個:除了最後四位數字之外,其餘皆被遮蔽。

從結構描述分類欄位

最近一次變動於第 2026.11.27 版

關於欄位,成本最低的證據就是它的結構描述(schema):欄位名稱與型別。依名稱建立的規則不讀取任何資料,因此可以在一次 pull request 中跑過整個目錄(catalog),並找出名稱本身就說明內容的欄位,例如 email 或 date_of_birth。目錄工具也是從這裡開始:OpenMetadata 的自動分類先以欄位名稱掃描器,用正規表示式檢查名稱(OpenMetadata 文件)。型別則決定下一步。文字欄位的值可以抽樣並掃描,但時間戳記不可能存放電子郵件,若用日期樣式去掃描,每一列都會被標記。因此數值、日期、時間戳記與布林值只依名稱分類。

平台將規則存放在 governance/pii.toml,這個檔案包含偵測器的所有設定,並與中繼資料一樣經過審查。這些規則是 8 條作用於小寫欄位名稱的正規表示式,每條都指出它所暗示的實體,並以第一個符合者為準。同一個檔案也將每個實體對應到一個分類:電話號碼對應到 personal,卡號或 IBAN 對應到 sensitive。lib.pii.by_name 負責套用這些規則。規則會處理目錄中的每個欄位,結果與經過審查的中繼資料並列存放:

column-name-rules.py Python 3.12.15 · duckdb 2.0.0 · phonenumbers 9.0.43 · presidio-analyzer 2.2.366 · pyarrow 26.0.1 · pyiceberg 0.12.0 · pyyaml 6.0.3 · tldextract 5.4.0 通過
"""Runs the name rules over every column of the catalog and compares each column they mark with the reviewed metadata.The prose rests on the count of marked columns that agree, and on the personal columns no rule marks."""from pyiceberg.types import StringTypefrom lib import metadata, piifrom lib.catalog import open_catalogcatalog = open_catalog()described = metadata.load()                   # the reviewed metadata, keyed by table and then by field ID# every column as (table, field): the field carries the ID the metadata is keyed by, so a renamed column still matchescolumns = [(table, f) for (namespace,) in sorted(catalog.list_namespaces())           for _, name in sorted(catalog.list_tables(namespace))           for table in [f"{namespace}.{name}"] for f in catalog.load_table(table).schema().fields]print("columns in the catalog:", len(columns))print("name rules:", len(pii.NAME_RULES))named = [(table, f, pii.by_name(f.name)) for table, f in columns if pii.by_name(f.name)]print("columns a name rule marks:", len(named))for table, f, entity in named:    print(f"  {table}.{f.name}: {entity}, so {pii.CLASSIFICATION[entity]}")agree = sum(pii.CLASSIFICATION[entity] == described[table].columns[f.field_id].classification            for table, f, entity in named)print("of them, agreeing with the metadata:", agree)# Only text is sampled: numbers, dates, timestamps and booleans are classified by name alone.text = [(table, f) for table, f in columns if isinstance(f.field_type, StringType)]print("columns of text, whose values can be sampled:", len(text))# Columns the metadata, reviewed by people, calls personal or sensitive, that no name rule marks.missed = [f"{table}.{f.name}" for table, f in columns if not pii.by_name(f.name)          and described[table].columns[f.field_id].classification in ("personal", "sensitive")]print("personal columns no name rule marks:", ", ".join(missed))
輸出python column-name-rules.py
columns in the catalog: 64
name rules: 8
columns a name rule marks: 11
  billing.payments.card_number: CREDIT_CARD, so sensitive
  crm.instructors.full_name: PERSON, so personal
  crm.instructors.email: EMAIL_ADDRESS, so personal
  crm.instructors.payout_iban: IBAN_CODE, so sensitive
  crm.learners.full_name: PERSON, so personal
  crm.learners.email: EMAIL_ADDRESS, so personal
  crm.learners.phone: PHONE_NUMBER, so personal
  crm.learners.city: LOCATION, so personal
  crm.learners.street_address: LOCATION, so personal
  crm.learners.date_of_birth: DATE_TIME, so personal
  learning.events.ip_address: IP_ADDRESS, so personal
of them, agreeing with the metadata: 11
columns of text, whose values can be sampled: 29
personal columns no name rule marks: crm.learners.notes, learning.reviews.body, support.tickets.body

規則只要觸發,結果就與中繼資料一致,但它們會漏掉自由文字。在目錄的 64 個欄位中,規則標記了 11 個,其中 11 個全都與人工給定的分類一致。它們漏掉了審查者分類為 personal 的三個欄位:crm.learners.notes、learning.reviews.body 與 support.tickets.body。這三個都是自由文字,名稱描述的是容器,而不是內容。平台最擔心的已標記欄位 billing.payments.card_number,僅憑名稱就得到 sensitive,並沒有人去了解它實際存放什麼。完整的卡號、末四碼,或支付服務提供者的權杖(token),都會符合 card_?(number|num|no)。

名稱規則以已知的方式顯得脆弱。欄位型別偵測模型 Sherlock 的作者指出,名稱比對對髒資料不夠穩健,而且只能偵測少數型別(Hulsebos et al., 2019)。因此平台也會讀取值,範圍是其目錄中所有 29 個文字欄位。

實用原則: 先對每個欄位執行名稱規則,因為它們不需成本,並且只讓它們提高分類,絕不降低分類。規則要保持簡短、放在程式碼中,並像其他規則一樣經過審查。任何自由文字欄位,無論名稱為何,都應視為需要抽樣其值的欄位;其他型別的欄位則只依名稱分類。

此部分尚未翻譯,因此以英文呈現。

Run Presidio offline

最近一次變動於第 2026.11.31 版

To read values, the platform uses Presidio, an open-source library for finding and masking personal data. Presidio's analyzer runs recognizers: regular expressions with checksums, phone-number parsing, and a named-entity model reached through an NLP engine. The NLP engine does two jobs, named-entity recognition and feature extraction for rules, such as the context words that raise a score. Its default model is spaCy's en_core_web_lg (Presidio docs). At 2.2.366, AnalyzerEngine() with no arguments downloads that model when it isn't installed (source). So a pipeline without network access fails on its first line, and one with access installs a model of about 400 MB as a side effect (en_core_web_lg 3.8.0).

lib.pii.detector configures the engine itself, with NoOpNlpEngine, which loads no language model (source). What that costs is context. With a model, Presidio raises a pattern's score when a word such as "phone" appears near the match, by comparing the lemma of each nearby word with each recognizer's context terms (Presidio's tutorial on context). With no model there are no lemmas, so the engine can't use the text's context words at all. The engine has one more way onto the network. Its email recognizer checks domains with tldextract, which fetches the Public Suffix List the first time it's used (tldextract 5.4.0's README). Importing Presidio imports tldextract, whose README says an empty TLDEXTRACT_PUBLIC_SUFFIX_LIST_URLS turns the fetch off if it's set before the import. So lib/__init__.py, which Python runs before any module of the book's lib, empties the variable. If an example imported Presidio first, the same file replaces tldextract's shared extractor with one that has no URLs. lib.pii refuses to load if any remain, and tldextract uses the copy of the list it ships with, so the same text gives the same answer on every machine.

The defaults are tested on real text: load every predefined recognizer on one of the platform's support tickets, which holds a name, an email, a phone number and a date:

presidio-offline-engine.py Python 3.12.15 · duckdb 2.0.0 · phonenumbers 9.0.43 · presidio-analyzer 2.2.366 · pyarrow 26.0.1 · pyiceberg 0.12.0 · pyyaml 6.0.3 · spacy 3.8.18 · tldextract 5.4.0 通過
"""Loads Presidio's predefined recognizers on an engine that loads no language model, and runs them on one supportticket. The prose's point rests on the detections printed under the ticket, and on the assert that the model isn'tinstalled."""import spacyfrom lib import piifrom lib.platform import load# AnalyzerEngine() with no arguments loads spaCy's en_core_web_lg, and downloads it when it isn't installed.installed = spacy.util.is_package("en_core_web_lg")assert installed is False       # the pipeline assumes no model: a machine that has one fails here, not silentlyprint("en_core_web_lg installed:", installed)print("URLs tldextract would fetch the Public Suffix List from:", len(pii.suffix_list_urls()))detector = pii.detector()              # every predefined recognizer, on an NLP engine that loads no modelnames = sorted(r.name for r in detector.registry.recognizers)print("recognizers loaded:", len(names))print("recognizers:", ", ".join(names))print("default score threshold:", detector.default_score_threshold)# The ticket holds a phone number, which may be in service: masked() prints it with only its last four digits.ticket = pii.texts("tickets")[1421]         # one ticket with a name, an email, a phone number and a date of the problemfound = sorted(pii.detect(detector, ticket))print("ticket 1421:", pii.masked(ticket))for start, end, entity, score in found:    print(f"  {entity} on {pii.masked(ticket[start:end])}: {score:.2f}")# Where is the phone number valid? The recognizer reads it in each of its regions.phone = next(ticket[start:end] for start, end, entity, _ in found if entity == "PHONE_NUMBER")[learner] = [row["learner_id"] for row in load("support.tickets").to_pylist() if row["ticket_id"] == 1421][country] = [row["country"] for row in load("crm.learners").to_pylist() if row["learner_id"] == learner]print("the learner's country:", country)# The recognizer's built-in regions, not the learner's country.print("default regions where it's valid:",      ", ".join(r for r in pii.PhoneRecognizer.DEFAULT_SUPPORTED_REGIONS if pii.valid_phone(phone, r)))
輸出python presidio-offline-engine.py
en_core_web_lg installed: False
URLs tldextract would fetch the Public Suffix List from: 0
recognizers loaded: 17
recognizers: CreditCardRecognizer, CryptoRecognizer, DateRecognizer, EmailRecognizer, IbanRecognizer, IpRecognizer, MacAddressRecognizer, MedicalLicenseRecognizer, NhsRecognizer, PhoneRecognizer, UrlRecognizer, UsBankRecognizer, UsItinRecognizer, UsLicenseRecognizer, UsPassportRecognizer, UsSsnRecognizer, UuidRecognizer
default score threshold: 0
ticket 1421: I can't open the videos in Introduction to Design Systems, they stop after a few seconds. It happened on 12/03 around 9pm. My colleague Jennifer Abbott (jennifer.abbott@example.net) wants to enrol as well. You can reach me on •••••••0440 after 6pm.
  DATE_TIME on 12/03: 0.10
  EMAIL_ADDRESS on jennifer.abbott@example.net: 1.00
  URL on example.net: 0.50
  PHONE_NUMBER on •••••••0440: 0.40
  US_BANK_NUMBER on •••••••0440: 0.05
  US_DRIVER_LICENSE on •••••••0440: 0.01
the learner's country: US
default regions where it's valid: BR

Presidio's defaults on this engine see the email and the phone number, and miss the colleague's name. en_core_web_lg isn't installed, so a bare AnalyzerEngine() would try to download it, and tldextract has 0 URLs to fetch its list from. The no-model engine loads 17 recognizers, and none of them looks for names, so Jennifer Abbott, the colleague in the ticket, goes unnoticed. Her email is found with a score of 1.00, and its domain is found again as a URL, at 0.50. The learner's phone number, an 11-digit number typed without separators, is found three times: as a phone number at 0.40, a US bank number at 0.05 and a US driver's licence at 0.01. Even the phone number is found by chance. The learner's country is US, but of Presidio's default regions, the US among them, the digits are valid only in BR. The date of the problem, 12/03, is read as a date too, which says nothing about anyone.

Those scores aren't probabilities. Each pattern carries a constant score, and weak patterns get low ones, as Presidio's tutorial explains for a five-digit zip code (Presidio docs). A US bank number is any run of 8 to 17 digits (supported entities), and the FAQ admits that a driver's licence "could be any 9-digit number" (FAQ). The analyzer's default threshold is 0, so every weak match comes back.

Working rule: Configure Presidio's engine explicitly: NoOpNlpEngine when you run patterns only, or a pinned model you install yourself when you need its names. Never call AnalyzerEngine() bare in a pipeline, and pin anything a recognizer would download, such as the Public Suffix List. Read each score as a constant set for its pattern, never as a probability.

每次執行都以相同方式抽樣欄位的值

最近一次變動於第 2026.11.27 版

掃描每個文字欄位的每一個值,對任何資料倉儲而言都太慢,因此工具會採用抽樣。OpenMetadata 的工作流程會抽取資料列,並以實體辨識引擎檢查這些資料列(OpenMetadata 文件)。抽樣需要做三項決定。抽哪些資料列:要用隨機抽樣,而不是取最前面的資料列,或從隨機起點開始的一段連續區塊。Google 的 Sensitive Data Protection 提供一種隨機起點模式 RANDOM_START,它從隨機的某一列開始,往下連續讀取(Google Cloud 文件)。在依註冊日期排序的資料表上,這樣的區塊只會是同一批學員。每次執行都抽到相同的資料列,這樣分類結果才能重現,而結果出現變化就代表資料已經改變。還有抽多少,這取決於必須抓出的資料有多罕見。

DuckDB 的 reservoir 抽樣可以指定種子(seed),但如同 DuckDB 2.0.0 文件所警告的,只有在 DuckDB 以單一執行緒執行時,種子才能產生可重現的樣本。lib.pii.sample 會將執行緒數設為一個,濾除 NULL,再抽取帶有種子的 reservoir 樣本。

罕見的值需要更大的樣本,而算式可以說明要大多少。設含有個人資料的值所占比例為 p。抽樣的 n 個值中至少包含一個的機率為

chance = 1 − (1 − p)ⁿ

而達到 99% 機率所需的 n 為 ln(0.01) / ln(1 − p)。這是經驗法則:它假設每次抽取彼此獨立,而從數千列中進行不放回抽樣,幾乎就符合這項假設。

接著運用這個公式:從 crm.learners 抽取 20 則備註,共抽兩次,再比較以所有資料列計算的機率,與只以備註計算的機率。

seeded-column-sample.py Python 3.12.15 · duckdb 2.0.0 · phonenumbers 9.0.43 · presidio-analyzer 2.2.366 · pyarrow 26.0.1 · pyiceberg 0.12.0 · pyyaml 6.0.3 · tldextract 5.4.0 通過
"""Draws the same twenty notes twice from a seeded reservoir sample, then counts the notes that hold personal data, to seehow many sampled values a rare value needs. The prose rests on the assert that the draws match, the two chances and the sample sizes."""import mathfrom lib import piifrom lib.catalog import open_catalogfrom lib.warehouse import connectcon = connect(open_catalog(["crm.learners"]))first = pii.sample(con, "crm.learners", "notes", rows=20)second = pii.sample(con, "crm.learners", "notes", rows=20)     # the same call: the seed and one thread (set in lib.pii.sample) make the draws matchassert second == firstprint("second draw, same notes as the first:", "yes" if second == first else "no")# Notes can hold a phone number, which may be in service: masked() prints it with only its last four digits.print("first sampled notes:", " | ".join(pii.masked(note) for note in first[:3]))# How rare is personal data in the column? Run the detector over every note, once, to know.detector = pii.platform_detector()[(rows, notes)] = con.sql("select count(*), count(notes) from crm.learners").fetchall()   # count(notes) skips NULLsevery_note = [note for (note,) in con.sql("select notes from crm.learners where notes is not null").fetchall()]holding = sum(bool(pii.detect(detector, note)) for note in every_note)print("learners:", f"{rows:,}")print("learners with notes:", notes)print("notes holding personal data the detector finds:", holding)# The chance that n sampled values hold at least one of them is 1 - (1 - p)^n, for a share p of the values.for unit, p in (("rows", holding / rows), ("notes", holding / notes)):   # p counted over all learners, then over the notes    print(f"chance 20 sampled {unit} catch one:", f"{1 - (1 - p) ** 20:.1%}")    print(f"sampled {unit} needed for a 99% chance:", math.ceil(math.log(0.01) / math.log(1 - p)))   # solves for nprint("values the platform samples per text column:", pii.ROWS)# The rarest share p that a sample of ROWS values still catches 99% of the time, printed as 1 in a whole number.print("rarest share that sample catches 99% of the time:", f"1 in {math.floor(1 / (1 - 0.01 ** (1 / pii.ROWS)))}")
輸出python seeded-column-sample.py
second draw, same notes as the first: yes
first sampled notes: VIP: bought a team licence. | Asked about invoices for their employer. | Called about a refund, call back on (•••)•••-1366x•••.
learners: 5,000
learners with notes: 527
notes holding personal data the detector finds: 138
chance 20 sampled rows catch one: 42.9%
sampled rows needed for a 99% chance: 165
chance 20 sampled notes catch one: 99.8%
sampled notes needed for a 99% chance: 16
values the platform samples per text column: 200
rarest share that sample catches 99% of the time: 1 in 43

兩次抽取的結果相同,因此樣本可以稽核。使用相同的種子時,第二次抽取會回傳同樣的 20 則備註,而第三則備註已含有電話號碼,輸出時已將它遮蔽。

在 5,000 位學員中,有 527 位有備註,其中有 138 含有平台偵測器(本章後段會調校)找得到的個人資料。NULL 會稀釋樣本,因為 NULL 不可能含有個人資料。抽取 20 個資料列,有 42.9% 的機率抓到個人資料,要達到 99% 的機率則需要 165 列。若只計算實際存在的備註,抽 20 則有 99.8% 的機率抓到,而 16 則就足以達到 99%。

**實用原則:**抽樣時使用非空值,固定種子,以單一執行緒執行,並將種子與結果一併保存。依據必須抓出的最罕見個人資料決定樣本大小,要達到 99% 就用 n = ln(0.01) / ln(1 − p)。平台對每個文字欄位抽取 200 個值,此數值設定於 governance/pii.toml,這能在 99% 的情況下抓到個人資料占比為 1 in 43 以上的值。若比這更罕見,就掃描整個欄位,或改以規則分類。

從欄位的值來分類

最近一次變動於第 2026.11.28 版

名稱規則與可重現的取樣合併後,每個欄位會得到一項建議。lib.pii.classify 會採用兩者所支持的最嚴格分類:名稱所暗示的實體,以及在任何取樣值中發現的每一種實體。沒有任何證據的欄位會得到 internal,因為偵測器什麼都沒找到,就等於什麼都沒證明,而只有人才能判定某個欄位是 public。

採用最嚴格者是一種選擇。Presidio 處理表格的工具 Presidio structured,為每個欄位只指定一種實體,其預設的 Most Common 會挑出出現次數最多的實體(Presidio 文件)。這適合用來決定如何遮罩一個欄位。但就分類而言,一個欄位即使在 200 個值中只有一個信用卡號,它仍然含有信用卡號。

以下是目錄中每個欄位的建議,由平台的偵測器產生,其辨識器由後續一節說明如何選擇,並顯示各建議與已審查的中繼資料有何不同:

classify-sampled-columns.py Python 3.12.15 · duckdb 2.0.0 · phonenumbers 9.0.43 · presidio-analyzer 2.2.366 · pyarrow 26.0.1 · pyiceberg 0.12.0 · pyyaml 6.0.3 · tldextract 5.4.0 通過
"""Proposes a classification for every column, the strictest of what its name and its sampled values support, and compareseach proposal with the reviewed metadata. The prose's table rests on the proposed and found lines of each column, andits counts on the totals printed after them."""from lib import metadata, piifrom lib.catalog import open_catalogfrom lib.warehouse import connectcatalog = open_catalog()findings = pii.classify(catalog, connect(catalog), pii.platform_detector())    # the platform's tuned detector, not the defaultsprint("columns classified:", len(findings))print("columns sampled:", sum(f.sampled > 0 for f in findings))holding = [f for f in findings if f.found]          # columns where sampled values hold personal dataprint("columns whose sampled values hold personal data:", len(holding))# A column counts when its values hold an entity other than the one its name suggests (named is None when no rule matches).print("of them, holding something their name doesn't suggest:", sum(set(f.found) != {f.named} for f in holding))for f in holding:    print(f"{f.table}.{f.column}, proposed: {f.classification}")    print(f"{f.table}.{f.column}, found: " + ", ".join(f"{e} in {n} of {f.sampled}" for e, n in f.found.most_common()))[cards] = [f for f in findings if f.column == "card_number"]     # the one card_number column in the catalog# The card recognizer's count: values that match its pattern and pass its Luhn check, out of the values sampled.print("sampled card numbers the card recognizer accepts:", f"{cards.found['CREDIT_CARD']} of {cards.sampled}")# Classifications the values raise above what the name alone gives: the floor is used for a name no rule matches.named_only = [f for f in findings if pii.CLASSIFICATION.get(f.named, pii.FLOOR) != f.classification]print("columns the values classify more strictly than the name:", ", ".join(f"{f.table}.{f.column}" for f in named_only))# Compare with the metadata people reviewed. A column the detector found nothing in proposes only the floor, internal.described = metadata.load()evidenced = [f for f in findings if f.named or f.found]print("columns with evidence:", len(evidenced))print("of them, equal to the metadata:",      sum(f.classification == described[f.table].columns[f.field_id].classification for f in evidenced))for f in evidenced:    reviewed = described[f.table].columns[f.field_id].classification    if metadata.stricter(f.classification, reviewed) != reviewed:     # the stricter of the two: a lower proposal prints nothing        print(f"stricter than the metadata, {f.table}.{f.column}: {f.classification}, reviewed as {reviewed}")
輸出python classify-sampled-columns.py
columns classified: 64
columns sampled: 29
columns whose sampled values hold personal data: 9
of them, holding something their name doesn't suggest: 5
billing.payments.card_number, proposed: sensitive
billing.payments.card_number, found: CREDIT_CARD in 151 of 200, PHONE_NUMBER in 3 of 200
crm.instructors.email, proposed: personal
crm.instructors.email, found: EMAIL_ADDRESS in 50 of 50
crm.instructors.payout_iban, proposed: sensitive
crm.instructors.payout_iban, found: IBAN_CODE in 50 of 50
crm.learners.email, proposed: personal
crm.learners.email, found: EMAIL_ADDRESS in 200 of 200
crm.learners.phone, proposed: personal
crm.learners.phone, found: PHONE_NUMBER in 175 of 200
crm.learners.notes, proposed: personal
crm.learners.notes, found: PHONE_NUMBER in 29 of 200, EMAIL_ADDRESS in 18 of 200
learning.events.ip_address, proposed: personal
learning.events.ip_address, found: IP_ADDRESS in 200 of 200, PHONE_NUMBER in 39 of 200
learning.reviews.body, proposed: personal
learning.reviews.body, found: EMAIL_ADDRESS in 4 of 200
support.tickets.body, proposed: sensitive
support.tickets.body, found: EMAIL_ADDRESS in 47 of 200, PHONE_NUMBER in 37 of 200, DATE_TIME in 29 of 200, IBAN_CODE in 21 of 200, CREDIT_CARD in 12 of 200
sampled card numbers the card recognizer accepts: 151 of 200
columns the values classify more strictly than the name: crm.learners.notes, learning.reviews.body, support.tickets.body
columns with evidence: 14
of them, equal to the metadata: 13
stricter than the metadata, support.tickets.body: sensitive, reviewed as personal

名稱未能確定的部分,由值來決定。取樣值含有個人資料的 9 個欄位中,有一些只含有其名稱所說的內容,例如 crm.learners.email。其餘 5 個則含有名稱並未暗示的內容,或是名稱本身看不出任何線索:

欄位 建議 在取樣中發現
billing.payments.card_number sensitive CREDIT_CARD in 151 of 200, PHONE_NUMBER in 3 of 200
crm.learners.notes personal PHONE_NUMBER in 29 of 200, EMAIL_ADDRESS in 18 of 200
support.tickets.body sensitive EMAIL_ADDRESS in 47 of 200, PHONE_NUMBER in 37 of 200, DATE_TIME in 29 of 200, IBAN_CODE in 21 of 200, CREDIT_CARD in 12 of 200
learning.reviews.body personal EMAIL_ADDRESS in 4 of 200
learning.events.ip_address personal IP_ADDRESS in 200 of 200, PHONE_NUMBER in 39 of 200

舊版信用卡欄位的描述寫著「Should not be stored at all」,而取樣顯示它存有完整的信用卡號,資料表的擁有者 team-billing 可以據此採取行動。在取樣到的信用卡號中,有 151 of 200 符合 Presidio 的信用卡模式,並通過它所檢查的 Luhn 檢查碼(原始碼),其餘則是該模式未涵蓋的格式(參見模式在哪些情況下失效)。僅憑值本身,備註欄位即為 personal。工單文字為 sensitive,因為學員會把信用卡號和 IBAN 貼進工單,這使它成為唯一比中繼資料更嚴格的建議,而人員審查時將它列為 personal。在有任何證據的 14 個欄位中,有 13 個與已審查的分類相符。

要閱讀證據,而不只是結論。有些 IP 位址也會被讀成電話號碼。這在此處不會造成影響,因為 IP 位址本來就是 personal,但偵測器的標籤說明的是值看起來像什麼,而不是它實際上是什麼。

**實用原則:**採用名稱或任何取樣值所支持的最嚴格分類,並以 internal 作為下限,同時將證據與建議一併呈現。比已審查的中繼資料更嚴格的建議,對資料表的擁有者而言是一項發現,擁有者應提高分類,或記錄不提高的理由;絕不讓偵測器降低分類。OpenMetadata 會略過任何已帶有 PII 標籤的欄位;我們建議改為每次執行時對每個文字欄位都取樣,因為已審查的標籤正是被檢查的對象。

此部分尚未翻譯,因此以英文呈現。

Measure a detector against labelled spans

最近一次變動於第 2026.11.31 版

A column-level answer is enough to classify support.tickets.body, but not to mask it: masking replaces each span of personal data in each text, and a span the detector misses stays readable. To know what a detector misses, compare its spans with the truth, the labelled spans of every ticket and review. Two matching rules are standard, and lib.pii applies both.

Exact matching, from the CoNLL-2003 shared task, counts an entity "correct only if it is an exact match of the corresponding entity in the data file" (Tjong Kim Sang and De Meulder, 2003). Overlap matching asks only for one shared character, with a label of its type, which the SemEval-2013 task on drug interactions calls "Type matching" (Segura-Bedmar et al., 2013). Neither credits a span hidden under another label, so lib.pii adds a third measure, coverage: whether every character of a labelled span is inside some detection, whatever its label. Exact asks whether the detector labelled the whole span right, overlap whether it noticed the span as what it is, and coverage whether masking its detections hides the span.

lib.pii.score computes exact and overlap matching as recall, the share of labelled spans of each type that a detector finds with a label of that type, and precision, the share of each label's detections that match a labelled span. lib.pii.coverage gives the share of each type's spans covered. score also counts the texts in which every labelled character falls inside some detection, the texts that masking would leave with nothing personal. The Text Anonymization Benchmark protects a person only when all of their mentions are masked (Pilán et al., 2022), and the platform counts whole texts, which is stricter. Recall comes first: Presidio's evaluation guide says it often matters more than precision, because missing personal data is the error to avoid (Presidio docs).

Presidio's defaults, scored against the labelled spans of every ticket and review, give the baseline the rest of the chapter improves on:

measure-pattern-recognizers.py Python 3.12.15 · duckdb 2.0.0 · phonenumbers 9.0.43 · presidio-analyzer 2.2.366 · pyarrow 26.0.1 · pyiceberg 0.12.0 · pyyaml 6.0.3 · tldextract 5.4.0 通過
"""Runs Presidio with its defaults over every ticket and review, and scores its detections against the truth's labelledspans. The prose rests on the recall and precision lines, the count of texts left with nothing personal, and the twodate lines that test the truth's limits."""import refrom lib import piilabelled = pii.truth()print("texts:", f"{sum(len(pii.texts(source)) for source in pii.SOURCES):,}")print("labelled spans:", f"{len(labelled):,}")detector = pii.detector()                 # Presidio's defaults: every recognizer, threshold 0found = pii.find(detector)# scoring needs a label of the span's type; coverage doesn'tscores, covered = pii.score(found, labelled), pii.coverage(found, labelled)print("detections:", f"{len(found):,}")for entity, (n, exact, overlap) in scores.recall.items():    print(f"{entity}, labelled: {n:,}")    print(f"{entity}, recall, exact: {exact / n:.2f}")    print(f"{entity}, recall, overlap: {overlap / n:.2f}")    print(f"{entity}, covered, any label: {covered[entity][1] / n:.2f}")    # covered[entity] is (spans, covered), so n is the same countfor label, (n, exact, overlap) in scores.precision.items():     # precision is per label: what each detection was called    print(f"{label}, detections: {n:,}")    print(f"{label}, precision, exact: {exact / n:.2f}")# every bank number and driver's licence detection is a false alarm: none matches a labelled spanassert all(scores.precision[entity][1] == 0 for entity in ("US_BANK_NUMBER", "US_DRIVER_LICENSE"))print("precision of all detections, exact:", f"{pii.precision_overall(scores):.2f}")print("texts holding personal data:", f"{scores.texts:,}")print("texts with every labelled span masked:", f"{scores.masked:,}")# The truth's limits. The generator builds every text from a few sentence templates, and writes every date of birth# one way. Presidio's date patterns don't read every way of writing one.print("sentence templates the texts are built from:", len(pii.templates()))births = [pii.text_of(s) for s in labelled if s.entity == "DATE_TIME"]    # the truth's dates are the dates of birth: dates of a problem are unlabelledwritten_ymd = sum(re.fullmatch(r"\d{4}-\d\d-\d\d", d) is not None for d in births)   # year first, the generator's one formassert written_ymd == len(births)           # so every date of birth is written this wayprint("dates of birth written like 1985-03-12:", written_ymd)for written in ("1985-03-12", "12 March 1985"):    text = f"My date of birth on the account is wrong, it should be {written}."     # a sentence in the generator's shape    print(f"dates found in a date of birth written {written}:", sum(e == "DATE_TIME" for _, _, e, _ in pii.detect(detector, text)))
輸出python measure-pattern-recognizers.py
texts: 5,000
labelled spans: 4,292
detections: 3,902
CREDIT_CARD, labelled: 213
CREDIT_CARD, recall, exact: 0.71
CREDIT_CARD, recall, overlap: 0.71
CREDIT_CARD, covered, any label: 0.89
DATE_TIME, labelled: 196
DATE_TIME, recall, exact: 1.00
DATE_TIME, recall, overlap: 1.00
DATE_TIME, covered, any label: 1.00
EMAIL_ADDRESS, labelled: 543
EMAIL_ADDRESS, recall, exact: 1.00
EMAIL_ADDRESS, recall, overlap: 1.00
EMAIL_ADDRESS, covered, any label: 1.00
IBAN_CODE, labelled: 236
IBAN_CODE, recall, exact: 1.00
IBAN_CODE, recall, overlap: 1.00
IBAN_CODE, covered, any label: 1.00
LOCATION, labelled: 210
LOCATION, recall, exact: 0.00
LOCATION, recall, overlap: 0.00
LOCATION, covered, any label: 0.00
PERSON, labelled: 2,455
PERSON, recall, exact: 0.00
PERSON, recall, overlap: 0.00
PERSON, covered, any label: 0.00
PHONE_NUMBER, labelled: 439
PHONE_NUMBER, recall, exact: 0.78
PHONE_NUMBER, recall, overlap: 0.79
PHONE_NUMBER, covered, any label: 0.91
CREDIT_CARD, detections: 151
CREDIT_CARD, precision, exact: 1.00
DATE_TIME, detections: 421
DATE_TIME, precision, exact: 0.47
EMAIL_ADDRESS, detections: 543
EMAIL_ADDRESS, precision, exact: 1.00
IBAN_CODE, detections: 236
IBAN_CODE, precision, exact: 1.00
MEDICAL_LICENSE, detections: 18
MEDICAL_LICENSE, precision, exact: 0.00
PHONE_NUMBER, detections: 384
PHONE_NUMBER, precision, exact: 0.89
UK_NHS, detections: 8
UK_NHS, precision, exact: 0.00
URL, detections: 751
URL, precision, exact: 0.00
US_BANK_NUMBER, detections: 689
US_BANK_NUMBER, precision, exact: 0.00
US_DRIVER_LICENSE, detections: 683
US_DRIVER_LICENSE, precision, exact: 0.00
US_ITIN, detections: 2
US_ITIN, precision, exact: 0.00
US_PASSPORT, detections: 8
US_PASSPORT, precision, exact: 0.00
US_SSN, detections: 8
US_SSN, precision, exact: 0.00
precision of all detections, exact: 0.38
texts holding personal data: 2,586
texts with every labelled span masked: 425
sentence templates the texts are built from: 30
dates of birth written like 1985-03-12: 196
dates found in a date of birth written 1985-03-12: 1
dates found in a date of birth written 12 March 1985: 0
Entity type Labelled Recall, exact Recall, overlap Covered
PERSON 2,455 0.00 0.00 0.00
EMAIL_ADDRESS 543 1.00 1.00 1.00
PHONE_NUMBER 439 0.78 0.79 0.91
IBAN_CODE 236 1.00 1.00 1.00
CREDIT_CARD 213 0.71 0.71 0.89
LOCATION 210 0.00 0.00 0.00
DATE_TIME 196 1.00 1.00 1.00

The defaults find every email, IBAN and date of birth, most phone and card numbers, and no names or street addresses at all, which is most of what the tickets hold. Most of what they flag is wrong: 0.38 of all detections match a labelled span. The 689 US bank numbers, 683 driver's licences and 751 URLs are all wrong. Phone, card and invoice numbers, all runs of digits, come out as bank numbers and licences, and pieces of emails and a few addresses come out as URLs. A wrong label still hides the digits, so phone and card numbers are covered more often than they're found.

Masking would leave personal data in most texts that hold it. Only 425 of the 2,586 texts would come out with nothing personal left.

The truth has its own limits, because the text is generated. Every ticket and review is built from 30 sentence templates in prep/generate.py, so each hard case is one sentence frame, and all 196 dates of birth are written like 1985-03-12. The date recall of 1.00 therefore measures one format: in the generator's own sentence, Presidio finds 1 date when the date of birth is written 1985-03-12, and 0 when it's written 12 March 1985. A measure on this text tells you how a detector behaves on these shapes, not how it will do on yours.

Working rule: Measure every detector against labelled spans before it masks anything: recall per entity type by exact and by overlap matching, both with a label of the right type, coverage whatever the label, precision per label, and the share of texts with nothing personal left. Weigh recall first. Before you trust a detector on your text, label a few hundred real records by hand and measure on those too.

依據真實標註調校辨識器

最近一次變動於第 2026.11.28 版

Presidio 的預設值會回傳每一個弱匹配。FAQ 提出兩種補救方式:調整接受門檻,以及移除資料集用不到的辨識器,並指出誤報與漏報之間存在取捨(FAQ)。Presidio 2.2.364 新增了依辨識器與依實體類型設定的分數門檻,並以分析器的預設值作為後備(變更紀錄、發行說明)。對於學習者分布在多個國家的平台,電話辨識器還有一項重要設定:它所讀取其編號計畫的地區(原始碼)。

governance/pii.toml 保留了 6 個辨識器,對應平台文字可能包含的識別碼,且各自有獨立的門檻。電話地區並不是手動設定的項目:lib.pii.countries 會從 crm.learners 讀取學習者所居住的國家,因此地區隨資料而定。四種設定以同一份真實標註評分:

tune-pattern-recognizers.py Python 3.12.15 · duckdb 2.0.0 · phonenumbers 9.0.43 · presidio-analyzer 2.2.366 · pyarrow 26.0.1 · pyiceberg 0.12.0 · pyyaml 6.0.3 · tldextract 5.4.0 通過
"""Scores four detector settings against the same truth: Presidio's defaults, one threshold for every recognizer, the keptrecognizers with their own thresholds, and those with phone numbers read in the learners' countries. The prose rests onthe precision, phone recall, coverage and fully masked texts of each setting."""from lib import piilabelled = pii.truth()print("recognizers kept:", len(pii.THRESHOLDS))                  # read from governance/pii.tomlprint("recognizers kept, with their thresholds:", ", ".join(f"{name} {t}" for name, t in pii.THRESHOLDS.items()))print("phone regions, Presidio's default:", ", ".join(pii.PhoneRecognizer.DEFAULT_SUPPORTED_REGIONS))   # Presidio's own listprint("phone regions, the learners' countries:", ", ".join(pii.countries()))   # read from crm.learners# In the order the prose builds them: the defaults, one threshold, the kept recognizers, then their regions.settings = {    "Presidio's defaults": pii.detector(),    "one threshold of 0.5": pii.detector(default_threshold=0.5),    "kept recognizers, own thresholds": pii.detector(pii.THRESHOLDS),    "kept recognizers, learners' phone regions": pii.platform_detector(),}for name, detector in settings.items():    found = pii.find(detector)    scores, covered = pii.score(found, labelled), pii.coverage(found, labelled)    n, exact, _ = scores.recall["PHONE_NUMBER"]         # the table reads exact recall, so the overlap count is left out    print(f"{name}, detections: {len(found):,}")    print(f"{name}, precision, exact: {pii.precision_overall(scores):.2f}")    print(f"{name}, phone recall, exact: {exact / n:.2f}")    # coverage takes a detection under any label    for entity, short in (("PHONE_NUMBER", "phones"), ("CREDIT_CARD", "cards")):        print(f"{name}, {short} covered, any label: {covered[entity][1] / covered[entity][0]:.2f}")    print(f"{name}, texts with every labelled span masked: {scores.masked:,}")
輸出python tune-pattern-recognizers.py
recognizers kept: 6
recognizers kept, with their thresholds: EmailRecognizer 0.5, PhoneRecognizer 0.4, CreditCardRecognizer 0.5, IbanRecognizer 0.5, DateRecognizer 0.5, IpRecognizer 0.5
phone regions, Presidio's default: US, GB, DE, FR, IL, IN, CA, BR
phone regions, the learners' countries: AU, BR, CA, DE, ES, FR, GB, IN, JP, MX, PL, US
Presidio's defaults, detections: 3,902
Presidio's defaults, precision, exact: 0.38
Presidio's defaults, phone recall, exact: 0.78
Presidio's defaults, phones covered, any label: 0.91
Presidio's defaults, cards covered, any label: 0.89
Presidio's defaults, texts with every labelled span masked: 425
one threshold of 0.5, detections: 1,903
one threshold of 0.5, precision, exact: 0.59
one threshold of 0.5, phone recall, exact: 0.00
one threshold of 0.5, phones covered, any label: 0.01
one threshold of 0.5, cards covered, any label: 0.71
one threshold of 0.5, texts with every labelled span masked: 284
kept recognizers, own thresholds, detections: 1,510
kept recognizers, own thresholds, precision, exact: 0.97
kept recognizers, own thresholds, phone recall, exact: 0.78
kept recognizers, own thresholds, phones covered, any label: 0.78
kept recognizers, own thresholds, cards covered, any label: 0.72
kept recognizers, own thresholds, texts with every labelled span masked: 394
kept recognizers, learners' phone regions, detections: 1,539
kept recognizers, learners' phone regions, precision, exact: 0.97
kept recognizers, learners' phone regions, phone recall, exact: 0.84
kept recognizers, learners' phone regions, phones covered, any label: 0.84
kept recognizers, learners' phone regions, cards covered, any label: 0.72
kept recognizers, learners' phone regions, texts with every labelled span masked: 405
指標 預設值 單一門檻 0.5 保留的辨識器 再加上學習者的地區
偵測數 3,902 1,903 1,510 1,539
精確率(完全吻合) 0.38 0.59 0.97 0.97
電話召回率(完全吻合) 0.78 0.00 0.78 0.84
涵蓋的電話 0.91 0.01 0.78 0.84
涵蓋的信用卡 0.89 0.71 0.72 0.72
完全遮蔽的文字 425 284 394 405

顯而易見的做法行不通。單一門檻 0.5 會移除弱匹配,但也移除了所有電話號碼,因為在沒有任何上下文字詞可提高分數時,電話號碼的分數是 0.40。只保留 6 個辨識器、各自使用自己的門檻,並把電話門檻設在電話號碼實際得到的分數,可將精確率提高到 0.97,且電話召回率維持不變。以學習者所在地區而非 Presidio 的預設地區來讀取電話號碼,可將電話召回率提高到 0.84。

有一項數字朝錯誤的方向變動:完全遮蔽的文字比使用預設值時更少。被移除的辨識器原本會以錯誤的標籤遮蔽部分電話與信用卡號碼,這樣仍然能隱藏數字。少了它們之後,電話號碼的涵蓋率由 0.91 降到 0.78,信用卡號碼則由 0.89 降到 0.72。我們認為精確率值得這個代價,因為偵測結果是審查人員閱讀的證據,而使用預設值時,工單與評論中會出現 689 筆銀行帳號與 683 筆駕照號碼,沒有一筆是真的。governance/pii.toml 並未將這兩種實體對應到任何分類,所以它們不會改變任何提案,但每一筆都是需要有人閱讀並排除的發現。移除美國的辨識器,同時也是對平台文字的一項主張:真實標註中沒有美國銀行帳號或駕照號碼。若平台日後需要處理這類號碼,這項主張與測量結果都必須改變。

實用原則: 絕不使用 Presidio 預設的門檻 0,也不要讓所有辨識器共用同一個門檻。只保留文字可能包含的識別碼所對應的辨識器,依各辨識器自身的分數設定其門檻,並以使用者所居住的地區(由你的資料推導)來讀取電話號碼。把這一切寫進程式碼,且每當有變動就重新測量。

此部分尚未翻譯,因此以英文呈現。

See where patterns fail

最近一次變動於第 2026.11.28 版

The tuned recognizers score well on totals, and totals hide where they fail. A pattern detector fails in three ways: there's no pattern for the thing at all, its pattern or its validation rejects the value, or it matches something that only looks personal. The platform's tickets plant cases of each kind, as its data card says: names typed in lowercase, initials, first names that are ordinary words, phone numbers typed without separators, and invoice numbers, order numbers and dates of a problem, which the truth leaves unlabelled. Count each case with the platform's detector. A case counts as caught when some detection overlaps it, whatever its label:

pattern-hard-cases.py Python 3.12.15 · duckdb 2.0.0 · phonenumbers 9.0.43 · presidio-analyzer 2.2.366 · pyarrow 26.0.1 · pyiceberg 0.12.0 · pyyaml 6.0.3 · tldextract 5.4.0 通過
"""Counts each hard case the tickets and reviews plant, and each look-alike number or date the platform's detector flags,then checks the card and phone numbers it misses. The prose rests on the caught counts for names and invoice numbers,and on the misses' lengths, first digits and validity."""from collections import Counterfrom lib import piifrom lib.platform import loadlabelled = pii.truth()found = pii.find(pii.platform_detector())cases = pii.hard_cases(labelled)               # a case is caught when some detection overlaps it, whatever its labelfor case, spans in cases.items():    print(f"{case}, labelled: {len(spans)}")    print(f"{case}, caught: {pii.caught(spans, found)}")print("hard cases of a name, caught:", pii.caught([s for spans in cases.values() for s in spans if s.entity == "PERSON"], found))# Look-alikes are numbers and dates that aren't personal data, so any detection on one is a false alarm.for kind, spans in pii.lookalikes().items():    print(f"{kind}, in the tickets: {len(spans)}")    print(f"{kind}, flagged as personal data: {pii.caught(spans, found)}")# Card numbers the card pattern misses: too short or long for its 13 to 17 digits, or a first digit it doesn't accept.# Out of range, the length is the reason; in range, the first digit is, since the pattern accepts only some of them.missed = pii.missed([s for s in labelled if s.entity == "CREDIT_CARD"], found, {"CREDIT_CARD"})why = lambda digits: f"{len(digits)} digits" if not 13 <= len(digits) <= 17 else f"{len(digits)} digits starting with {digits[0]}"print("CREDIT_CARD, missed:", len(missed))for reason, n in Counter(why(pii.text_of(s)) for s in missed).most_common():    print(f"CREDIT_CARD, missed, {reason}: {n}")print("IBAN_CODE, missed:", len(pii.missed([s for s in labelled if s.entity == "IBAN_CODE"], found, {"IBAN_CODE"})))# Are the phone numbers the patterns miss valid where the ticket's learner lives? Phones appear only in tickets,# so each phone's learner comes from its ticket.learner_of = {row["ticket_id"]: row["learner_id"] for row in load("support.tickets").to_pylist()}country_of = {row["learner_id"]: row["country"] for row in load("crm.learners").to_pylist()}phones = pii.missed([s for s in labelled if s.entity == "PHONE_NUMBER"], found, {"PHONE_NUMBER"})print("phone numbers missed:", len(phones))print("of them, invalid in the learner's country:",      sum(not pii.valid_phone(pii.text_of(s), country_of[learner_of[s.row_id]]) for s in phones))
輸出python pattern-hard-cases.py
names in lowercase, labelled: 211
names in lowercase, caught: 0
initial and surname, labelled: 163
initial and surname, caught: 0
first names that are words, labelled: 187
first names that are words, caught: 0
instructors named in reviews, labelled: 798
instructors named in reviews, caught: 0
phones typed as digits only, labelled: 257
phones typed as digits only, caught: 220
hard cases of a name, caught: 0
invoice numbers, in the tickets: 224
invoice numbers, flagged as personal data: 31
order numbers, in the tickets: 212
order numbers, flagged as personal data: 0
dates of a problem, in the tickets: 225
dates of a problem, flagged as personal data: 0
CREDIT_CARD, missed: 62
CREDIT_CARD, missed, 19 digits: 23
CREDIT_CARD, missed, 12 digits: 22
CREDIT_CARD, missed, 16 digits starting with 2: 10
CREDIT_CARD, missed, 15 digits starting with 2: 7
IBAN_CODE, missed: 0
phone numbers missed: 65
of them, invalid in the learner's country: 63

Names are the gap, since no pattern matches a name. Across the names in lowercase, the initials with a surname, the first names that are ordinary words and the instructors named in reviews, the patterns catch 0 of them. Phones typed as digits only do well, 220 of 257. The price is the look-alikes: 31 of the 224 invoice numbers are flagged as personal data.

The platform's detector flags 0 of the 212 order numbers and 0 of the 225 dates of a problem. With Presidio's defaults, 12/03 was a date at 0.10, and the date recognizer returned 421 dates where the truth labels 196. The threshold in governance/pii.toml keeps such weak matches out.

The card pattern's shape causes the card misses. It accepts 13 to 17 digits starting with 1, 3, 4, 50 to 55 or 6, but no 13-digit number starting with 1 (source), and every card number in the data passes the Luhn checksum, so validation rejects none. Of the 62 card numbers it misses, 23 have 19 digits and 22 have 12, and the rest start with 2, which the pattern doesn't accept.

Validation causes the other misses. The IBAN recognizer checks each country's format, length and check digits (source). Every labelled IBAN passes, so it misses 0, but a learner who mistypes one digit writes a value it rejects. The phone recognizer accepts only numbers that python-phonenumbers calls valid (source), and the generator doesn't always write those. Of the 65 phone numbers the detector misses, 63 aren't valid in the country the ticket's learner lives in, so most of the phone misses are the generator's, not the detector's.

Working rule: Read a detector's misses by case, not by total, and keep a list of the cases your text holds. Use patterns for what has a checkable shape, such as emails, IBANs, card numbers, phone numbers and dates, and never for names or addresses. Keep the checksums and numbering plans, because they're what makes the patterns precise. Check the card pattern against the card numbers your payment provider issues, and the date patterns against the ways your users write dates.

此部分尚未翻譯,因此以英文呈現。

Compare a PII model

最近一次變動於第 2026.11.27 版

The patterns have nothing for names or addresses, and those are most of the personal data in the platform's text: 2,455 names and 210 addresses among the 4,292 labelled spans. Finding them takes a model that reads context, not a general named-entity model. The authors of the SPY dataset define PII as the entities that can identify, contact or locate a specific individual, and found that dedicated named-entity models have limits in PII contexts (Savkin et al., 2025). We use a PII model, one trained for personally identifiable information: OpenAI's Privacy Filter, an Apache-2.0 model with eight fixed labels, such as private_person, private_address and account_number (model card).

The labels are the model's own. OpenAI's card says its label policies can't be set while the model runs (OpenAI, 2026, section 4.2). We chose it over GLiNER2-PII, the runner-up among open PII models, for those fixed labels. GLiNER2-PII takes the labels to look for at inference time, along with a threshold (model card), so a recording of it would have to pin both. Privacy Filter has 1.5 billion parameters, 50 million of them active, and runs on a CPU.

The book's examples never run a model, so they read a recording of its output. The recording, data/recorded/pii-entities/, which this chapter and chapter 9 use, holds every span the model found in every ticket and review, with its offsets, its text and the model's label. recordings/pii_entities.py made it on a CPU with the model's reference decoder, opf (code), which chooses the best sequence of labels for a whole text, so most spans come back in one piece rather than token by token. It used the default operating point, with all six transition biases at 0 (calibration file); the model can also be set "for improved recall" (section 2.3.2). The manifest pins the model's revision and the SHA-256 of its weights, so anyone can record it again and compare, and lib.pii.recorded refuses spans that don't match the text at their offsets. The recording keeps the model's labels, and the example maps them to the truth's types, so a different mapping never needs a new recording. Score the model, the platform's patterns and both together, masking whatever either finds:

privacy-filter-spans.py Python 3.12.15 · duckdb 2.0.0 · phonenumbers 9.0.43 · presidio-analyzer 2.2.366 · pyarrow 26.0.1 · pyiceberg 0.12.0 · pyyaml 6.0.3 · tldextract 5.4.0 通過
"""Scores the recorded spans of OpenAI's Privacy Filter against the truth, alone and beside the platform's patterns. Theprose rests on the recall lines for names, addresses, phones and cards, the precision lines, and the texts left withnothing personal."""from lib import pii# Privacy Filter's eight labels, mapped to the truth's entity types. A label the truth has no type for matches nothing.MATCHES = {"private_person": {"PERSON"}, "private_email": {"EMAIL_ADDRESS"}, "private_phone": {"PHONE_NUMBER"},           "private_address": {"LOCATION"}, "account_number": {"CREDIT_CARD", "IBAN_CODE"},           "private_date": {"DATE_TIME"}, "private_url": set(), "secret": set()}model, manifest = pii.recorded("pii-entities")      # the model's spans, from a recording: no model runs hereprint("model:", manifest["model"], "decoded with", manifest["settings"]["decode_mode"])labelled = pii.truth()detectors = {"patterns": pii.find(pii.platform_detector()), "model": model}# both lists joined: a span either one finds is masked, and a span both find is simply counted twicedetectors["both"] = detectors["patterns"] + modelfor name, found in detectors.items():    scores = pii.score(found, labelled, MATCHES)           # Presidio's labels aren't in MATCHES, so match themselves    print(f"{name}, detections: {len(found):,}")    for entity, (n, exact, overlap) in scores.recall.items():        print(f"{name}, {entity}, recall, exact: {exact / n:.2f}")        print(f"{name}, {entity}, recall, overlap: {overlap / n:.2f}")    print(f"{name}, precision, exact: {pii.precision_overall(scores):.2f}")    print(f"{name}, texts with every labelled span masked: {scores.masked:,} of {scores.texts:,}")# Precision by the model's own label: a label that's right for one truth type can still be wrong for anotherfor label, (n, exact, overlap) in pii.score(model, labelled, MATCHES).precision.items():    print(f"model, {label}, detections: {n:,}")    print(f"model, {label}, precision, exact: {exact / n:.2f}")    print(f"model, {label}, precision, overlap: {overlap / n:.2f}")
輸出python privacy-filter-spans.py
model: openai/privacy-filter decoded with viterbi
patterns, detections: 1,539
patterns, CREDIT_CARD, recall, exact: 0.71
patterns, CREDIT_CARD, recall, overlap: 0.71
patterns, DATE_TIME, recall, exact: 1.00
patterns, DATE_TIME, recall, overlap: 1.00
patterns, EMAIL_ADDRESS, recall, exact: 1.00
patterns, EMAIL_ADDRESS, recall, overlap: 1.00
patterns, IBAN_CODE, recall, exact: 1.00
patterns, IBAN_CODE, recall, overlap: 1.00
patterns, LOCATION, recall, exact: 0.00
patterns, LOCATION, recall, overlap: 0.00
patterns, PERSON, recall, exact: 0.00
patterns, PERSON, recall, overlap: 0.00
patterns, PHONE_NUMBER, recall, exact: 0.84
patterns, PHONE_NUMBER, recall, overlap: 0.85
patterns, precision, exact: 0.97
patterns, texts with every labelled span masked: 405 of 2,586
model, detections: 4,424
model, CREDIT_CARD, recall, exact: 0.69
model, CREDIT_CARD, recall, overlap: 0.69
model, DATE_TIME, recall, exact: 0.97
model, DATE_TIME, recall, overlap: 0.97
model, EMAIL_ADDRESS, recall, exact: 1.00
model, EMAIL_ADDRESS, recall, overlap: 1.00
model, IBAN_CODE, recall, exact: 0.99
model, IBAN_CODE, recall, overlap: 1.00
model, LOCATION, recall, exact: 0.67
model, LOCATION, recall, overlap: 0.79
model, PERSON, recall, exact: 0.84
model, PERSON, recall, overlap: 0.84
model, PHONE_NUMBER, recall, exact: 0.94
model, PHONE_NUMBER, recall, overlap: 0.94
model, precision, exact: 0.84
model, texts with every labelled span masked: 2,095 of 2,586
both, detections: 5,963
both, CREDIT_CARD, recall, exact: 0.92
both, CREDIT_CARD, recall, overlap: 0.92
both, DATE_TIME, recall, exact: 1.00
both, DATE_TIME, recall, overlap: 1.00
both, EMAIL_ADDRESS, recall, exact: 1.00
both, EMAIL_ADDRESS, recall, overlap: 1.00
both, IBAN_CODE, recall, exact: 1.00
both, IBAN_CODE, recall, overlap: 1.00
both, LOCATION, recall, exact: 0.67
both, LOCATION, recall, overlap: 0.79
both, PERSON, recall, exact: 0.84
both, PERSON, recall, overlap: 0.84
both, PHONE_NUMBER, recall, exact: 0.99
both, PHONE_NUMBER, recall, overlap: 0.99
both, precision, exact: 0.88
both, texts with every labelled span masked: 2,120 of 2,586
model, account_number, detections: 746
model, account_number, precision, exact: 0.51
model, account_number, precision, overlap: 0.51
model, private_address, detections: 177
model, private_address, precision, exact: 0.79
model, private_address, precision, overlap: 1.00
model, private_date, detections: 384
model, private_date, precision, exact: 0.50
model, private_date, precision, overlap: 0.50
model, private_email, detections: 543
model, private_email, precision, exact: 1.00
model, private_email, precision, overlap: 1.00
model, private_person, detections: 2,110
model, private_person, precision, exact: 0.98
model, private_person, precision, overlap: 0.98
model, private_phone, detections: 464
model, private_phone, precision, exact: 0.89
model, private_phone, precision, overlap: 0.89
Measure Patterns Model Both
Recall, exact, PERSON 0.00 0.84 0.84
Recall, exact, LOCATION 0.00 0.67 0.67
Recall, exact, PHONE_NUMBER 0.84 0.94 0.99
Recall, exact, CREDIT_CARD 0.71 0.69 0.92
Recall, exact, IBAN_CODE 1.00 0.99 1.00
Precision, exact 0.97 0.84 0.88
Texts fully masked 405 of 2,586 2,095 of 2,586 2,120 of 2,586

The model finds most of the names and addresses the patterns can't, and the two detectors complement each other on card numbers. The model and the patterns find about the same share of cards, 0.69 and 0.71, but not the same cards, so together they find 0.92. A later section shows why the model's share is low.

The model pays in precision, 0.84 overall, and only 0.51 for account_number and 0.50 for private_date. That partly measures a difference in label policy. The truth leaves invoice numbers and dates of a problem unlabelled, and OpenAI's card says appropriate label policies can differ (section 4.2).

Together, the two detectors leave 2,120 of 2,586 texts with nothing personal in them, against 405 of 2,586 for the patterns alone.

解讀模型的漏判

最近一次變動於第 2026.11.27 版

OpenAI 的模型卡說明了這個模型在哪些地方會失敗:它會漏掉罕見姓名、來自其他命名傳統的姓名、縮寫、包在頭銜裡的姓名,以及特定領域的識別碼;當局部脈絡不明確時,它也會過度遮蔽公開實體、地點與普通名詞(OpenAI, 2026,第 4.3 節)。請對照工單中刻意埋入的困難案例,檢查這份清單:

privacy-filter-hard-cases.py Python 3.12.15 · duckdb 2.0.0 · phonenumbers 9.0.43 · presidio-analyzer 2.2.366 · pyarrow 26.0.1 · pyiceberg 0.12.0 · pyyaml 6.0.3 · tldextract 5.4.0 通過
"""Counts how many of each hard case the model's recorded spans catch, and reads the names it misses by case and length.The prose rests on the missed-name counts and on the first lowercase ticket that both detectors miss."""from collections import Counterfrom lib import piimodel, _ = pii.recorded("pii-entities")labelled = pii.truth()cases = pii.hard_cases(labelled)for case, spans in cases.items():    print(f"{case}, labelled: {len(spans)}")    print(f"{case}, caught by the model: {pii.caught(spans, model)}")for kind, spans in pii.lookalikes().items():    print(f"{kind}, flagged by the model: {pii.caught(spans, model)}")# The names the model misses, by case: read them before trusting a recall figure.missed = pii.missed([s for s in labelled if s.entity == "PERSON"], model, {"private_person"})print("names the model misses:", len(missed))for case in ("first names that are words", "instructors named in reviews"):    often = Counter(pii.text_of(s) for s in pii.missed(cases[case], model, {"private_person"})).most_common(3)    print(f"{case}, missed most often:", ", ".join(name for name, _ in often))# A first name on its own says less than a full name: how many of the misses are one word?print("missed names that are one word:", sum(len(pii.text_of(s).split()) == 1 for s in missed))print("missed names of two words or more:", sum(len(pii.text_of(s).split()) > 1 for s in missed))# islower() counts only names typed entirely in lowercase, so a name with a capital letter, such as "Jane", doesn't.print("of them, typed in lowercase:", sum(len(pii.text_of(s).split()) > 1 and pii.text_of(s).islower() for s in missed))# A lowercase name that neither the model nor the platform's patterns cover any part of, in the first ticket with one.# The model's spans and the patterns' together: a name is missed only when both miss it. The lowest ticket ID is taken,# so the same ticket is printed on every run.first = min(pii.missed(cases["names in lowercase"], model + pii.find(pii.platform_detector())), key=lambda s: s.row_id)print("first ticket with a lowercase name both detectors miss:", first.row_id)print("the name, as typed:", pii.text_of(first))
輸出python privacy-filter-hard-cases.py
names in lowercase, labelled: 211
names in lowercase, caught by the model: 167
initial and surname, labelled: 163
initial and surname, caught by the model: 162
first names that are words, labelled: 187
first names that are words, caught by the model: 128
instructors named in reviews, labelled: 798
instructors named in reviews, caught by the model: 535
phones typed as digits only, labelled: 257
phones typed as digits only, caught by the model: 256
invoice numbers, flagged by the model: 224
order numbers, flagged by the model: 125
dates of a problem, flagged by the model: 188
names the model misses: 387
first names that are words, missed most often: Summer, Will, Hope
instructors named in reviews, missed most often: Marie, Lori, Augustin
missed names that are one word: 337
missed names of two words or more: 50
of them, typed in lowercase: 43
first ticket with a lowercase name both detectors miss: 19
the name, as typed: juan fonseca

漏判的是同時也是一般詞彙的名字,以及評論中單獨出現的講師名字。在同時也是一般詞彙的名字中,模型抓到 128(共 187 個);在講師姓名中,模型抓到 535(共 798 個)。

縮寫不是問題:模型在「縮寫加姓氏」的案例中抓到 162(共 163 個),小寫的姓名大多也能抓到,為 167(共 211 個)。

模型最常漏掉的是那些一般詞彙,漏掉 Summer, Will, Hope,以及講師的姓名 Marie, Lori, Augustin。模型標記的相似字串也遠多於調校過的樣式:發票號碼 224 個、訂單編號 125 個、問題發生日期 188 個,而樣式分別只有 31、0 與 0。

漏判的影響有多大,取決於漏掉的姓名長度。在模型漏掉的 387 個姓名中,337 個只有一個詞,也就是單獨出現的名字,50 個較長,其中 43 個是以小寫輸入。文本匿名化基準(Text Anonymization Benchmark)的作者也做了同樣的區分:漏掉直接識別碼(例如全名),比漏掉準識別碼的危害大得多(Pilán et al., 2022)。單獨的名字仍然需要遮蔽,因為評論帶有學習者的 ID,而講師在自己課程的評論中被提到的名字,也會指向特定的一個人。

模型卡建議在正式使用前,以自己的領域資料對照在地政策參考來測試模型(第 4.5 節)。它也把教育列為需要格外謹慎的情境之一(第 4.4 節),並將此模型稱為「遮蔽與資料最小化的輔助工具,而不是匿名化、合規或安全的保證」(第 4.1 節)。

實用原則: 處理自由文字時,應同時使用 PII 模型與調校過的樣式進行遮蔽,無論哪一方找到都要遮蔽:姓名與地址交給模型,信用卡號與 IBAN 則靠樣式的檢查碼。記錄模型輸出時,要固定其修訂版本、權重與操作點,保留它自己的標籤,並在程式碼中進行對應。使用前以及每次更換模型之後,都要在自己的文字上依困難案例逐項量測;若漏判太多,就改試偏向召回率的操作點,再重新記錄。遮蔽後的文字只能視為已降低識別風險,絕不能視為已匿名。

此部分尚未翻譯,因此以英文呈現。

Count what a wrong label costs

最近一次變動於第 2026.11.27 版

The comparison with the patterns scored the model by exact and overlap matching, which both need a label of the span's type. Masking doesn't read the label: a card number the model labels private_phone is still hidden. Classification does, because governance/pii.toml maps each entity type to a classification: a phone number to personal, a card number to sensitive. Measure the model's coverage beside its recall, and the labels it gives the platform's card numbers, phone numbers and addresses:

privacy-filter-labels.py Python 3.12.15 · duckdb 2.0.0 · phonenumbers 9.0.43 · presidio-analyzer 2.2.366 · pyarrow 26.0.1 · pyiceberg 0.12.0 · pyyaml 6.0.3 · tldextract 5.4.0 通過
"""Measures the model's coverage of card numbers, phone numbers and addresses whatever their label, and counts the labelsthat would misclassify them. The prose rests on the coverage lines, the mislabelled counts and the uncovered addresses."""from lib import pii# What the model's labels cost. A span is covered when every character of it is inside some detection, whatever its# label: that's what masking needs. Exact and overlap matching also need a label of the span's type.model, _ = pii.recorded("pii-entities")labelled = pii.truth()covered = pii.coverage(model, labelled)for entity in ("CREDIT_CARD", "PHONE_NUMBER", "LOCATION"):    n, fully = covered[entity]    print(f"{entity}, covered by the model, any label: {fully / n:.2f}")spans = {entity: [s for s in labelled if s.entity == entity] for entity in ("CREDIT_CARD", "PHONE_NUMBER", "LOCATION")}print("card numbers labelled private_phone:", pii.caught(spans["CREDIT_CARD"], model, {"private_phone"}))print("phone numbers labelled account_number:", pii.caught(spans["PHONE_NUMBER"], model, {"account_number"}))# For each address, the labels of the detections that overlap it, and the characters no detection covers.on = pii.labels_on(spans["LOCATION"], model)print("addresses with no detection at all:", sum(not labels for labels in on))print("addresses read, at least in part, as a person:", sum("private_person" in labels for labels in on))# A person label and no address label: the address was read as a name, which overlap matching can't credit.print("of them, with no address label:", sum({"private_person"} <= labels and "private_address" not in labels for labels in on))left = [pii.uncovered(address, model) for address in spans["LOCATION"]]print("addresses not fully covered:", sum(bool(chars) for chars in left))print("of them, with only spaces and punctuation left:", sum(bool(chars) and not any(c.isalnum() for c in chars) for chars in left))
輸出python privacy-filter-labels.py
CREDIT_CARD, covered by the model, any label: 0.87
PHONE_NUMBER, covered by the model, any label: 1.00
LOCATION, covered by the model, any label: 0.67
card numbers labelled private_phone: 40
phone numbers labelled account_number: 27
addresses with no detection at all: 24
addresses read, at least in part, as a person: 28
of them, with no address label: 20
addresses not fully covered: 70
of them, with only spaces and punctuation left: 21

A wrong label costs classification more than it costs masking. The model covers 0.87 of card numbers, but labels 40 of the 213 as phone numbers, so its recall for cards is only 0.69. If the platform classified columns from the model's labels, a column of those card numbers would be proposed personal, not sensitive. That's what the patterns add: their Luhn check keeps card numbers labelled as cards.

The mistake also runs the other way, toward the stricter side. The model labels 27 phone numbers account_number, which maps to card numbers and IBANs, so its phone coverage is 1.00 and its recall 0.94.

Addresses show what each measure counts. The model's exact recall for them is 0.67 and its overlap recall 0.79, and its coverage is 0.67. The model finds nothing at all in 24 addresses.

The model reads part of 28 addresses as a person, since Faker builds many street names from surnames, as the platform's data card notes. It gives 20 of those no address label at all, so overlap matching misses them too. Other addresses come back in pieces with the right label, such as a street and a town split at a comma. Overlap matching credits those, and coverage doesn't. Of the 70 addresses not fully covered, 21 have every letter and digit masked and only a comma or a space left. Coverage is the measure masking needs, and it's strict.

Working rule: Report coverage, whatever the label, beside recall with the right label: masking needs the first, classification the second. Read which labels a detector gives each type of span, and keep a recognizer with a checksum for every identifier whose label decides a classification, such as card numbers.

此部分尚未翻譯,因此以英文呈現。

Classify again when the schema changes

最近一次變動於第 2026.11.27 版

A classification describes a column at one moment, and chapter 2 showed how schemas move under it. So the question is when to classify again, not on every schema ID: in PyIceberg, editing a column's description gives the schema a new ID with no new column. What needs classifying is a field ID, the number Iceberg gives each column, that the metadata has never seen. Iceberg gives every added column a new one, and a renamed column keeps its ID and so its reviewed classification. Values can also change without a schema change, so the samples of text columns need a rerun on a schedule too. Vendors promise this as a service: Databricks says its classification scans incrementally and classifies new data with no manual setup (Databricks docs), a claim we don't test.

The support team wants a number to call learners back on for tickets opened by phone, so it adds a column called callback_number to support.tickets. The change comes in four steps: edit a description, add the column, fill it from the learners' phone numbers, then classify only the field IDs that governance/metadata.toml lacks:

reclassify-new-columns.py Python 3.12.15 · duckdb 2.0.0 · phonenumbers 9.0.43 · presidio-analyzer 2.2.366 · pyarrow 26.0.1 · pyiceberg 0.12.0 · pyyaml 6.0.3 · tldextract 5.4.0 通過
"""Edits the tickets table twice, first a column's description and then a new column, and after each change counts the fieldIDs the metadata lacks. The prose rests on the schema IDs, the field counts and the proposal for the new column."""from pyiceberg.types import StringTypefrom lib import metadata, piifrom lib.catalog import fill_column, open_catalogfrom lib.platform import loadfrom lib.warehouse import connectcatalog = open_catalog(["support.tickets"])tickets = catalog.load_table("support.tickets")reviewed = metadata.load()["support.tickets"].columns           # field ID -> the column as people reviewed it# A field ID the reviewed metadata doesn't have: Iceberg gives each added column a new one.unclassified = lambda: {f.field_id for f in tickets.schema().fields if f.field_id not in reviewed}print("schema ID at the start:", tickets.schema().schema_id)with tickets.update_schema() as update:                         # a doc edit: a new schema, no new column    update.update_column("body", doc="What the learner wrote, as they wrote it.")print("schema ID after a doc edit:", tickets.schema().schema_id)print("fields to classify after the doc edit:", len(unclassified()))# The support team starts storing the number to call back for tickets opened by phone: the learner's phone number.with tickets.update_schema() as update:    update.add_column("callback_number", StringType(), doc="Where to call the learner back.")phone_of = {row["learner_id"]: row["phone"] for row in load("crm.learners").to_pylist()}# tickets from other channels have no callback number, so their value is Nonefill_column(tickets, "callback_number", lambda row: phone_of[row["learner_id"]] if row["channel"] == "phone" else None)print("schema ID after adding a column:", tickets.schema().schema_id)print("fields to classify after adding a column:", len(unclassified()))# only= keeps the proposals to the field IDs the metadata lacks.for f in pii.classify(catalog, connect(catalog), pii.platform_detector(), only={"support.tickets": unclassified()}):    found = ", ".join(f"{entity} in {n} of {f.sampled}" for entity, n in f.found.items())    print(f"{f.table}.{f.column}, field ID: {f.field_id}")    print(f"{f.table}.{f.column}, entity its name suggests: {f.named}")    print(f"{f.table}.{f.column}, found in its sampled values: {found}")    print(f"{f.table}.{f.column}, proposed classification: {f.classification}")
輸出python reclassify-new-columns.py
schema ID at the start: 0
schema ID after a doc edit: 1
fields to classify after the doc edit: 0
schema ID after adding a column: 2
fields to classify after adding a column: 1
support.tickets.callback_number, field ID: 9
support.tickets.callback_number, entity its name suggests: None
support.tickets.callback_number, found in its sampled values: PHONE_NUMBER in 154 of 173
support.tickets.callback_number, proposed classification: personal

A doc edit moves the schema to a new ID and leaves 0 fields to classify, since it adds no column. Adding the column moves the schema to 2, with 1 field to classify: field 9.

No name rule matches callback_number, since it contains none of the words the phone rule looks for, which is how a name rule misses a column nobody wrote it for. Its values settle it. The sample finds PHONE_NUMBER in 154 of 173, and the proposal is personal.

The pull request that adds the column is where its classification belongs, beside the metadata entry it needs: chapter 10 fails that pull request until the column has one. A proposal from values needs a column that holds values, so this one comes from the scheduled sample.

Working rule: Classify every field ID the metadata doesn't have, in the pull request that adds it, and don't let it merge without a reviewed classification. Key the trigger on field IDs, never on schema IDs. Sample every text column again on a schedule too, because values change without a schema change.

此部分尚未翻譯,因此以英文呈現。

What goes wrong in production

最近一次變動於第 2026.11.27 版

A detector's worst failure is a miss, and its output always looks like a list of findings: what it misses never appears in it:

  • Phone numbers read as bank numbers. With Presidio's defaults, every run of 8 to 17 digits is a US bank number, and so is every phone number typed without separators. The evidence for the ticket table lists 689 bank numbers that aren't there, a report of "bank numbers in tickets" sends someone to the wrong team, and mapping that entity to a classification would raise columns for the wrong reason. Keep only the recognizers your text needs, each with its own threshold, as the tuning section does, and count detections per label: a label with hundreds of hits and no labelled spans is a false positive.
  • Names in free text missed. Patterns find no names, and the model misses some lowercase names, first names that are also words, and first names standing alone. Ticket 19 still says "this is juan fonseca, i still can't log in" after both detectors have run, and passes every check that counts detections. You notice only by measuring recall against labelled text, per hard case, and by reading a sample of masked output.
  • Personal data in a column called notes. No name rule marks it, and a reviewer who never read it classifies it internal. Sample every text column's values, and treat free-text columns as personal until a sample says otherwise.
  • A column more sensitive than its metadata. Learners paste card numbers into tickets, so support.tickets.body holds sensitive data under a personal classification. Compare every proposal with the reviewed metadata, and route a stricter one to the table's owner.
  • A detector that reaches the network. AnalyzerEngine() downloads a model, and tldextract fetches the Public Suffix List. Offline, the pipeline fails; online, the same text can be classified differently on two machines. A setting read at import holds only if it's made before anything imports the library. Configure the engine, pin what it would fetch, check that the setting held, and run the detector's tests with the network off.
  • Truth shaped by its generator. The platform's text comes from 30 sentence templates filled by Faker, which writes every date of birth one way, phone numbers that aren't valid where the learner lives, and street names built from people's names. Read the misses before trusting a recall figure, and measure again on labelled samples of your own text.

此部分尚未翻譯,因此以英文呈現。

What this chapter decided

最近一次變動於第 2026.11.27 版
  • Columns are classified from their name, their type and their values. Name rules run on every column; text columns are sampled; the strictest evidence wins, with internal as the floor, and only a person sets public.
  • Samples are reproducible: 200 non-null values per text column, by a seeded reservoir sample on one DuckDB thread.
  • Presidio runs offline, on NoOpNlpEngine, with 6 recognizers, each with its own threshold, phone regions read from the learners' countries, and tldextract's own copy of the Public Suffix List, checked when lib.pii loads.
  • Detectors are measured against labelled spans: recall per entity type by exact and overlap matching with a label of the right type, coverage whatever the label, precision per label and texts left with nothing personal, with recall weighed first.
  • A proposal stricter than the metadata is a finding: support.tickets.body goes to team-support as sensitive, and billing.payments.card_number to team-billing, with the evidence that it holds full card numbers.
  • Free text is masked with two detectors: OpenAI's Privacy Filter, recorded at a pinned revision and operating point, for names and addresses, and the tuned patterns for card numbers, IBANs and phone numbers, whose checksums keep card numbers labelled as cards. Masking hides whatever either finds; neither is trusted to make text anonymous.
  • No example prints a whole phone number, card number or IBAN: lib.pii.masked keeps the last four digits.
  • Reclassification follows field IDs: every field the metadata lacks is classified in the pull request that adds it, and text columns are sampled again on a schedule. A detector is only as good as the measurement behind it, and what it misses never appears in its output. The measurement is what tells a team which of its classifications it can rely on, and which it has to check by hand.

本章,於本版次中

版次 2026.11.31範例於 2026年11 月 19 通過測試本章 11 個範例 個範例中有 11 個通過

譯自英文書,版次 2026.11.31。

引用本節

格式

各種格式皆引用英文原書,本譯本即依循該原書。

每種格式都會標示本版。其連結會開啟目前版次,節標題會說明該節內文最後一次變動的時間。存取日期與擷取日期即為你開啟此頁的當日。

本頁內容

第 3 章,共 10 章: 找出個人資料

  1. 從結構描述分類欄位
  2. Run Presidio offline
  3. 每次執行都以相同方式抽樣欄位的值
  4. 從欄位的值來分類
  5. Measure a detector against labelled spans
  6. 依據真實標註調校辨識器
  7. See where patterns fail
  8. Compare a PII model
  9. 解讀模型的漏判
  10. Count what a wrong label costs
  11. Classify again when the schema changes
  12. What goes wrong in production
  13. What this chapter decided

返回頂端所有章節