Glossary
The 151 terms Data Governance as Code defines, each in the book's own words, with the section that defines it.
From Edition 2026.11.31, the book's current edition. Where a definition depends on a package, it says the version this edition pins and runs its examples on.
A
- access log
-
a separate access log, with learner IDs, a retention period and no chain
Defined in Keep values out of the record, chapter 8. In the book since Edition 2026.11.31. Permanent link to access log
- access review also access reviews
-
An access review compares what each person may read with what they use.
Defined in Audit and evidence, chapter 8. In the book since Edition 2026.11.31. Permanent link to access review
- approximate threshold
-
The curve's approximate threshold is (1/b)^(1/r).
(1/b)^(1/r)Defined in Tune LSH for recall, then confirm every pair, chapter 9. In the book since Edition 2026.11.31. Permanent link to approximate threshold
- AssertCurrentSchemaId
-
A REST catalog commit can carry AssertCurrentSchemaId, which requires the table's current schema ID to match the one the requirement names
Defined in Report what a schema change reaches, chapter 10. In the book since Edition 2026.11.31. Permanent link to AssertCurrentSchemaId
- attribute also attributes
-
an attribute is any other fact a rule reads, about the principal, the data or the request
Defined in Access policies as code, chapter 5. In the book since Edition 2026.11.31. Permanent link to attribute
- attribute-based access control also ABAC
-
NIST's guide to attribute-based access control defines the general approach, in which access is decided by attributes of the subject, the object, the requested operations and, sometimes, the environment.
Defined in Narrow by attribute: region, consent and purpose, chapter 5. In the book since Edition 2026.11.31. Permanent link to attribute-based access control
- audit log also audit logs
-
an audit log is a record of who did what, when, and with what outcome, kept so that someone can check it later.
Defined in Audit and evidence, chapter 8. In the book since Edition 2026.11.31. Permanent link to audit log
B
- branch protection
-
GitHub's branch protection can require that every required status check pass before collaborators can merge changes into the protected branch
Defined in Gate the pull request, not the commit, chapter 10. In the book since Edition 2026.11.31. Permanent link to branch protection
- breaking-change checker also breaking-change checkers
-
breaking-change checkers, the tools that compare two versions of a schema for changes that break its readers
Defined in Governance in CI, chapter 10. In the book since Edition 2026.11.31. Permanent link to breaking-change checker
C
- candidate pair also candidate pairs
-
two texts become a candidate pair when all the rows of any one band match
- Part of: locality-sensitive hashing
Defined in Tune LSH for recall, then confirm every pair, chapter 9. In the book since Edition 2026.11.31. Permanent link to candidate pair
- catalog
-
A catalog lists a platform's tables and their schemas
Defined in Catalog a table with its owner and classifications, chapter 1. In the book since Edition 2026.11.31. Permanent link to catalog
- catalog label also labels
-
labels, ownership or classification that a catalog returns with a table, per table and per field ID, kept out of properties on purpose
- Contrasts with: table property
Defined in Write descriptions into the table's schema, chapter 2. In the book since Edition 2026.11.31. Permanent link to catalog label
- Cedar schema
-
A schema declares every type of entity, the kinds of principal and resource a request can name, with its attributes
- Comes before: policy validation
Defined in Write the same rules in Cedar, chapter 5. In the book since Edition 2026.11.31. Permanent link to Cedar schema
- chance of a candidate
-
Leskovec, Rajaraman and Ullman derive the chance that a pair with Jaccard similarity s becomes a candidate pair
P(candidate) = 1 − (1 − sʳ)ᵇ- Measured from: candidate pair
Defined in Tune LSH for recall, then confirm every pair, chapter 9. In the book since Edition 2026.11.31. Permanent link to chance of a candidate
- checkout also checkouts
-
two folders called checkouts, one of the main branch and one of the pull request's branch
Defined in Governance in CI, chapter 10. In the book since Edition 2026.11.31. Permanent link to checkout
- classification also classifications
-
A classification is one of four values, ordered from least to most protected
- Part of: metadata
Defined in Keep the metadata in a reviewed file, chapter 2. In the book since Edition 2026.11.31. Permanent link to classification
- code owner also code owners, CODEOWNERS
-
GitHub requests code owners' review automatically when a pull request modifies code they own
Defined in Route each domain's reviews by path, chapter 10. In the book since Edition 2026.11.31. Permanent link to code owner
- column lineage
-
Column lineage says which input columns each output column came from, and that is what governance needs
Defined in Trace each output column to its sources, chapter 4. In the book since Edition 2026.11.31. Permanent link to column lineage
- column mask also column masks
-
a column mask changes how a column's values are shown
- Part of: decision document
Defined in Access policies as code, chapter 5. In the book since Edition 2026.11.31. Permanent link to column mask
- columnLineage facet
-
OpenLineage puts column lineage in a dataset facet, columnLineage, on the output: fields maps each output column to its inputFields, and an optional dataset list holds inputs that affect the whole dataset
- A kind of: facet
- Computed from: column lineage
Defined in Trace each output column to its sources, chapter 4. In the book since Edition 2026.11.31. Permanent link to columnLineage facet
- complete mediation
-
Saltzer and Schroeder's principle of complete mediation holds that "every access to every object must be checked for authority"
Defined in Hand the same policies to an engine that enforces them, chapter 6. In the book since Edition 2026.11.31. Permanent link to complete mediation
- consent
-
Consent is a person's agreement to one use of their data, such as marketing email, which they can withdraw.
Defined in Retention, consent and erasure, chapter 7. In the book since Edition 2026.11.31. Permanent link to consent
- copy-on-write
-
Copy-on-write rewrites each affected data file without the deleted rows.
- Contrasts with: merge-on-read
Defined in Delete expired rows with row deletes, chapter 7. In the book since Edition 2026.11.31. Permanent link to copy-on-write
- coverage
-
lib.pii adds a third measure, coverage: whether every character of a labelled span is inside some detection, whatever its label.
- Measured from: truth
Defined in Measure a detector against labelled spans, chapter 3. In the book since Edition 2026.11.31. Permanent link to coverage
- coverage check
-
The coverage check is the catalog check, lib.metadata.problems, run on the branch's platform, and for each column nobody classified it adds what a rule on column names suggests
Defined in Fail the pull request that adds an unclassified column, chapter 1. In the book since Edition 2026.11.31. Permanent link to coverage check
- CRM
-
The CRM, the application where learners manage their accounts
Defined in Read consent where the query runs, chapter 7. In the book since Edition 2026.11.31. Permanent link to CRM
- Croissant
-
Croissant is MLCommons' metadata format for machine-learning datasets
True for mlcroissant 1.1.1, as this edition pins it.
- Implemented by: mlcroissant 1.1.1
Defined in Generate the dataset card from the run, chapter 9. In the book since Edition 2026.11.31. Permanent link to Croissant
- cross-check also cross-check against the query log
-
Audited.check_engine_log matches each query in DuckDB's log, by keyed digest, against the rewritten_digest of the records written since the last check, then empties DuckDB's log, which holds the queries' values.
True for duckdb 2.0.0, as this edition pins it.
- Computed from: rewritten digest
Defined in Cross-check the log against DuckDB's query log, chapter 8. In the book since Edition 2026.11.31. Permanent link to cross-check
D
- data document
-
Everything else a rule needs besides the request, such as each user's groups and each column's classification, comes from a second document, which OPA calls data.
Defined in Generate the data the policy reads, chapter 5. In the book since Edition 2026.11.31. Permanent link to data document
- dataset also datasets
-
A dataset is a table or file, named by a namespace and a name that together identify it.
Defined in Send a run event from every pipeline, chapter 4. In the book since Edition 2026.11.31. Permanent link to dataset
- dataset card also dataset cards
-
A dataset card tells whoever trains or audits a model what's in its data, where it came from and what was done to it.
Defined in Generate the dataset card from the run, chapter 9. In the book since Edition 2026.11.31. Permanent link to dataset card
- decision also decisions
-
A decision is a policy's answer.
Defined in Access policies as code, chapter 5. In the book since Edition 2026.11.31. Permanent link to decision
- decision document
-
The platform's policy uses structured data to return its whole answer as one document, data.platform.access.decision, with four keys: allow, the row_filters, the column_masks, and the reasons.
- A kind of: decision
Defined in Return the decision as one document, chapter 5. In the book since Edition 2026.11.31. Permanent link to decision document
- decision record also decision records
-
The platform's audit log holds one decision record per query.
- Part of: audit log
Defined in Audit and evidence, chapter 8. In the book since Edition 2026.11.31. Permanent link to decision record
- decision table also who-sees-what table
-
A decision table is one row per request, saying who reads which table and what they must get back, in the terms of their job.
Defined in Check who sees what from a decision table, chapter 5. In the book since Edition 2026.11.31. Permanent link to decision table
- deny by default also default deny
-
With default allow := false, a request that no allow rule matches is denied, whoever sends it and whatever it asks for.
- A kind of: fail-safe defaults
Defined in Deny by default, then allow by role, chapter 5. In the book since Edition 2026.11.31. Permanent link to deny by default
- deny list also deny lists
-
a Presidio deny list of the names the platform knows: every word of every instructor's name
True for presidio-analyzer 2.2.366, as this edition pins it.
Defined in Replace personal data in the text, then measure what's left, chapter 9. In the book since Edition 2026.11.31. Permanent link to deny list
- deploy
-
the deploy, which writes the metadata into the catalog and sets every table's generated properties
Defined in Governance in CI, chapter 10. In the book since Edition 2026.11.31. Permanent link to deploy
- deprecated
-
Any element of a contract, a schema object or a property, nested ones included, may set the optional deprecated boolean to signal that it is no longer recommended for use
Defined in Deprecate a column before removing it, chapter 10. In the book since Edition 2026.11.31. Permanent link to deprecated
- deprecation rule
-
Our second rule makes the deprecation warning a required step: a column may leave a table only once the main branch has marked it deprecated.
Defined in Deprecate a column before removing it, chapter 10. In the book since Edition 2026.11.31. Permanent link to deprecation rule
- differencing also tracker
-
Differencing, comparing consecutive reports, is the attack Denning, Denning and Schwartz called a tracker.
- Contrasts with: minimum group size
Defined in Hold back small groups in reports, chapter 6. In the book since Edition 2026.11.31. Permanent link to differencing
- DIRECT also DIRECT input
-
DIRECT means the output value is derived from the input value, through IDENTITY, TRANSFORMATION or AGGREGATION.
- Contrasts with: INDIRECT
Defined in Add the inputs that decide which rows exist, chapter 4. In the book since Edition 2026.11.31. Permanent link to DIRECT
- doc
-
each column can carry a doc, an optional comment that travels with the schema
True for pyiceberg 0.12.0, as this edition pins it.
Defined in Write descriptions into the table's schema, chapter 2. In the book since Edition 2026.11.31. Permanent link to doc
- domain
-
a domain, the part of the business the table belongs to
- Part of: metadata
Defined in Keep the metadata in a reviewed file, chapter 2. In the book since Edition 2026.11.31. Permanent link to domain
- domain rule also domain rules
-
Each domain has a package of its own, owned by its team
Defined in Layer domain rules on the global rules, chapter 10. In the book since Edition 2026.11.31. Permanent link to domain rule
E
- effective false positive
-
Google's Tricorder authors define an effective false positive as any report that a user chooses not to act on
Defined in Count waived findings per check, chapter 10. In the book since Edition 2026.11.31. Permanent link to effective false positive
- enforcement layer
-
The book's shared module lib.enforce enforces decisions; we call it the enforcement layer.
Defined in Enforcing policies on queries, chapter 6. In the book since Edition 2026.11.31. Permanent link to enforcement layer
- engine query log also query log, DuckDB's query log
-
its logging framework has a QueryLog type, which records the queries DuckDB executes, turned on with CALL enable_logging('QueryLog') and read from the duckdb_logs view
True for duckdb 2.0.0, as this edition pins it.
- Contrasts with: audit log
Defined in Cross-check the log against DuckDB's query log, chapter 8. In the book since Edition 2026.11.31. Permanent link to engine query log
- equality delete also equality deletes
-
equality deletes, which mark it by the values of one or more columns
Defined in Delete expired rows with row deletes, chapter 7. In the book since Edition 2026.11.31. Permanent link to equality delete
- erasure
-
Erasure is removing a person's data on request.
Defined in Retention, consent and erasure, chapter 7. In the book since Edition 2026.11.31. Permanent link to erasure
- erasure register
-
an erasure register: for each request, its ID and a keyed hash (HMAC-SHA256) of the erased learner's ID, and nothing else
Defined in An erased learner comes back, chapter 7. In the book since Edition 2026.11.31. Permanent link to erasure register
- erasure scan
-
lib.retention.scan reads every file under the catalog's warehouse, each with its own format's reader, and looks for the learner's traces.
Defined in Prove the erasure by reading every file, chapter 7. In the book since Edition 2026.11.31. Permanent link to erasure scan
- evidence
-
Evidence is a report that shows a control works, generated from records rather than captured from a screen.
Defined in Audit and evidence, chapter 8. In the book since Edition 2026.11.31. Permanent link to evidence
- exact matching
-
Exact matching, from the CoNLL-2003 shared task, counts an entity "correct only if it is an exact match of the corresponding entity in the data file"
- Measured from: truth
- Contrasts with: overlap matching
Defined in Measure a detector against labelled spans, chapter 3. In the book since Edition 2026.11.31. Permanent link to exact matching
F
- facet also facets
-
A facet is OpenLineage's name for a block of metadata attached to a run, a job or a dataset
Defined in Send a run event from every pipeline, chapter 4. In the book since Edition 2026.11.31. Permanent link to facet
- fail-safe defaults
-
Saltzer and Schroeder's principle of fail-safe defaults is to "Base access decisions on permission rather than exclusion"
Defined in Deny by default, then allow by role, chapter 5. In the book since Edition 2026.11.31. Permanent link to fail-safe defaults
- federated computational governance
-
federated computational governance, "a set of global rules - rules applied to all data products and their interfaces - to ensure a healthy and interoperable ecosystem"
Defined in Layer domain rules on the global rules, chapter 10. In the book since Edition 2026.11.31. Permanent link to federated computational governance
- field ID also field IDs
-
Iceberg gives every column a field ID, an integer unique within the table's schema
True for pyiceberg 0.12.0, as this edition pins it.
Defined in Key column metadata by field ID, chapter 2. In the book since Edition 2026.11.31. Permanent link to field ID
- finding already on main also findings already on main
-
A finding already on main is reported on every run, counted and named with the team that owns it, but it fails no pull request that leaves it as it is.
Defined in Run every check as one job, chapter 10. In the book since Edition 2026.11.31. Permanent link to finding already on main
- finding ID also finding IDs
-
A violation's subject and the rule's name make the finding's ID.
Defined in Layer domain rules on the global rules, chapter 10. In the book since Edition 2026.11.31. Permanent link to finding ID
- function with side effects also has_side_effects
-
has_side_effects is "true if this function changes the database state"
Defined in Allow only catalog tables, CTEs and subqueries, chapter 6. In the book since Edition 2026.11.31. Permanent link to function with side effects
G
- gate also gates
-
On a pull request, a check is a gate: it runs on every change, before the change merges, and it fails the pull request through its exit code.
Defined in Run every check as one job, chapter 10. In the book since Edition 2026.11.31. Permanent link to gate
- global rule also global rules
-
The platform team keeps one package of global rules, which the domains agree on
Defined in Layer domain rules on the global rules, chapter 10. In the book since Edition 2026.11.31. Permanent link to global rule
- greedy decoding
-
The script decodes greedily, taking the most likely next token
Defined in Draft metadata with a local model, chapter 2. In the book since Edition 2026.11.31. Permanent link to greedy decoding
H
- hash chain also chain
-
A hash chain detects a change to a log that's still growing: every record carries a sequence number, seq, and the SHA-256 of the line before it, prev_hash, so changing a record breaks the link to the next one, and removing one leaves a gap.
Defined in Chain the records so an edit shows, chapter 8. In the book since Edition 2026.11.31. Permanent link to hash chain
- head also chain head
-
a chain's head, the number of records and the hash of the last one, written down where the log's writer can't change it
- Part of: hash chain
Defined in Chain the records so an edit shows, chapter 8. In the book since Edition 2026.11.31. Permanent link to head
- HMAC
-
HMAC is the construction RFC 2104 describes for keying a hash
Defined in Keep pseudonyms consistent with a keyed hash, chapter 9. In the book since Edition 2026.11.31. Permanent link to HMAC
I
- IDENTITY edge also IDENTITY edges
-
a column copied unchanged, an IDENTITY edge in OpenLineage's terms
Defined in Plan an erasure from lineage, chapter 7. In the book since Edition 2026.11.31. Permanent link to IDENTITY edge
- impact analysis
-
Impact analysis asks what a change to one column can break.
- Computed from: lineage graph
Defined in Find what a column change breaks, chapter 4. In the book since Edition 2026.11.31. Permanent link to impact analysis
- implicit flow
-
Denning calls the information that flows through conditions "implicit flow": a statement that assigns only when a condition holds still tells you the condition's value
Defined in Add the inputs that decide which rows exist, chapter 4. In the book since Edition 2026.11.31. Permanent link to implicit flow
- INDIRECT also INDIRECT input
-
INDIRECT means the output value is impacted by an input without being derived from it, through JOIN, GROUP_BY, FILTER, SORT, WINDOW or CONDITIONAL.
- Contrasts with: DIRECT
Defined in Add the inputs that decide which rows exist, chapter 4. In the book since Edition 2026.11.31. Permanent link to INDIRECT
- input
-
The service sends a JSON document describing the request, and the policy reads it as input.
Defined in Shape the request like Trino's, chapter 5. In the book since Edition 2026.11.31. Permanent link to input
J
- job also jobs
-
A job is "a process that consumes or produces Datasets", such as a pipeline.
Defined in Send a run event from every pipeline, chapter 4. In the book since Edition 2026.11.31. Permanent link to job
K
- keyed pseudonym also keyed pseudonyms
-
lib.ai_data.pseudonym computes an HMAC with SHA-256 of the entity type and the case-folded value, under a key only the corpus job holds, and keeps eight hexadecimal digits
- Computed from: HMAC
Defined in Keep pseudonyms consistent with a keyed hash, chapter 9. In the book since Edition 2026.11.31. Permanent link to keyed pseudonym
L
- last4
-
last4 keeps the last four digits of a card number, after removing anything that isn't a digit.
- A kind of: column mask
Defined in Write each mask as SQL in one place, chapter 6. In the book since Edition 2026.11.31. Permanent link to last4
- lineage
-
Lineage is where each column's data came from, and where it goes.
Defined in Lineage from pipelines, chapter 4. In the book since Edition 2026.11.31. Permanent link to lineage
- lineage graph
-
With column lineage in every COMPLETE event, the platform has a graph: columns are nodes, and each input field is an edge to the column or table it feeds.
- Computed from: column lineage
Defined in Find what a column change breaks, chapter 4. In the book since Edition 2026.11.31. Permanent link to lineage graph
- locality-sensitive hashing also LSH
-
LSH cuts each signature into b bands of r rows
True for datasketch 2.0.0, as this edition pins it.
- Computed from: MinHash signature
- Implemented by: datasketch 2.0.0
Defined in Tune LSH for recall, then confirm every pair, chapter 9. In the book since Edition 2026.11.31. Permanent link to locality-sensitive hashing
M
- manifest also manifests
-
a manifest lists data files, with each file's column statistics
Defined in Remove the files no snapshot references, chapter 7. In the book since Edition 2026.11.31. Permanent link to manifest
- masking flag
-
OpenLineage's masking flag says only that a value was obfuscated, such as by a hash
Defined in Propose classifications from lineage, chapter 4. In the book since Edition 2026.11.31. Permanent link to masking flag
- merge-on-read
-
Merge-on-read leaves the data file alone and writes a delete file beside it.
- Contrasts with: copy-on-write
Defined in Delete expired rows with row deletes, chapter 7. In the book since Edition 2026.11.31. Permanent link to merge-on-read
- metadata
-
A catalog's metadata is what you know about each table that the data doesn't say: who owns it, which part of the business it belongs to, what it holds, and how each column must be protected.
Defined in Keep the metadata in a reviewed file, chapter 2. In the book since Edition 2026.11.31. Permanent link to metadata
- metrics mode also metrics modes
-
Iceberg lets a table choose, per column, what its manifests keep, with the property write.metadata.metrics.column.<name>: none keeps nothing, counts keeps value and null counts, truncate(n) adds bounds cut to n characters, and full adds whole bounds
True for pyiceberg 0.12.0, as this edition pins it.
Defined in Keep personal values out of manifests, chapter 7. In the book since Edition 2026.11.31. Permanent link to metrics mode
- migration
-
A schema change on the platform is a migration, an alter table statement in a file in the branch's migrations/ folder.
Defined in Fail the pull request that adds an unclassified column, chapter 1. In the book since Edition 2026.11.31. Permanent link to migration
- MinHash signature also MinHash signatures, MinHash
-
Taking the smallest shingle under each of many permutations gives a MinHash signature, which estimates the similarity of two texts without comparing their sets
True for datasketch 2.0.0, as this edition pins it.
- Computed from: shingle
- Implemented by: datasketch 2.0.0
Defined in Find near-duplicates with MinHash, chapter 9. In the book since Edition 2026.11.31. Permanent link to MinHash signature
- minimum group size
-
The usual defence is a minimum group size, below which a reported group is held back.
Defined in Hold back small groups in reports, chapter 6. In the book since Edition 2026.11.31. Permanent link to minimum group size
- model draft also model drafts
-
a model's draft is a proposal that a person accepts, corrects or rejects in the pull request
Defined in Draft metadata with a local model, chapter 2. In the book since Edition 2026.11.31. Permanent link to model draft
N
- near-duplicate also near-duplicates
-
A confirmed pair at 0.8 or more is a near-duplicate, unless it differs only in a name
Defined in Find near-duplicates with MinHash, chapter 9. In the book since Edition 2026.11.31. Permanent link to near-duplicate
- NLP engine
-
The NLP engine does two jobs, named-entity recognition and feature extraction for rules, such as the context words that raise a score.
True for presidio-analyzer 2.2.366, as this edition pins it.
Defined in Run Presidio offline, chapter 3. In the book since Edition 2026.11.31. Permanent link to NLP engine
- noise
-
measure each check's noise by the findings nobody acts on
Defined in Governance in CI, chapter 10. In the book since Edition 2026.11.31. Permanent link to noise
- normalized query also normalized text
-
lib.audit.normalized keeps the query's shape and drops its values: sqlglot parses the query, and every literal, in any of DuckDB's syntaxes, becomes ?.
True for sqlglot 30.33.0, as this edition pins it.
Defined in Keep values out of the record, chapter 8. In the book since Edition 2026.11.31. Permanent link to normalized query
- not-useful rate
-
the not-useful rate: an analyzer's Not useful clicks as a share of its Not useful, Please fix and Apply fix clicks
Defined in Count waived findings per check, chapter 10. In the book since Edition 2026.11.31. Permanent link to not-useful rate
O
- overlap matching
-
Overlap matching asks only for one shared character, with a label of its type
- Measured from: truth
Defined in Measure a detector against labelled spans, chapter 3. In the book since Edition 2026.11.31. Permanent link to overlap matching
- owner
-
an owner, the team answerable for the table, never a person
- Part of: metadata
Defined in Keep the metadata in a reviewed file, chapter 2. In the book since Edition 2026.11.31. Permanent link to owner
- owner rule
-
We add a first rule of our own: lowering a column's classification, changing a table's owner, or moving a table to another domain needs the approval of the team that owns the table on the main branch.
Defined in Ask the owner before lowering a classification, chapter 10. In the book since Edition 2026.11.31. Permanent link to owner rule
P
- partial evaluation
-
Cedar's experimental partial evaluation, which returns residual policies for the parts it can't yet decide, and whose result is not a final authorization decision
Defined in Write the same rules in Cedar, chapter 5. In the book since Edition 2026.11.31. Permanent link to partial evaluation
- personal data
-
The GDPR defines personal data as "any information relating to an identified or identifiable natural person"
Defined in Keep values out of the record, chapter 8. In the book since Edition 2026.11.31. Permanent link to personal data
- PII also personally identifiable information
-
The authors of the SPY dataset define PII as the entities that can identify, contact or locate a specific individual
Defined in Compare a PII model, chapter 3. In the book since Edition 2026.11.31. Permanent link to PII
- PII model
-
We use a PII model, one trained for personally identifiable information
- A kind of: PII
Defined in Compare a PII model, chapter 3. In the book since Edition 2026.11.31. Permanent link to PII model
- platform team
-
the platform team, the team answerable for tables without an owner
Defined in Fail the pull request that adds an unclassified column, chapter 1. In the book since Edition 2026.11.31. Permanent link to platform team
- policy also policies
-
A policy is code that answers whether a request may go ahead.
Defined in Access policies as code, chapter 5. In the book since Edition 2026.11.31. Permanent link to policy
- policy revision also revision
-
A policy revision is a fingerprint of everything a decision depended on besides the request.
Defined in Audit and evidence, chapter 8. In the book since Edition 2026.11.31. Permanent link to policy revision
- policy test also policy tests
-
a test is a rule whose name is prefixed with test_
Defined in Unit-test the policy in Rego, chapter 5. In the book since Edition 2026.11.31. Permanent link to policy test
- policy validation also validation
-
Validation checks policies against a schema before any request
Defined in Write the same rules in Cedar, chapter 5. In the book since Edition 2026.11.31. Permanent link to policy validation
- position delete also position deletes
-
position deletes, which mark a row by its data file and its position
Defined in Delete expired rows with row deletes, chapter 7. In the book since Edition 2026.11.31. Permanent link to position delete
- pre-commit hook also pre-commit hooks
-
A pre-commit hook is the other place a check can run, on the author's machine, before a commit exists.
- Contrasts with: gate
Defined in Gate the pull request, not the commit, chapter 10. In the book since Edition 2026.11.31. Permanent link to pre-commit hook
- precision
-
precision, the share of each label's detections that match a labelled span
- Measured from: truth
Defined in Measure a detector against labelled spans, chapter 3. In the book since Edition 2026.11.31. Permanent link to precision
- principal also principals
-
The principal is who asks
Defined in Access policies as code, chapter 5. In the book since Edition 2026.11.31. Permanent link to principal
- property id
-
ODCS defines a property's id as "A unique identifier for the element used to create stable, refactor-safe references"
Defined in Route each domain's reviews by path, chapter 10. In the book since Edition 2026.11.31. Permanent link to property id
- prov:wasDerivedFrom
-
the tables the pipeline's own SQL reads, which the values come from, as prov:wasDerivedFrom
Defined in Generate the dataset card from the run, chapter 9. In the book since Edition 2026.11.31. Permanent link to prov:wasDerivedFrom
- prov:wasInfluencedBy
-
the table the enforcement layer's rewrite adds, which only decides which rows exist, as prov:wasInfluencedBy
Defined in Generate the dataset card from the run, chapter 9. In the book since Edition 2026.11.31. Permanent link to prov:wasInfluencedBy
- provenance
-
Provenance at the level of the record answers three questions: which table it came from, which row, and which run of which job put it in the corpus.
Defined in Carry provenance on every record, chapter 9. In the book since Edition 2026.11.31. Permanent link to provenance
- pseudonym also pseudonyms
-
a pseudonym: a token for one name, the same wherever that name appears
Defined in Keep pseudonyms consistent with a keyed hash, chapter 9. In the book since Edition 2026.11.31. Permanent link to pseudonym
- pseudonymization
-
pseudonymization, which replaces an identifier with a consistent stand-in
- A kind of: column mask
Defined in Access policies as code, chapter 5. In the book since Edition 2026.11.31. Permanent link to pseudonymization
- pseudonymize also keyed hash
-
pseudonymize is a keyed hash, SHA-256 over the key followed by the value
- A kind of: column mask
Defined in Write each mask as SQL in one place, chapter 6. In the book since Edition 2026.11.31. Permanent link to pseudonymize
- pseudonymous identifier also pseudonymous ID, pseudonymous IDs
-
An ID with no meaning of its own, such as learner_id, is a pseudonymous identifier: it doesn't name anyone, but it leads to someone through another table.
Defined in Classify pseudonymous IDs for joining, chapter 2. In the book since Edition 2026.11.31. Permanent link to pseudonymous identifier
- purpose
-
The purpose is an attribute of the request.
Defined in Narrow by attribute: region, consent and purpose, chapter 5. In the book since Edition 2026.11.31. Permanent link to purpose
Q
- qualify
-
sqlglot parses a query into a tree, and its qualify step resolves every column against a schema, qualifies it with its table, and expands * into the columns it stands for when it has a schema to expand against
True for sqlglot 30.33.0, as this edition pins it.
- Implemented by: sqlglot 30.33.0
Defined in Find every table and column a query reads, chapter 6. In the book since Edition 2026.11.31. Permanent link to qualify
R
- recall
-
recall, the share of labelled spans of each type that a detector finds with a label of that type
- Measured from: truth
Defined in Measure a detector against labelled spans, chapter 3. In the book since Edition 2026.11.31. Permanent link to recall
- recognizer also recognizers
-
Presidio's analyzer runs recognizers: regular expressions with checksums, phone-number parsing, and a named-entity model reached through an NLP engine.
True for presidio-analyzer 2.2.366, as this edition pins it.
- Implemented by: presidio-analyzer 2.2.366
Defined in Run Presidio offline, chapter 3. In the book since Edition 2026.11.31. Permanent link to recognizer
- recording manifest
-
The recording's manifest names the model, its revision, a hash of the prompts and settings, and the date, so anyone can record again and compare.
Defined in Draft metadata with a local model, chapter 2. In the book since Edition 2026.11.31. Permanent link to recording manifest
- redact
-
redact is a NULL cast to the column's type, so the column keeps its type for anything the query does with it, such as a union or a date function.
- A kind of: column mask
Defined in Write each mask as SQL in one place, chapter 6. In the book since Edition 2026.11.31. Permanent link to redact
- redact_contacts
-
redact_contacts replaces emails and phone numbers in free text with [email] and [phone].
- A kind of: column mask
Defined in Write each mask as SQL in one place, chapter 6. In the book since Edition 2026.11.31. Permanent link to redact_contacts
- resemblance
-
Broder defined the resemblance of two documents as the Jaccard similarity of their sets of shingles
- Computed from: shingle
Defined in Find near-duplicates with MinHash, chapter 9. In the book since Edition 2026.11.31. Permanent link to resemblance
- retention
-
Retention is how long the platform keeps each kind of data.
Defined in Retention, consent and erasure, chapter 7. In the book since Edition 2026.11.31. Permanent link to retention
- retention rule also retention rules
-
A retention rule says how long one kind of data is kept, from which date the time counts, and which rows it applies to.
- Part of: retention
Defined in Keep retention rules in a reviewed file, chapter 7. In the book since Edition 2026.11.31. Permanent link to retention rule
- rewritten digest
-
The SQL the layer hands DuckDB is kept only as rewritten_digest, an HMAC-SHA256 under a key only the layer holds, which the cross-check matches against DuckDB's own log.
Defined in Keep values out of the record, chapter 8. In the book since Edition 2026.11.31. Permanent link to rewritten digest
- role also roles
-
a role is a group of principals with the same job
Defined in Access policies as code, chapter 5. In the book since Edition 2026.11.31. Permanent link to role
- role explosion
-
The alternative, a role for every combination, is what NIST's guide calls "role explosion".
- Contrasts with: attribute-based access control
Defined in Narrow by attribute: region, consent and purpose, chapter 5. In the book since Edition 2026.11.31. Permanent link to role explosion
- role-based access control also RBAC
-
Role-based access control, first formalized by Ferraiolo and Kuhn in 1992, grants access by the job a principal holds rather than by who they are.
Defined in Deny by default, then allow by role, chapter 5. In the book since Edition 2026.11.31. Permanent link to role-based access control
- row filter also row filters
-
A row filter limits which rows a query returns
- Part of: decision document
Defined in Access policies as code, chapter 5. In the book since Edition 2026.11.31. Permanent link to row filter
- run also runs
-
A run is one occurrence of a job, identified by a UUID the client generates, and the spec recommends UUIDv7.
- Part of: job
Defined in Send a run event from every pipeline, chapter 4. In the book since Edition 2026.11.31. Permanent link to run
- run event also run events
-
A run reports itself through run events: a START when it begins, then one terminal event, COMPLETE, FAIL or ABORT
Defined in Send a run event from every pipeline, chapter 4. In the book since Edition 2026.11.31. Permanent link to run event
S
- scratch platform also scratch platforms
-
lib.ci.build builds a scratch platform the way the platform runs: the source tables, the side's migrations, the deploy, which writes the metadata into the catalog and sets every table's generated properties, then the nightly run of the side's own pipelines, with lineage
Defined in Governance in CI, chapter 10. In the book since Edition 2026.11.31. Permanent link to scratch platform
- security barrier also security_barrier
-
In PostgreSQL, a view marked security_barrier keeps "maliciously-chosen functions and operators from being passed values from rows until after the view has done its work"
Defined in Gate every column on the row filters, chapter 6. In the book since Edition 2026.11.31. Permanent link to security barrier
- semi-join
-
a semi-join, a test that the review's learner_id is among the learners who consented
Defined in Build row filters from the decision's data, chapter 6. In the book since Edition 2026.11.31. Permanent link to semi-join
- shingle also shingles
-
shingles, the overlapping runs of characters or words in a text
Defined in Find near-duplicates with MinHash, chapter 9. In the book since Edition 2026.11.31. Permanent link to shingle
- skip on error
-
skip on error, which the documentation defines as "if a policy's evaluation returns error, the policy does not factor into the authorization response; it is skipped"
Defined in Write the same rules in Cedar, chapter 5. In the book since Edition 2026.11.31. Permanent link to skip on error
- snapshot expiry also expiry
-
Expiry removes snapshots, not files.
True for pyiceberg 0.12.0, as this edition pins it.
Defined in Remove the files no snapshot references, chapter 7. In the book since Edition 2026.11.31. Permanent link to snapshot expiry
- source allowlist
-
Enforcer.check_sources walks the whole query and requires every FROM and JOIN source, at every level, to be a catalog table, a CTE in scope or a subquery, and a LATERAL to be over a subquery.
- Part of: enforcement layer
Defined in Allow only catalog tables, CTEs and subqueries, chapter 6. In the book since Edition 2026.11.31. Permanent link to source allowlist
- storage limitation
-
personal data are to be "kept in a form which permits identification of data subjects for no longer than is necessary for the purposes for which the personal data are processed"
Defined in Keep retention rules in a reviewed file, chapter 7. In the book since Edition 2026.11.31. Permanent link to storage limitation
T
- table property also table properties
-
A table also has properties, a map of strings
True for pyiceberg 0.12.0, as this edition pins it.
Defined in Write descriptions into the table's schema, chapter 2. In the book since Edition 2026.11.31. Permanent link to table property
- table rewrite
-
Enforcer.rewrite replaces each catalog table in the query, under the query's own alias for it, with a subquery built from the user's decision
- Part of: enforcement layer
Defined in Replace each table with a filtered, masked subquery, chapter 6. In the book since Edition 2026.11.31. Permanent link to table rewrite
- tamper-evident
-
A log is tamper-evident when a change to it can be detected, though not prevented.
Defined in Audit and evidence, chapter 8. In the book since Edition 2026.11.31. Permanent link to tamper-evident
- trace also traces
-
lib.retention.traces notes the traces before the erasure: the IDs of every row the plan deletes, by column, and the first 15 characters, lowercased, of each identifying value.
- Comes before: erasure scan
Defined in Prove the erasure by reading every file, chapter 7. In the book since Edition 2026.11.31. Permanent link to trace
- truth
-
Every detector in this chapter is measured against one file, which we call the truth: data/platform/pii_spans.parquet.
Defined in Finding personal data, chapter 3. In the book since Edition 2026.11.31. Permanent link to truth
- type promotion also type promotions
-
The spec's valid primitive type promotions are int to long, float to double, and a decimal's precision, which may only widen, plus, in format v3, date to timestamp or timestamp_ns, and a column of type unknown to any type.
Defined in Report what a schema change reaches, chapter 10. In the book since Edition 2026.11.31. Permanent link to type promotion
U
- undefined
-
A rule whose body never evaluates to true has no value, which Rego calls undefined
Defined in Deny by default, then allow by role, chapter 5. In the book since Edition 2026.11.31. Permanent link to undefined
- unreferenced file also unreferenced files, orphan file, orphan files
-
An unreferenced file, which Iceberg calls an orphan, is under the table's location but in no snapshot's metadata.
Defined in Remove the files no snapshot references, chapter 7. In the book since Edition 2026.11.31. Permanent link to unreferenced file
W
- waiver also waivers
-
On the platform, the decision not to act on a new finding is a waiver.
Defined in Count waived findings per check, chapter 10. In the book since Edition 2026.11.31. Permanent link to waiver