Access policies as code

By the end of this chapter you can write a data platform's access rules as one policy that denies by default and returns row filters and column masks as data. The policy allows by role and narrows by attribute, and an engine can apply its whole decision. You'll write it in Rego, the policy language of Open Policy Agent (OPA), test it the way opa test, OPA's test runner, does, check it against a who-sees-what table that the data's owners can read, know where the in-process engine this book uses differs from OPA, and see the same rules in Cedar, a narrower policy language.

The platform, the online learning platform this book follows, sells professional courses to adult learners, and each of the people and programs that read its warehouse needs something different. Support agents each serve one region, americas, emea or apac, and need learners' contact details to answer tickets. Instructors see the learners and reviews of their own courses, without contact details. Marketing may reach only learners who agreed to marketing email. Finance reconciles payments, and the legacy card_number column of billing.payments must never show more than its last four digits. Data scientists see learner_id as stored, to join tables on it, with personal columns pseudonymized and free text redacted. The platform's course assistant, its AI, reads only the reviews whose authors agreed to AI training. Chapter 1 showed one rule, support agents see only their own region, and enforced it on one query. Rules for every group need to live in one place, stated exactly, and tested before anyone relies on them.

This chapter's terms are the ones the rest of the book uses. A policy is code that answers whether a request may go ahead; its answer is a decision. The principal is who asks, a role is a group of principals with the same job, and an attribute is any other fact a rule reads, about the principal, the data or the request. A row filter limits which rows a query returns, and a column mask changes how a column's values are shown, such as pseudonymization, which replaces an identifier with a consistent stand-in.

The platform's policy is governance/policies/access.rego, and its tests are in governance/policies/access_test.rego. The book's shared module lib.policy loads both into regopy 1.5.2, the Python binding of rego-cpp, Microsoft's C++ implementation of Rego, which evaluates policies inside the Python process with no server to run.

Shape the request like Trino's

A policy engine answers questions about requests, so the shape of a request decides which systems can ask it. OPA's premise is that policy is taken out of the service that enforces it: OPA "lets you decouple policy from that software service" so that it can be read, versioned and tested on its own (OPA philosophy). The service sends a JSON document describing the request, and the policy reads it as input. Trino, an open-source SQL engine, has a plugin that does exactly this: for each access check a query needs, it sends OPA a document with context.identity, the user and their groups, and action, the operation and the resource, down to the catalog, schema, table and columns (Trino 484, OPA access control).

The platform's policy reads the same fields, so it decides Trino's request to read columns, SelectFromColumns, as Trino sends it. Trino needs more than that one decision, though: it wants each row filter and column mask as SQL, a row filter as {"expression": "clause"}, whereas the policy returns a column and a value (Trino 484, OPA access control), and it also asks about other checks, such as ExecuteQuery and AccessCatalog, that this policy's allow denies (OpaAccessControl.java at Trino 484). Chapter 6 shows what a module beside the policy has to return for them. lib.policy.request builds the platform's requests in Trino's shape. Here an emea support agent, agent-a05, reads three columns of support.tickets:

1import json23from lib import policy45# A support agent's query reads three columns of the tickets table: the request Trino's OPA plugin would send for it.6request = policy.request("agent-a05", "support.tickets", ["ticket_id", "opened_at", "body"])7print(json.dumps(request, indent=2))

{
  "context": {
    "identity": {
      "user": "agent-a05"
    }
  },
  "action": {
    "operation": "SelectFromColumns",
    "resource": {
      "table": {
        "catalogName": "platform",
        "schemaName": "support",
        "tableName": "tickets",
        "columns": [
          "ticket_id",
          "opened_at",
          "body"
        ]
      }
    }
  }
}

The operation is SelectFromColumns, Trino's name for a read, and the table sits under action.resource.table with its catalog, schema, name and the columns the query touches. The platform's request differs from Trino's in two places that matter. It leaves out context.identity.groups, because the policy looks a user's groups up in the same file as their attributes, governance/users.toml, so the tests and the policy read one source of truth. And it may carry a top-level purpose, such as ai-training, which Trino doesn't send. Behind Trino, we recommend deriving the purpose from the identity, for instance by giving the training pipeline a service user of its own, rather than trusting a value the client sets.

Working rule: Shape the policy's input like the requests of the engine that will enforce it, field for field, and add only what that engine can't send, at the top level where a reviewer sees it. If your engine isn't Trino, start from its request format instead, and keep the shape in one function, as lib.policy.request does, so tests and callers can't drift apart.

Generate the data the policy reads

The request says who asks for what. Everything else a rule needs, such as each user's groups and each column's classification, comes from a second document that OPA calls data. A fact in that document is only as current as its source, and every fact the platform's policy reads has an owner already. Users and their attributes live in governance/users.toml. Each column's classification lives in governance/metadata.toml, the reviewed metadata file that chapter 2 keys by Iceberg field ID. And which instructor teaches which course is in the learning.courses table itself. Typing any of these into the policy, or copying them into a second file, makes a copy that someone has to remember to update.

lib.policy.data_document builds the document from those three sources each time a policy is loaded, and never from anything typed:

1from lib import policy23# What the policy reads besides the request, built from the files that own each fact.4data = policy.data_document()["platform"]56print("users, from governance/users.toml:", len(data["users"]))7print("agent-a05:", data["users"]["agent-a05"])8print("instructor-17:", data["users"]["instructor-17"])910classifications = data["classifications"]11print("tables, from governance/metadata.toml:", len(classifications))12print("columns classified:", sum(len(columns) for columns in classifications.values()))13print("crm.learners.email:", classifications["crm.learners"]["email"])14print("billing.payments.card_number:", classifications["billing.payments"]["card_number"])1516courses = data["instructor_courses"]17print("instructors with courses, from learning.courses:", len(courses))18print("instructor 17's courses:", courses["17"])

users, from governance/users.toml: 8
agent-a05: {'groups': ['support'], 'region': 'emea'}
instructor-17: {'groups': ['instructor'], 'instructor_id': 17}
tables, from governance/metadata.toml: 10
columns classified: 77
crm.learners.email: personal
billing.payments.card_number: sensitive
instructors with courses, from learning.courses: 48
instructor 17's courses: [2, 75, 80, 87, 103, 134]

The document holds 8 users, the classifications of 77 columns across 10 tables, and the courses of 48 instructors. agent-a05 has a group and a region, instructor-17 a group and an instructor ID, and nothing in users.toml says which courses instructor 17 teaches: the document finds [2, 75, 80, 87, 103, 134] in learning.courses. When the platform publishes another of their courses, the instructor's row filter includes it the next time the policy's data is generated, with no edit anywhere. A classification works the same way: changing card_number from sensitive would take one reviewed change to metadata.toml, and the policy would follow.

Working rule: Generate the policy's data from the file or table that owns each fact, every time you load the policy, and never copy a fact into the policy or into a second file. When a fact has no owner yet, such as a support agent's region, give it one file, reviewed like code, as the platform does with users.toml.

Deny by default, then allow by role

With the request and its data in place, the first thing a policy has to settle is what happens to a request no rule anticipated. Saltzer and Schroeder's principle of fail-safe defaults answers it: "Base access decisions on permission rather than exclusion" (Saltzer and Schroeder, 1975). In Rego, that is a default value. A rule whose body never evaluates to true has no value, which Rego calls undefined, and "The default value is used when all the rules sharing the same name are undefined" (OPA policy language), so with default allow := false, a request that no allow rule matches is denied, whoever sends it and whatever it asks for.

The rules that do allow start from roles: role-based access control, first formalized by Ferraiolo and Kuhn, grants access by the job a principal holds rather than by who they are (Ferraiolo and Kuhn, 1992). On the platform, a role is a group in users.toml, and the policy's readable object maps each group to the tables it may read. A request is allowed when it's a read, some group of the user may read the table, and nothing the group's row filters depend on is missing:

default allow := false

allow if {
	input.action.operation == "SelectFromColumns"
	readable_by_a_group
	not missing_attribute
}

The last line is a guard against a failure this chapter's section on production shows by removing it. The example prints the map and decides four requests:

1from lib import policy23rego = policy.interpreter()45# The groups the policy knows, and the tables each may read: the role part of the policy.6readable = policy.evaluate(rego, "data.platform.access.readable", {})7for group, tables in readable.items():8    print(f"{group} may read:", ", ".join(tables))910# Who may read what. Anything no rule allows is denied, including a user or an operation the policy has never seen.11requests = [12    policy.request("finance-1", "billing.payments", ["amount"]),13    policy.request("marketing-1", "billing.payments", ["amount"]),14    policy.request("someone-else", "crm.learners", ["learner_id"]),15]16delete = policy.request("finance-1", "billing.payments", ["amount"])17delete["action"]["operation"] = "DeleteFromTable"18for request in requests + [delete]:19    decision = policy.decide(request, rego)20    who, table = request["context"]["identity"]["user"], request["action"]["resource"]["table"]21    print(f"{who}, {request['action']['operation']} on {table['schemaName']}.{table['tableName']}:",22          "allowed" if decision["allow"] else "denied", "|", "; ".join(decision["reasons"]))

ai may read: learning.courses, learning.reviews
data-science may read: analytics.course_performance, analytics.learner_activity, billing.payments, crm.learners, learning.courses, learning.enrollments, learning.events, learning.reviews
finance may read: billing.payments, learning.courses, learning.enrollments
instructor may read: learning.courses, learning.enrollments, learning.reviews
marketing may read: crm.learners, learning.courses, marketing.audience
support may read: crm.learners, support.tickets
finance-1, SelectFromColumns on billing.payments: allowed | finance may read billing.payments
marketing-1, SelectFromColumns on billing.payments: denied | no group of this user may read this table
someone-else, SelectFromColumns on crm.learners: denied | no group of this user may read this table
finance-1, DeleteFromTable on billing.payments: denied | DeleteFromTable is not an operation this policy allows

Finance may read billing.payments, and the reason says which group allowed it. Marketing may not, and neither may someone-else, a user the policy has never heard of: nothing grants either of them anything, so the default holds. The fourth request is finance again, on the same table, asking to delete rather than read. No rule mentions DeleteFromTable, so it's denied too, and the reason says why. A policy that tried to list what is forbidden would have needed a rule for that operation; this one needs none.

Every decision carries its reasons, so a denial can be explained to the person who got it, and logged.

Working rule: Start every policy with default allow := false, and allow only through rules that name the role and the resource. Keep the map from roles to tables in one place, where a reviewer can read all of it in one sitting, and give every decision a reason. When a request is denied that should have been allowed, add a rule that allows it; never weaken the default.

Narrow by attribute: region, consent and purpose

Roles say which tables a group may read; they can't say which rows. NIST's guide to attribute-based access control defines the general approach: access "is determined by evaluating attributes associated with the subject, object, requested operations, and, in some cases, environment conditions", and it treats roles as one such attribute: "RBAC works on the attribute of 'role'" (NIST SP 800-162). The alternative, a role for every combination, ends in what the same guide calls "role explosion".

The platform shows why. Support by region would need three support roles; add marketing's consent and a purpose for AI training, and the roles multiply again. Instead, the policy keeps one role per job and reads three attributes. The region is an attribute of the principal, from users.toml. Consent is an attribute of the data: the learner's marketing_consent and ai_training_consent columns in crm.learners. The purpose is an attribute of the request. Each one becomes a row filter, a structured condition on a column that the engine adds to the query. The support agent's filter reads the agent's region:

row_filters contains {"column": "support_region", "equals": user.region} if {
	"support" in groups
	table in {"crm.learners", "support.tickets"}
}

The example asks for the filters of three support agents, an instructor, marketing, and a data scientist reading reviews twice, once with no purpose and once for ai-training:

1import json23from lib import policy45rego = policy.interpreter()67# Region is the agent's attribute, consent the learner's, and purpose the request's. Each becomes a row filter.8reads = [9    ("agent-a01", "support.tickets", ["ticket_id"], None),10    ("agent-a05", "support.tickets", ["ticket_id"], None),11    ("agent-a09", "support.tickets", ["ticket_id"], None),12    ("instructor-17", "learning.reviews", ["rating"], None),13    ("marketing-1", "crm.learners", ["email"], None),14    ("analyst-1", "learning.reviews", ["rating", "body"], None),15    ("analyst-1", "learning.reviews", ["rating", "body"], "ai-training"),16]17for user, table, columns, purpose in reads:18    decision = policy.decide(policy.request(user, table, columns, purpose), rego)19    who = user + (f" for {purpose}" if purpose else "")20    print(f"{who} reads {table}, row filters:", json.dumps(decision["row_filters"]))21    if "body" in columns:22        print(f"{who} reads {table}, body shown as:", decision["column_masks"]["body"])

agent-a01 reads support.tickets, row filters: [{"column": "support_region", "equals": "americas"}]
agent-a05 reads support.tickets, row filters: [{"column": "support_region", "equals": "emea"}]
agent-a09 reads support.tickets, row filters: [{"column": "support_region", "equals": "apac"}]
instructor-17 reads learning.reviews, row filters: [{"column": "course_id", "in": [2, 75, 80, 87, 103, 134]}]
marketing-1 reads crm.learners, row filters: [{"column": "marketing_consent", "equals": true}, {"column": "status", "equals": "active"}]
analyst-1 reads learning.reviews, row filters: []
analyst-1 reads learning.reviews, body shown as: redact
analyst-1 for ai-training reads learning.reviews, row filters: [{"learner_consented": "ai_training_consent"}]
analyst-1 for ai-training reads learning.reviews, body shown as: redact_contacts

The three agents get the same rule with three values, one per region, from one role. Instructor 17's filter lists their courses, generated from learning.courses. Marketing gets two filters, consent given and account active, and an engine applies both. The data scientist's two reads show what the purpose does. With no purpose, every review is visible, but the review text, body, is shown as redact. For ai-training, only the reviews of learners who agreed to AI training remain, and the text comes back as redact_contacts, with contact details taken out. That filter, learner_consented, tests the review's author, not the review, so enforcing it takes a join to crm.learners. A learner who signed up before the platform asked about AI training has no answer in ai_training_consent, according to the platform's data card, so enforcing the filter has to count an unknown answer as a no.

The purpose is the one attribute the requester supplies, so it reveals more only under a narrower condition: stating ai-training shows review text, but only for consenting learners and without contact details. Leaving it out, or giving any other purpose, gets what the role allows anyway, with the text hidden.

Working rule: Model each condition as an attribute of the principal, the data or the request, read by one rule per role, rather than as another role. When an attribute comes from the requester, as a purpose does, let it unlock access only under a narrower condition, so that leaving it out, or getting it wrong, falls back to what the role allows anyway.

Return the decision as one document

A yes or no can't carry the row filters that attributes produce, but a Rego decision isn't limited to yes or no: "OPA policies can represent decisions as arbitrary structured data (e.g., booleans, strings, maps, maps of lists of maps, etc.)" (OPA philosophy). The platform's policy uses that to return its whole answer as one document, data.platform.access.decision, with four keys: allow, the row_filters, the column_masks, and the reasons. lib.policy.decide asks for it with one query and returns it as a Python dictionary.

The filters and masks are data, never SQL. A filter names a column and a value or a list of values, or, for consent, the learner's consent column; a mask is a name, one of redact, last4, pseudonymize and redact_contacts. Whatever enforces the decision turns each into SQL with its own code, so nothing a request or a user record contains, such as a region typed into users.toml, can reach a query as SQL text. Masks follow each column's classification from the metadata. Sensitive columns are hidden from everyone, except the last four digits of a card number for finance. Personal columns are shown to support, which needs contact details, and masked for everyone else as their job requires. And a column the metadata doesn't classify is treated as sensitive:

classification(column) := level if {
	level := data.platform.classifications[table][column]
} else := "sensitive"

For the platform, take marketing reading four columns of crm.learners. Three are in the metadata. The fourth, linkedin_url, stands for a column that a pipeline has added to the table and that nobody has classified yet:

1import json23from lib import policy45# Marketing reads four columns of crm.learners. `linkedin_url` stands for a column added to the table and not yet6# classified in governance/metadata.toml.7request = policy.request("marketing-1", "crm.learners", ["full_name", "email", "phone", "linkedin_url"])8decision = policy.decide(request)9print(json.dumps(decision, indent=2))10print("keys:", ", ".join(sorted(decision)))

{
  "allow": true,
  "column_masks": {
    "linkedin_url": "redact",
    "phone": "redact"
  },
  "reasons": [
    "linkedin_url is sensitive, shown as redact",
    "marketing may read crm.learners",
    "phone is personal, shown as redact"
  ],
  "row_filters": [
    {
      "column": "marketing_consent",
      "equals": true
    },
    {
      "column": "status",
      "equals": "active"
    }
  ]
}
keys: allow, column_masks, reasons, row_filters

The document allows the read, with marketing's two row filters. Two columns are masked: phone, which is personal and which marketing doesn't need for email, and linkedin_url, which is unclassified and so treated as sensitive until someone reviews it. full_name and email have no mask, and a column without a mask is shown as stored: marketing needs both to send an email. The reasons give one line per group that allowed the read and one per mask, so the document explains itself.

A denial carries empty filters and masks. The policy builds the two outcomes with two separate rules, so nothing downstream can mistake a denial for a read with some columns masked.

Working rule: Return one decision document per request, holding the allow, the row filters, the column masks and the reasons, and read it as a whole. Keep filters and masks as names and values that your enforcement code translates, never as SQL that the policy writes. If an engine needs SQL from the policy, as Trino does for its row filters and column masks (Trino 484, OPA access control), generate it in one tested function between the two.

Unit-test the policy in Rego

The decision document is what a test can compare. That matters, because a policy decides who sees personal data, and a change to it can widen access without anyone seeing it in a diff. OPA's answer is tests written in Rego: a test is a rule whose name is "prefixed with test_", and "If the test rule is undefined or generates a non-true value the test result is reported as FAIL" (OPA policy testing). A test sets up its request with with, which replaces part of input or data for one expression.

The book's engine, regopy, has no opa test, so lib.policy.run_tests plays its part: it finds every test_ rule in the folder's _test.rego files, evaluates each, and reports PASS when the rule is true, FAIL when it's false or undefined, and ERROR when evaluation fails. Each of the platform's tests sets the user and the action as two separate paths, each to a literal value, for reasons the section on regopy shows:

test_finance_sees_only_the_last_four_digits if {
	decision := access.decision with input.context.identity.user as "finance-1"
		with input.action as {"operation": "SelectFromColumns", "resource": {"table": {"schemaName": "billing", "tableName": "payments", "columns": ["amount", "card_number"]}}}
	decision.allow
	decision.row_filters == []
	decision.column_masks == {"card_number": "last4"}
}

The tests cover each group's allow, its filters and its masks, and the ways a request is denied: a table the group isn't given, an operation other than a read, a user the policy doesn't know, and a support agent with no region.

1from collections import Counter23from lib import policy45# Every `test_` rule in governance/policies/access_test.rego, run by name: PASS, FAIL or ERROR, as `opa test` reports.6results = policy.run_tests()7for test, result in results.items():8    print(f"{result}: {test}")9print("tests passed:", f"{Counter(results.values())['PASS']} of {len(results)}")

PASS: test_support_reads_only_its_own_region
PASS: test_support_without_a_region_is_denied
PASS: test_marketing_sees_only_consenting_active_learners
PASS: test_a_group_only_reads_its_own_tables
PASS: test_finance_sees_only_the_last_four_digits
PASS: test_data_science_never_sees_card_numbers
PASS: test_data_science_sees_pseudonyms_not_people
PASS: test_an_unclassified_column_is_hidden
PASS: test_an_unknown_user_is_denied
PASS: test_an_instructor_sees_only_their_courses
PASS: test_the_assistant_reads_only_consented_reviews
PASS: test_training_reads_only_consented_reviews_without_contacts
tests passed: 12 of 12

The run passes 12 of 12 tests. A test that checks only decision.allow would miss most mistakes, so every test of an allowed read compares the row filters and the column masks whole. In finance's test, decision.row_filters == [] fails if the policy adds any filter to finance's reads, and decision.column_masks == {"card_number": "last4"} fails if it masks one column more or one fewer, or shows the card number another way.

Working rule: Write a test for every rule that grants or narrows access, and one for every way a request can be denied, and compare whole filters and masks, not single keys. Run the tests on every change to the policy and to the data the policy reads.

Check who sees what from a decision table

Unit tests are written in Rego by whoever writes the policy. The people who own the data, such as the support lead or the finance team, should be able to check the policy too, without reading Rego. A decision table gives them that: one row per request, saying who reads which table and what they must get back, in the terms of their job.

The platform's table is lib.policy.WHO_SEES_WHAT, a list of rows in Python. Each gives a user, a table, the columns read and an optional purpose, then whether the read is allowed, which columns the row filters test and which columns are masked, and how. The example runs every row through lib.policy.decide, which returns the decision document, and prints what the policy decided, one line per row: the form to show the data's owners.

1from lib import policy23rego = policy.interpreter()45# Each row of lib.policy.WHO_SEES_WHAT is a request and the decision a reviewer expects for it.6matching = 07for case in policy.WHO_SEES_WHAT:8    decision = policy.decide(policy.request(case.user, case.table, case.columns, case.purpose), rego)9    matching += policy.matches(case, decision)10    who = case.user + (f" ({case.purpose})" if case.purpose else "")11    shown = "allowed" if decision["allow"] else "denied"12    filters = ", ".join(policy.filtered_on(decision)) or "-"13    masks = ", ".join(f"{column}: {mask}" for column, mask in decision["column_masks"].items()) or "-"14    print(f"{who:26} {case.table:17} {shown:8} rows: {filters:29} masks: {masks}")15print("rows of the table the policy matches:", f"{matching} of {len(policy.WHO_SEES_WHAT)}")

agent-a05                  crm.learners      allowed  rows: support_region                masks: -
agent-a05                  billing.payments  denied   rows: -                             masks: -
instructor-17              learning.reviews  allowed  rows: course_id                     masks: body: redact_contacts
instructor-17              crm.learners      denied   rows: -                             masks: -
marketing-1                crm.learners      allowed  rows: marketing_consent, status     masks: phone: redact
finance-1                  billing.payments  allowed  rows: -                             masks: card_number: last4
analyst-1                  billing.payments  allowed  rows: -                             masks: card_number: redact
analyst-1                  crm.learners      allowed  rows: -                             masks: email: pseudonymize, notes: redact
analyst-1                  learning.reviews  allowed  rows: -                             masks: body: redact
analyst-1 (ai-training)    learning.reviews  allowed  rows: ai_training_consent           masks: body: redact_contacts
assistant-1                learning.reviews  allowed  rows: ai_training_consent           masks: body: redact_contacts
someone-else               crm.learners      denied   rows: -                             masks: -
rows of the table the policy matches: 12 of 12

12 of 12 rows match. Read down the table and each role's rules show up as outcomes: the support agent filtered by region and denied payments; the instructor filtered by course and denied the learners table; finance seeing card_number as last4 where the data scientist sees redact; the data scientist's two reads of reviews, with and without a purpose. A reviewer who thinks a row is wrong can say so in a pull request without knowing which rule produced it.

Working rule: Keep a who-sees-what table in the repository with the policy, run it with the unit tests, and show its printed form to the data's owners, in terms they can check. When an owner disagrees with a row, change the row first, and let the failing row drive the change to the policy.

Know where regopy differs from OPA

The platform's tests run in regopy because the book's examples must run offline, in seconds, inside Python, and OPA is a separate program. regopy 1.5.2 is close to OPA: its engine says "We support v1.18.1 of Rego as defined by OPA" and that it passes "all the non-builtin specific test suites" OPA publishes, and it lists the built-ins it doesn't support, such as net.* and http.send (rego-cpp README at v1.5.2). regopy also differs from OPA in ways its README doesn't list. The platform's policy and tests avoid each of the cases that follow, and OPA itself is the reference for anything in production.

The first group runs without any error:

1import json23from regopy import Input, Interpreter456def ask(rules: str, query: str, request: dict) -> str:7    """One query's answer from regopy: the value bound to x, or "undefined"."""8    rego = Interpreter()9    rego.add_module("example", "package example\n" + rules)10    rego.set_input(Input(request))11    output = str(rego.query(query))12    return output if output == "undefined" else json.dumps(json.loads(output)["bindings"]["x"])131415admins_only = 'default allow := false\nallow if input.user == "admin"\n'16print("a false decision, queried on its own:", ask(admins_only, "data.example.allow", {"user": "guest"}))17print("the same decision, bound to a variable:", ask(admins_only, "x := data.example.allow", {"user": "guest"}))1819pre_1_0 = 'allow { input.user == "admin" }\n'                      # no `if`: OPA 1.x refuses to load this20print("a rule in pre-1.0 syntax:", ask(pre_1_0, "x := data.example.allow", {"user": "admin"}))2122typo = 'region := "emea"\ndefault allow := false\nallow if not deny\ndeny if input.region != regoin\n'23print("a deny with a misspelt, unbound variable, for apac:", ask(typo, "x := data.example.allow", {"region": "apac"}))2425mocks = """region := input.user.region26emea := {"region": "emea"}27with_a_literal := seen if { seen := region with input.user as {"region": "emea"} }28with_a_reference := seen if { seen := region with input.user as emea }29"""30print("region under a mock written as a literal:", ask(mocks, "x := data.example.with_a_literal", {"user": {"region": "apac"}}))31print("region under a mock that refers to a rule:", ask(mocks, "x := data.example.with_a_reference", {"user": {"region": "apac"}}))

a false decision, queried on its own: undefined
the same decision, bound to a variable: false
a rule in pre-1.0 syntax: true
a deny with a misspelt, unbound variable, for apac: true
region under a mock written as a literal: "emea"
region under a mock that refers to a rule: "apac"

• A false decision comes back undefined. Asked for data.example.allow on its own, regopy answers undefined where the value is false, so "denied" can't be told from "no decision". Bound to a variable, the same query gives false. lib.policy binds every decision it reads.
• Pre-1.0 syntax runs. Since OPA 1.0, "Using if for all rule definitions and contains for multi-value rules is now mandatory" (OPA v1.0.0 release notes), and rego-cpp's changelog says that "Support for v0 Rego has been dropped" (rego-cpp changelog). Yet regopy evaluates allow { ... } and answers true, so a policy can pass here and fail to load in OPA, and so behind Trino.
• An unbound variable switches a rule off. OPA requires a variable in a comparison to "appear in another expression in the same rule that would cause the variable to be bound" (OPA policy language), and rejects the policy otherwise. regopy runs it: a deny with regoin misspelt for region is silently undefined, so allow if not deny allows an apac request: true.
• A mock that refers to another rule is ignored. In a policy whose region rule reads input.user.region, a mock written as a literal, with input.user as {"region": "emea"}, gives "emea". Written as a reference to a rule holding the same object, it gives the real input's "apac".

The second group fails, and one of the failures takes Python down with it. The first example adds a test to a copy of the platform's tests, and the second formats the same string from nine sources:

1import signal2import subprocess3import sys4import tempfile5from collections import Counter6from pathlib import Path78from lib import policy910# The platform's tests, plus one that replaces the whole input at once rather than one path at a time.11whole_input = """12test_finance_reads_payments if {13	access.decision.allow with input as {"context": {"identity": {"user": "finance-1"}}, "action": {"operation": "SelectFromColumns", "resource": {"table": {"schemaName": "billing", "tableName": "payments", "columns": ["amount"]}}}}14}15"""16with tempfile.TemporaryDirectory() as name:17    folder = Path(name)18    for path in policy.POLICIES.glob("*.rego"):19        (folder / path.name).write_text(path.read_text(encoding="utf-8"), encoding="utf-8")20    with open(folder / "access_test.rego", "a", encoding="utf-8") as tests:21        tests.write(whole_input)22    results = policy.run_tests(folder)23    print("tests that end in ERROR:", f"{Counter(results.values())['ERROR']} of {len(results)}")24    rego = policy.interpreter(folder, tests=True)25    print("the query true succeeds:", rego.query("true").ok())2627    # lib.policy checks each output before reading it, so a decision asked of this interpreter raises an error.28    try:29        policy.decide(policy.request("finance-1", "billing.payments", ["amount"]), rego)30    except RuntimeError as error:31        print("lib.policy.decide raises:", error)3233    # Printing one of those failed outputs, in a separate Python process.34    crash = "import sys; from pathlib import Path; from lib import policy; print(policy.interpreter(Path(sys.argv[1]), tests=True).query('true'))"35    ended = subprocess.run([sys.executable, "-c", crash, str(folder)], capture_output=True)36    print("printing the failed output ends Python with:", signal.Signals(-ended.returncode).name)

tests that end in ERROR: 13 of 13
the query true succeeds: False
lib.policy.decide raises: regopy couldn't evaluate data.platform.access.decision, so there is no decision to read
printing the failed output ends Python with: SIGSEGV

1import json23from regopy import Input, Interpreter45# The same label, sprintf("%s.%s", [schema, input.table]), with the schema "crm" taken from nine places.6helpers = 'crm := "crm"\nschema_of(x) := "crm" if x == 1\n'7cases = [8    ("from input.schema", "input.schema", "true"),9    ("from a rule", "crm", "true"),10    ("from a function", "schema_of(1)", "true"),11    ("from iterating over a set", "s", 'some s in {"crm"}'),12    ("from object.get, the key found", 'object.get(input, "schema", "public")', "true"),13    ("from object.get, its default", 'object.get(input, "namespace", "crm")', "true"),14    ("from lower(input.schema)", "lower(input.schema)", "true"),15    ("from concat", 'concat("", ["c", "rm"])', "true"),16    ("from a local variable set to a literal", "v", 'v := "crm"'),17]18for name, schema, body in cases:19    rego = Interpreter()20    rego.add_module("example", f'package example\n{helpers}label := sprintf("%s.%s", [{schema}, input.table]) if {{ {body} }}\n')21    rego.set_input(Input({"schema": "crm", "table": "learners"}))22    try:23        print(f"{name}:", json.loads(str(rego.query("x := data.example.label")))["bindings"]["x"])24    except json.JSONDecodeError as error:25        print(f"{name}, regopy's invalid JSON:", error.doc)

from input.schema: crm.learners
from a rule: crm.learners
from a function: crm.learners
from iterating over a set: crm.learners
from object.get, the key found: crm.learners
from object.get, its default, regopy's invalid JSON: {"expressions":[true], "bindings":{"x":""crm".learners"}}
from lower(input.schema), regopy's invalid JSON: {"expressions":[true], "bindings":{"x":""crm".learners"}}
from concat, regopy's invalid JSON: {"expressions":[true], "bindings":{"x":""crm".learners"}}
from a local variable set to a literal, regopy's invalid JSON: {"expressions":[true], "bindings":{"x":""crm".learners"}}

• Replacing the whole input breaks every query. One extra test that writes its request as with input as {...}, and evaluates rules that call built-ins such as sprintf, makes every query in that interpreter fail. Every test then ends in ERROR, 13 of 13, and so does the first query asked of a fresh interpreter loaded with the same files, before any test runs: asked whether the query true succeeded, regopy says False. Replacing one path at a time, as the platform's tests do, works.
• Printing a failed output crashes Python. Converting one of those failed outputs to a string ends the process with SIGSEGV. So lib.policy checks output.ok() before it reads anything, and a decision asked of that interpreter raises an error instead: regopy couldn't evaluate data.platform.access.decision, so there is no decision to read.
• sprintf keeps stray quotes. Formatting a string returned by some built-ins, such as object.get's default, lower or concat, or a local variable set to a literal, keeps its quotes, and regopy's own output is then invalid JSON, {"expressions":[true], "bindings":{"x":""crm".learners"}}, which its Python wrapper fails to parse. Strings from input, a rule, a function, iteration and object.get's found value format cleanly, and the platform's policy formats only strings like those.

Working rule: Treat regopy as a fast local evaluator and OPA as the reference. Write Rego v1 only, bind every decision to a variable, mock input one path at a time with literal values, and check that an output succeeded before reading it. Before a policy guards production data, check and test it with OPA itself, with opa check --strict and opa test.

Write the same rules in Cedar

Rego isn't the only language for policies like the platform's. Cedar, which its designers present as a language for "Expressive, Fast, Safe, and Analyzable Authorization" (Cutler et al., 2024), takes a narrower view of a decision. A Cedar request names a principal, an action and a resource, and the answer follows three rules: "default deny", "forbid overrides permit", and "skip on error: if a policy's evaluation returns error, the policy does not factor into the authorization response; it is skipped" (Cedar, authorization). A schema declares every type of entity, the kinds of principal and resource a request can name, with its attributes, and "By default, attributes that you define are required" (Cedar, schema). Validating policies against the schema catches mistakes before any request: "If your policies are deemed valid, they are sure not to exhibit most errors that could arise during request evaluation, for requests that adhere to the expectations defined by the schema" (Cedar, validation). The book's examples run Cedar with cedarpy 4.12.2, which "helps you use the (Rust) Cedar Policy library from Python" (cedarpy).

The platform's rules in Cedar are in governance/policies/access.cedar, with the schema in access.cedarschema. A permit per group repeats the Rego policy's readable map. A forbid keeps support agents to learners in their own region, and another hides sensitive columns from everyone except finance's view of card numbers. lib.policy.cedar_entities generates the users, groups, tables and columns from the same data document the Rego policy reads, and the example adds the first learner of crm.learners in each region:

1import cedarpy23from lib import policy4from lib.platform import load56policies = policy.CEDAR_POLICIES.read_text(encoding="utf-8")7schema = policy.CEDAR_SCHEMA.read_text(encoding="utf-8")8print("policies valid against the schema:", cedarpy.validate_policies(policies, schema).validation_passed)9typo = cedarpy.validate_policies(policies.replace("principal.region", "principal.regoin"), schema)10print("with region misspelt:", typo.errors[0].error)1112# Entities generated from the files the Rego policy reads, plus the first learner of crm.learners in each region.13first = {}14for learner in load("crm.learners").select(["learner_id", "support_region"]).to_pylist():15    first.setdefault(learner["support_region"], learner)16entities = policy.cedar_entities(learners=first.values())171819def ask(user: str, resource: str) -> cedarpy.AuthzResult:20    request = {"principal": f'User::"{user}"', "action": 'Action::"read"', "resource": resource, "context": {}}21    return cedarpy.is_authorized(request, policies, entities, schema)222324for user, kind, name in [("finance-1", "Column", "billing.payments.card_number"),25                         ("analyst-1", "Column", "billing.payments.card_number"),26                         ("analyst-1", "Column", "billing.payments.amount"),27                         ("marketing-1", "Table", "billing.payments")]:28    print(f"{user} reads {name}:", ask(user, f'{kind}::"{name}"').decision.value)29for region, learner in sorted(first.items()):30    print(f"agent-a05 (emea) reads a learner in {region}:", ask("agent-a05", f'Learner::"{learner["learner_id"]}"').decision.value)31answer = ask("finance-1", 'Column::"billing.payments.card_number"')32print("everything Cedar returns for finance-1's card numbers:", answer.decision.value, list(answer.diagnostics.reasons), list(answer.diagnostics.errors))

policies valid against the schema: True
with region misspelt: for policy `policy6`, attribute `regoin` on entity type `User` not found
finance-1 reads billing.payments.card_number: Allow
analyst-1 reads billing.payments.card_number: Deny
analyst-1 reads billing.payments.amount: Allow
marketing-1 reads billing.payments: Deny
agent-a05 (emea) reads a learner in americas: Deny
agent-a05 (emea) reads a learner in apac: Deny
agent-a05 (emea) reads a learner in emea: Allow
everything Cedar returns for finance-1's card numbers: Allow ['policy3'] []

The policies validate against the schema, and misspelling region gives a precise error before anything runs: for policy policy6, attribute regoin on entity type User not found. Rego has no such check by default. The same typo in an attribute, user.regoin in the support agent's row filter, is undefined in OPA and regopy alike, and opa check --strict passes it, because OPA checks references against a schema only when you give it one (OPA policy language, schemas), and the platform gives none. The agent's row filter disappears, and only a test that compares the filters, such as test_support_reads_only_its_own_region, catches it. The decisions agree with the Rego policy where Cedar can express them. Finance may read the card_number column, and the data scientist may not, where Rego allows the read and redacts the column. The data scientist may read amount, and marketing may not read payments. The emea agent may read the emea learner and neither of the others.

The last line of the output is everything Cedar returns for finance's card numbers: a decision, and the ID of the policy that decided it. cedarpy names the policies by position, policy3 for the fourth, rather than by their @id annotations, such as finance-tables. Cedar's documentation explains why that's allowed: "@id is not special in the Cedar language … The Cedar CLI uses the @id annotation to set policy IDs, but other interfaces, such as the Cedar APIs, have other ways to set policy IDs" (Cedar, syntax). Nothing in the answer says last4, the mask the Rego policy returns for finance's card numbers, because Cedar can permit a column but has no way to say "show the last four digits": that rule would move into the enforcement code, outside the policy and its tests. Rows are the same. Cedar decides one learner at a time, so filtering the platform's learners means one request per learner, or Cedar's experimental partial evaluation, which "evaluates whatever is known and returns residual policies for the parts that aren't", and whose result "is not a final authorization decision" (cedarpy). Cedar's paper makes the trade-off from the other side: "Rego's added expressiveness is a blessing but also a curse. It provides no built-in authorization concepts such as 'allow' or 'deny'". The same paper reports Cedar 42.8 to 80.8 times faster than Rego at the median, on three workloads with transitive relations, with Cedar 3.0.1 against OPA's Rego 0.61.0. Those are its designers' measurements, not this book's, and they say nothing about regopy.

Working rule: Choose Cedar when every question is "may this principal do this to this resource?", as in an application's permission checks, and you want a schema to check every policy before it runs. Choose Rego when the answer has to carry data, as a warehouse's row filters and column masks do, or when the engine already asks OPA, as Trino does. For the platform's warehouse, we recommend Rego.

What goes wrong in production

We consider the most damaging failure a rule that silently doesn't apply because an attribute is missing, and it happens in both languages. In Rego, "Expressions that refer to undefined values are also undefined" (OPA policy language). The support agent's row filter takes its value from user.region, so for a user without a region the filter doesn't exist at all. A rule that allows by group still allows, and no filter means every row.

To see it, take a copy of the platform's policy and tests, in a temporary folder, with the one line not missing_attribute taken out of allow. Then add a support agent to the data with their group and no region, as happens when someone is given a role before anyone sets their region, and ask both policies for the agent's tickets:

1import tempfile2from pathlib import Path34from lib import policy5from lib.platform import load67# A support agent added to users.toml with their group, before anyone has set their region.8data = policy.data_document()9data["platform"]["users"]["agent-new"] = {"groups": ["support"]}10request = policy.request("agent-new", "support.tickets", ["ticket_id", "body"])11regions = sorted(set(load("support.tickets").column("support_region").to_pylist()))1213# A copy of the platform's policy and tests, with one line taken out of `allow`: the missing-attribute guard.14with tempfile.TemporaryDirectory() as name:15    folder = Path(name)16    for path in policy.POLICIES.glob("*.rego"):17        (folder / path.name).write_text(path.read_text(encoding="utf-8").replace("\tnot missing_attribute\n", ""), encoding="utf-8")1819    for label, policies in [("with the guard", policy.POLICIES), ("without the guard", folder)]:20        decision = policy.decide(request, policy.interpreter(policies, data=data))21        seen = [r for r in regions if decision["allow"] and all(f["equals"] == r for f in decision["row_filters"])]22        print(f"{label}, allowed:", decision["allow"])23        print(f"{label}, reasons:", "; ".join(decision["reasons"]))24        print(f"{label}, row filters:", decision["row_filters"])25        print(f"{label}, regions whose tickets the agent sees:", ", ".join(seen) or "none")2627    failing = [test for test, result in policy.run_tests(folder).items() if result != "PASS"]28print("tests that fail without the guard:", ", ".join(failing))

with the guard, allowed: False
with the guard, reasons: support's row filter needs the user's region, and there is none
with the guard, row filters: []
with the guard, regions whose tickets the agent sees: none
without the guard, allowed: True
without the guard, reasons: support may read support.tickets; support's row filter needs the user's region, and there is none
without the guard, row filters: []
without the guard, regions whose tickets the agent sees: americas, apac, emea
tests that fail without the guard: test_support_without_a_region_is_denied

With the guard, the agent is denied, and the reason names what is missing: support's row filter needs the user's region, and there is none. Without it, the request is allowed with row filters [], and the agent sees the tickets of americas, apac, emea: every region, including every learner's message and contact details. The decision's own reasons say the region is missing, but nothing in allow asked. default allow := false didn't help either, because a rule did allow. One test catches it, test_support_without_a_region_is_denied, which is why the policy's tests include the case of the missing attribute.

Cedar fails the same way, through skip on error. Take the support agents' forbid from the platform's Cedar policies without its has region check, so it reads principal.region on an agent who has none:

1import cedarpy23from lib import policy4from lib.platform import load56guarded = policy.CEDAR_POLICIES.read_text(encoding="utf-8")7unguarded = guarded.replace("principal has region && ", "")            # the same forbid, without its `has` check8schema = policy.CEDAR_SCHEMA.read_text(encoding="utf-8")910# A new support agent with no region yet, and the first learner of crm.learners in each region.11data = policy.data_document()12data["platform"]["users"]["agent-new"] = {"groups": ["support"]}13first = {}14for learner in load("crm.learners").select(["learner_id", "support_region"]).to_pylist():15    first.setdefault(learner["support_region"], learner)16entities = policy.cedar_entities(learners=first.values(), data=data)1718for name, policies in [("with `has region`", guarded), ("without it", unguarded)]:19    for region, learner in sorted(first.items()):20        request = {"principal": 'User::"agent-new"', "action": 'Action::"read"',21                   "resource": f'Learner::"{learner["learner_id"]}"', "context": {}}22        answer = cedarpy.is_authorized(request, policies, entities, schema)23        print(f"{name}, a learner in {region}:", answer.decision.value)24    print(f"{name}, policies skipped on an error:", list(answer.diagnostics.errors))25print("validation without it:", cedarpy.validate_policies(unguarded, schema).errors[0].error)

with `has region`, a learner in americas: Deny
with `has region`, a learner in apac: Deny
with `has region`, a learner in emea: Deny
with `has region`, policies skipped on an error: []
without it, a learner in americas: Allow
without it, a learner in apac: Allow
without it, a learner in emea: Allow
without it, policies skipped on an error: ['error while evaluating policy `policy6`: `User::"agent-new"` does not have the attribute `region`']
validation without it: for policy `policy6`, unable to guarantee safety of access to optional attribute `region` on entity type `User`

With the check, the agent is denied every learner. Without it, evaluating the forbid raises an error, Cedar skips the policy, the support permit still applies, and the agent may read the learners of all three regions. Passing the schema to is_authorized didn't stop it, because the schema declares region optional: users outside support have none. Validation did, before any request: for policy policy6, unable to guarantee safety of access to optional attribute region on entity type User.

Other failures to watch for:

• A misspelt attribute is undefined, not an error. user.regoin for user.region passes opa check --strict, and the support filter disappears as it does for a missing region. Compare whole filters in the tests, and give OPA a schema if you want the check before any test runs.
• A deny rule that reads a missing attribute fails open. The guard fixes allow; a deny or forbid needs the same care. Deny explicitly when a required attribute is absent, and write the test for it before the rule.
• Policies nobody can test. Rules kept in an engine's settings or in hand-made views per role can't be unit-tested or diffed, so they drift. Keep the policy, its tests and its decision table in the repository, and run all three on every change.
• regopy passes and OPA refuses. Pre-1.0 syntax and unbound variables run in regopy. Check every policy with opa check --strict before deploying it to OPA.
• Data that changed under the policy. A support agent moved to another region is one edit to users.toml, and the decision follows only when the data document is rebuilt. Rebuild it whenever its sources change, and deploy it with the policy.

What this chapter decided

• One policy, in Rego, governance/policies/access.rego, denies by default, allows by group, and narrows by three attributes: the agent's region, the learner's consent and the request's purpose.
• Requests are shaped like Trino's OPA requests, built by lib.policy.request, plus an optional top-level purpose; groups come from users.toml, not from the request.
• The policy's data is generated by lib.policy.data_document from users.toml, metadata.toml and learning.courses, and never typed.
• Each decision is one document with allow, row_filters, column_masks and reasons. Filters and masks are data, never SQL, and the masks are redact, last4, pseudonymize and redact_contacts. A denial carries no filters or masks.
• Missing facts fail closed: a column the metadata doesn't classify is treated as sensitive, and a user whose row filter needs a missing attribute is denied.
• The one purpose the policy knows is ai-training, which limits reviews to learners who agreed to AI training and shows their text with contact details removed. A read without a purpose gets what the role allows, with review text hidden.
• Two kinds of test run on every change: the Rego unit tests through lib.policy.run_tests, and the who-sees-what table, lib.policy.WHO_SEES_WHAT.
• regopy for the examples, OPA as the reference: Rego v1 only, every decision bound to a variable, input mocked one path at a time with literals, and every output checked before it's read.
• Cedar is for comparison (governance/policies/access.cedar). It suits allow-or-deny checks on single resources; the platform's warehouse needs filters and masks as data, so it uses Rego.