Give the agent layered context

Unos 30 minutos de lectura, con 9 ejemplos en 9 secciones.

Edición 2026.11.15 Ejemplos verificados el 19 de nov de 2026 78 de 78 ejemplos pasan Qué ha cambiado

Traducido con IA. Verificado por personas en inglés. Cómo hacemos los libros

Traducido del libro en inglés, edición 2026.11.15. Lee el original

Esta parte aún no está traducida, así que está en inglés.

By the end of this chapter, you can give a model that writes SQL more than the schema, in layers: notes on columns whose names mislead, a glossary of the shop's business terms, and verified queries retrieved for each question. You can fit those layers to a token budget, measure what each one adds on the dev split of the golden questions, and check the best set once on the held-out test split.

A schema can't say what its columns mean. Chapter 1 gave a model only the shop's schema and asked for net revenue in each region in March 2026. The query it wrote ran, looked right and counted cancelled orders. The rule it missed exists, in the finance team's definition of net revenue, but nothing the model was given contains it. Layered context is the most direct fix for what a schema can't say: write the rules down where the model reads them. It isn't the fix this book ends with, since chapters 5 and 6 move the definitions into a semantic layer the model can't misread. But any agent that writes SQL needs context, including the guarded one in chapter 7, and the habit this chapter teaches, adding one layer at a time and measuring each, applies to every change that follows.

Qué dice el esquema y qué no dice

Último cambio en la edición 2026.11.12

Un esquema le indica a un modelo qué tablas existen, cómo se llaman sus columnas y qué tipo de dato guarda cada columna. Es mucho, y nunca queda desactualizado, porque lo produce la propia base de datos. Lo que no puede indicarle al modelo es qué significan las columnas: qué filas contiene una tabla, cuáles cuentan para un indicador de negocio, en qué zona horaria está una marca de tiempo y cuál de dos tablas parecidas es la que la gente considera confiable. Esos significados determinan si una consulta devuelve el número correcto, y viven en el SQL que construyó los marts y en la cabeza de las personas que las mantienen.

La medida del libro para esto son las preguntas doradas, que construyó el capítulo 3: preguntas con una respuesta verificada, evaluadas sobre la partición de desarrollo mientras se ajustan el contexto y los prompts, y una vez sobre la partición de prueba reservada. Cada capa de este capítulo se añade al esquema, así que el esquema es la primera capa, y el punto de partida es lo que deja abierto. build_context() en lib/context.py, el módulo compartido que construye cada capa de este capítulo, devuelve la capa de esquema cuando no se le pide nada más.

schema-digest.py Python 3.12.15 · duckdb 2.0.0 Superado
"""What the schema tells a model about revenue, and three March 2026 totals that all fit it. The three printed totalscarry the paragraph's point: they differ, and only the one that leaves out cancelled orders follows the finance definition."""from lib.context import build_contextfrom lib.warehouse import connectcon = connect()schema = build_context(con, "", layers=("schema",))      # no question, so only the schema layer is builtprint(f"tables in the schema: {schema.text.count('CREATE TABLE')}")print(f"schema tokens (estimated): {schema.tokens}")for line in schema.text.splitlines():    if "analytics.fct_orders " in line or "analytics.orders_legacy " in line:        print(line)march = "BETWEEN DATE '2026-03-01' AND DATE '2026-03-31'"   # the window of the question, on the business daycandidates = {    "orders_legacy.revenue": f"SELECT sum(revenue) FROM analytics.orders_legacy WHERE order_date {march}",    "fct_orders.net_revenue_eur, every order":        f"SELECT sum(net_revenue_eur) FROM analytics.fct_orders WHERE order_date_local {march}",    "fct_orders.net_revenue_eur, not cancelled":        f"SELECT sum(net_revenue_eur) FROM analytics.fct_orders WHERE order_date_local {march} "        "AND status <> 'cancelled'",          # net revenue leaves out cancelled orders, as the status note says}for name, sql in candidates.items():    (total,), = con.execute(sql).fetchall()    print(f"{name}: {total:,.2f}")
Salidapython schema-digest.py
tables in the schema: 16
schema tokens (estimated): 912
CREATE TABLE analytics.fct_orders (order_id INTEGER, customer_id INTEGER, region VARCHAR, country VARCHAR, channel VARCHAR, status VARCHAR, placed_at TIMESTAMP, order_date_local DATE, currency VARCHAR, line_count BIGINT, gross_revenue_eur DECIMAL(12,2), net_revenue_eur DECIMAL(12,2), discount_eur DECIMAL(12,2), tax_eur DECIMAL(12,2), _loaded_at TIMESTAMP);
CREATE TABLE analytics.orders_legacy (id INTEGER, cust_id INTEGER, order_date DATE, order_status VARCHAR, sales_region VARCHAR, revenue DECIMAL(12,2));
orders_legacy.revenue: 186,635.93
fct_orders.net_revenue_eur, every order: 162,542.79
fct_orders.net_revenue_eur, not cancelled: 150,082.83

La capa de esquema es el almacén completo: las 16 tablas, unos 912 tokens. Lee las dos líneas impresas como las leería un modelo. analytics.fct_orders tiene una columna net_revenue_eur y una columna status, y analytics.orders_legacy tiene una columna revenue y una columna order_date.

Cada uno de los tres totales que calcula el ejemplo para marzo de 2026 es una lectura razonable de esas dos líneas, y los tres se contradicen. La columna revenue de la tabla heredada da 186,635.93 euros, y la columna net_revenue_eur del mart sobre todos los pedidos da 162,542.79, que es lo que calculó el modelo del capítulo 1. Solo el tercero, 150,082.83, sigue la definición del equipo de finanzas, y nada en el esquema apunta a él en lugar de a los otros dos.

Todo el almacén de la tienda cabe en esos 912 tokens, así que no hay motivo para recortarlo: un estudio de modelos potentes sobre el benchmark BIRD encontró que pasar el esquema completo cuando cabe puede superar a seleccionar primero las tablas, porque el paso de selección puede descartar una columna que la consulta necesita (Maamari et al., 2024). Un almacén con miles de tablas es otro problema, que este capítulo no resuelve. El problema de la tienda no es qué tablas mostrar. Es que lo que muestran las tablas no dice cuál número es el correcto.

Regla práctica: Mantén el esquema completo en el contexto cuando quepa, y añádele significado en lugar de quitarle tablas. Antes de escribir cualquier otra capa, toma una pregunta real y enumera cada lectura del esquema que la responde. Cada lectura que el esquema no descarta es una regla que otra capa tiene que establecer.

Esta parte aún no está traducida, así que está en inglés.

Describe columns where names mislead

Último cambio en la edición 2026.11.15

The schema leaves open which of several readings is right, and the cheapest place to close some of them is the column itself. A column note is a sentence about one table or column that says what its name doesn't: which rows the table holds, the unit and sign of an amount, the time zone of a timestamp, the values a column takes, or that a table is deprecated. A note isn't documentation of every column. A column called product_name needs none, and a note that repeats the name costs tokens and says nothing. The notes worth writing are on the columns a careful analyst new to the warehouse would misread.

Published evidence points the same way. A study that generated column descriptions with a language model found that adding them to the prompt improved text-to-SQL accuracy and, in an ablation that replaced column names with uninformative ones, that the descriptions helped most there (Wretblad et al., 2024); it is a single preprint on the BIRD benchmark. Databricks' guidance for its Genie Agents says that "quality table and column descriptions in Unity Catalog are critical for Genie accuracy" (Databricks), which is the vendor's advice, not a measurement.

The shop's notes live in examples/data/shop/columns.toml, one entry per table with an optional note per column, and load_column_notes() in lib/context.py turns each into one line, such as analytics.fct_orders.placed_at: When the order was placed, in UTC. The notes cover the traps chapter 2 found: orders_legacy is deprecated, net_revenue_eur still holds a value for cancelled orders, placed_at is UTC while order_date_local is the business day, discount is negative in raw.order_lines and positive in the marts, and the fiscal year starts on February 1.

The recording context-layers shows what the notes do once the examples replay it. The recording holds the pinned model's SQL for every dev question of kind answer, written once with each layer set this chapter measures, using the prompt LAYERED_SQL in lib/prompts.py. The recording was made by recordings/record_context_layers.py, with the same model, sampling and seed as chapter 1's, and the examples replay it by the hash of the exact messages.

column-descriptions.py Python 3.12.15 · duckdb 2.0.0 · pyyaml 6.0.3 · sqlglot 30.31.0 Superado
"""Add the column notes on columns whose names mislead, and count the dev questions they fix and break under the recordedmodel. The `fixed` and `broken` lines settle the net effect of the notes, and the broken question shows how a note misleads."""from lib.context import LAYER_SETS, build_context, grade_with_context, load_column_notesfrom lib.golden import load_goldenfrom lib.model import RecordedModelfrom lib.warehouse import connectcon = connect()notes = load_column_notes()print(f"column notes: {len(notes)}")print(f"column notes tokens (estimated): {build_context(con, '', layers=('columns',)).tokens}")for note in notes:    if note.startswith(("analytics.fct_orders.status", "analytics.fct_orders.placed_at")):        print(note)model = RecordedModel("context-layers")before, after = LAYER_SETS[0], LAYER_SETS[1]          # the sets are cumulative: [0] is the schema alone, [1] adds the notesdev = [q for q in load_golden() if q.split == "dev" and q.kind == "answer"]fixed, broken = [], []for q in dev:    was, now = grade_with_context(con, model, q, before), grade_with_context(con, model, q, after)    if was != "correct" and now == "correct":          # fixed: wrong or error without the notes, correct with them        fixed.append(q.question_id)    if was == "correct" and now != "correct":        broken.append(q.question_id)print(f"fixed by column notes: {len(fixed)} ({', '.join(fixed)})")print(f"broken by column notes: {len(broken)} ({', '.join(broken)})")
Salidapython column-descriptions.py
column notes: 33
column notes tokens (estimated): 1087
analytics.fct_orders.status: One of placed, shipped, delivered, cancelled, returned. Revenue leaves out cancelled orders; order and customer counts include them. Returned orders count everywhere.
analytics.fct_orders.placed_at: When the order was placed, in UTC. Don't filter or group dates on it; use order_date_local.
fixed by column notes: 7 (gq-001, gq-002, gq-003, gq-004, gq-008, gq-009, gq-011)
broken by column notes: 1 (gq-018)

The 33 notes add about 1087 tokens, more than the schema itself. On the dev questions they fix 7 (gq-001, gq-002, gq-003, gq-004, gq-008, gq-009, gq-011), among them gq-002, the March question from chapter 1, whose query now filters out cancelled orders. They also break 1 (gq-018). That question asks for the refunds on orders placed in March 2026, and with the notes the model filtered on both the order's date and refund_month, so it dropped refunds paid in a later month. The note on refund_month was accurate. The model used it on a question that didn't call for it, which is what a note can't prevent.

Working rule: Write notes only for tables and columns whose names mislead: deprecated tables, amounts with a sign or unit you wouldn't guess, timestamps in a time zone other than the business's, and columns whose values include a placeholder such as (none). Write them as facts about the column, not as instructions for one question. Expect a note to fix many questions and occasionally mislead on one, and keep the questions it broke in the golden set so you see it happen.

Esta parte aún no está traducida, así que está en inglés.

Add the business glossary

Último cambio en la edición 2026.11.15

Column notes describe the tables. Many questions turn on something else: what a business word means at this company. "Revenue" could be gross or net. "Orders" could include cancelled ones or not. Refunds "in March" could mean refunds paid in March or refunds on March's orders. These are choices, made by the teams that own the numbers, and no table states them. A business glossary lists the business words whose meaning a company's teams chose, and writes each meaning in terms of the warehouse's tables and columns.

Evidence for this layer comes from BIRD, a text-to-SQL benchmark whose questions come with "external knowledge evidence": hints that link a question's words to the database, such as what a term means as a formula over columns. In the benchmark's paper, adding the evidence raised GPT-4's execution accuracy on the dev set from 30.90% to 46.35%, with 2023 models and zero-shot prompts (Li et al., 2023). The hints are written per question, though, which is closer to giving the model the answer's ingredients than to giving it a glossary written once.

The shop's glossary is examples/data/shop/glossary.toml. Its entries define revenue as net revenue unless the question says gross, orders as every order including cancelled ones, average order value, fiscal periods, new customers, refunds by the month they were paid, units sold, and the two meanings of "channel". load_glossary() turns each into one line.

The glossary wasn't written blind, and the book's history shows how it changed. In the same change that recorded the model's first dev replies, with the schema alone, the entry for orders was corrected: it had said that order counts leave out cancelled orders, which disagreed with the finance team's golden SQL. The entries for units sold and for customers who ordered were added in that change too, and the column note on status was reworded to match. Later, after the test questions had been recorded with the schema alone, the refund-rate entry was changed to the book-wide definition that chapter 5 uses and that test question gq-028 also follows: refunded amount divided by gross revenue, because a refund pays back tax and only gross revenue includes it. Each change made the glossary agree with golden SQL that it was then scored against. That is how a dev score gets fitted to dev, and the check on test comes back to what it costs.

glossary-context.py Python 3.12.15 · duckdb 2.0.0 · pyyaml 6.0.3 · sqlglot 30.31.0 Superado
"""Add the business glossary on top of the column notes, and count what it changes on the dev questions. The gq-002lines show the query the prose describes: with the glossary, the March query no longer filters on status."""from lib.context import LAYER_SETS, build_context, grade_with_context, load_glossaryfrom lib.golden import load_goldenfrom lib.model import RecordedModelfrom lib.prompts import layered_sql_messagesfrom lib.warehouse import connectcon = connect()terms = load_glossary()print(f"glossary terms: {len(terms)}")print(f"glossary tokens (estimated): {build_context(con, '', layers=('glossary',)).tokens}")print(next(t for t in terms if t.startswith("revenue:")))model = RecordedModel("context-layers")before, after = LAYER_SETS[1], LAYER_SETS[2]          # schema and column notes; then the glossary toodev = [q for q in load_golden() if q.split == "dev" and q.kind == "answer"]changes = {(was, now): [] for was in ("correct", "other") for now in ("correct", "other")}   # "other" is wrong or errorfor q in dev:    was, now = (grade_with_context(con, model, q, layers) for layers in (before, after))    changes["correct" if was == "correct" else "other", "correct" if now == "correct" else "other"].append(q.question_id)print(f"fixed by the glossary: {len(changes['other', 'correct'])} ({', '.join(changes['other', 'correct'])})")print(f"broken by the glossary: {len(changes['correct', 'other'])} ({', '.join(changes['correct', 'other'])})")q = next(q for q in dev if q.question_id == "gq-002")reply = model.complete(layered_sql_messages(build_context(con, q.question, after).text, q.question))filters_status = "status" in reply     # a query that filters on status has to name the columnassert not filters_status              # the chapter states it: with the glossary, gq-002 no longer filters on statusprint(f"gq-002 with the glossary filters status: {'yes' if filters_status else 'no'}")
Salidapython glossary-context.py
glossary terms: 13
glossary tokens (estimated): 609
revenue: Net revenue unless the question says gross: sum of analytics.fct_orders.net_revenue_eur over orders whose status isn't 'cancelled'. In EUR.
fixed by the glossary: 3 (gq-012, gq-015, gq-018)
broken by the glossary: 3 (gq-002, gq-009, gq-016)
gq-002 with the glossary filters status: no

The glossary's gains and losses cancel out: it fixes 3 (gq-012, gq-015, gq-018) dev questions and breaks 3 (gq-002, gq-009, gq-016). Its 13 terms add about 609 tokens on top of the schema and the column notes. The broken ones are the instructive part. For gq-002, the same March question, the query written with the glossary no longer filters on status, although the revenue entry and the note on status both say to leave cancelled orders out. For gq-016, the model left cancelled orders out of a customer count, applying the revenue rule where the glossary says it doesn't apply. For gq-009, the average order value, it divided the average order's net revenue by the number of orders again, although the glossary defines the measure as net revenue divided by the number of orders.

Each configuration was recorded once, with one sampled reply per question, so a change of one or two questions out of 17 is within what a second sample could undo. Chapter 10 measures that spread with repeated runs. What this result supports is narrower: the glossary didn't make this model better on these questions, and a rule stated twice was still missed. A plausible reason, which we haven't measured here, is that a longer context makes a small model use each part of it less reliably. Studies of long contexts find that models use information in the middle of a prompt worse than at its edges (Liu et al., 2024), and that accuracy falls with length when the question and the passage don't share words (Modarressi et al., 2025).

Working rule: Write a glossary entry for each business word whose meaning is a choice someone made, and state it in terms of tables and columns. Prefer one place for each rule, a column note or a glossary entry; the shop's files state the cancelled-orders rule in both, and the miss on gq-002 shows that saying it twice doesn't make it stick. Measure the glossary on your dev questions before you trust it, and keep it only if it helps there or the next layer depends on it.

Esta parte aún no está traducida, así que está en inglés.

Retrieve verified queries as examples

Último cambio en la edición 2026.11.15

Column notes and a glossary state rules. A verified query shows the shop's rules applied: a question someone asked, with SQL a person checked against the shop's definitions. Given a few such pairs that resemble the question, a model can copy their shape, such as the join to dim_date for a fiscal period or the filter on status, instead of inventing one. A systematic comparison of prompt designs for text-to-SQL found that the best setup chose examples by the similarity of their questions, after masking domain-specific words, and by the similarity of their SQL to a draft query written first, and showed each as a question and its SQL alone (Gao et al., 2023). Warehouse vendors build the same idea into their assistants. Snowflake's Cortex Analyst keeps a verified query repository of "questions and corresponding SQL queries to answer them" (Snowflake), and Databricks says a Genie Agent can use an example SQL query directly or learn from it, and treats parameterized ones as trusted assets (Databricks). Both are the vendors' own descriptions, not measurements.

The shop's verified queries live in examples/data/shop/verified_queries.toml. Their fields follow Snowflake's field names, name, question, sql, verified_at and verified_by, plus the role the query was checked for. Two differ in content. Snowflake's verified_at is optional and counted in seconds since 1970, and its verified_by names the person who checked the query; the shop's files give a date and name a team, never a person. None of the verified queries is a golden question. They are near neighbours of questions in both splits, such as net revenue by fiscal quarter for fiscal 2025 or average order value by region for 2024, which is how a real repository grows: from questions people asked before. A verified query that was itself a golden question would make the score measure copying.

Retrieval in lib/context.py is lexical, with no embeddings. question_terms() lowercases the question, drops common words and masks three kinds of word: numbers, month names and the shop's three region codes. Every other word stays as it is. QueryIndex scores each verified question against the masked question with TF-IDF and cosine similarity, and search() keeps at most three matches that score at least 0.4. This is a cheap stand-in for the selection in Gao et al., not a reproduction of it: it masks far fewer words, and it drops the second step, which compares draft SQL.

retrieve-similar-queries.py Python 3.12.15 · duckdb 2.0.0 Superado
"""Retrieve the verified queries most like a question, by TF-IDF over the masked question's words. The matches kept at the0.4 default show that a threshold can leave a question with no example, and the masked phrases show what masking does."""from lib.context import QueryIndex, load_verified_queries, question_termsqueries = load_verified_queries()index = QueryIndex(queries)print(f"verified queries: {len(queries)}")for label, question in (("aov", "What was the average order value in each region in March 2026?"),                        ("parcels", "Which warehouse ships the most parcels?")):    print(f"question: {question}")    print(f"masked terms: {' '.join(question_terms(question))}")    for score, query in index.search(question, k=3, min_score=0.0):     # no threshold here, so the scores show the gap        print(f"  {score:.3f}  {query.name}")    print(f"{label} matches kept: {len(index.search(question))}")         # the default threshold, 0.4, is the one the book uses# Months and numbers are masked; "calendar" and "year" are not common words, and "Q3" is not masked at all.for phrase in ("March 2026", "January 2025", "calendar year 2024", "EMEA in Q3"):    print(f"masked \"{phrase}\": {' '.join(question_terms(phrase))}")
Salidapython retrieve-similar-queries.py
verified queries: 16
question: What was the average order value in each region in March 2026?
masked terms: average order value region <month> <number>
  0.835  average_order_value_by_region
  0.422  gross_revenue_by_region_month
  0.191  orders_of_one_customer
aov matches kept: 2
question: Which warehouse ships the most parcels?
masked terms: warehouse ships most parcels
  0.375  units_sold_by_product
  0.357  net_revenue_by_category
  0.000  average_order_value_by_region
parcels matches kept: 0
masked "March 2026": <month> <number>
masked "January 2025": <month> <number>
masked "calendar year 2024": calendar year <number>
masked "EMEA in Q3": <region> q3

For the average order value in each region in March 2026, the closest of the 16 verified queries is the same measure by region for 2024, and 2 matches pass the threshold. The second question, about which warehouse ships the most parcels, is one the shop's data can't answer at all. Its best scores come from the single shared word "most", and none passes the threshold, so 0 queries are offered. Without the threshold, the model would get three confident-looking examples, two of them about products, for a question about shipping.

The last lines of the output show what masking does and doesn't do. "March 2026" and "January 2025" both become <month> <number>, so a question about one month matches a verified query about another. "Calendar year 2024" becomes calendar year <number>, because "calendar" and "year" are neither common words nor values, and a quarter written as "Q3" isn't masked at all. That is why the March question's best match, a question about calendar year 2024, scores less than a perfect 1.

Working rule: Retrieve a few verified queries per question, chosen by similarity of the masked question, and show each as its question and SQL. Set a minimum score and check it on questions the warehouse can't answer, because lexical retrieval always returns something. Never let a golden question into the verified set. Move to embeddings only when your own measurement shows lexical retrieval missing queries that a person would call similar.

Ajusta el contexto a un presupuesto

Último cambio en la edición 2026.11.12

Con el esquema, las notas de columnas, el glosario y las consultas verificadas recuperadas, el contexto de la tienda ya ocupa varias veces más tokens que el esquema por sí solo, y cada capa que añades lo alarga. Un presupuesto de tokens es la cantidad máxima de contexto que el agente puede enviar con una pregunta. Existe por tres razones. La ventana del modelo es finita: el entorno de ejecución con el que se hicieron las grabaciones del libro le da al modelo 8,192 tokens, y hasta 384 de ellos se reservan para la respuesta. Cada token del prompt cuesta tiempo en una CPU y dinero con un modelo alojado. Y un contexto más largo no se aprovecha de manera uniforme, ya que los modelos hacen menos uso de la información situada en medio de un prompt largo que de la que aparece al principio o al final (Liu et al., 2024).

Ajustar un contexto a un presupuesto significa decidir qué dejar fuera, y esa decisión necesita un orden de prioridad. build_context() recorre las capas en orden de prioridad, añade sus elementos uno a uno y descarta cualquier elemento que llevaría el contexto por encima del presupuesto, de modo que un elemento posterior y más corto aún puede caber. El esquema nunca se descarta, por la razón que expuso la primera sección: un esquema que cabe debe enviarse completo. Sea cual sea la prioridad, el contexto se genera en un orden fijo: primero las capas estables, después las consultas recuperadas y, por último, la pregunta. Así la pregunta queda junto a los ejemplos más parecidos a ella, y el entorno que ejecuta el modelo puede reutilizar la parte del prompt que comparten todas las preguntas, en lugar de volver a leerla en cada llamada. El entorno que produjo las grabaciones del libro, llama-cpp-python 0.3.36, hace esto en Llama.generate: compara los tokens del nuevo prompt con los del anterior y evalúa solo los tokens posteriores al prefijo más largo que comparten (llama-cpp-python 0.3.36, llama.py).

Los recuentos de tokens en lib/context.py son estimaciones: estimate_tokens() divide el número de caracteres entre 3.5, porque los ejemplos no cargan el tokenizador del modelo. A continuación, esa estimación se compara con el recuento que informó el tokenizador del modelo cuando se registró el prompt, y el contexto de la pregunta sobre el valor medio del pedido, gq-009, se ajusta a un presupuesto ilustrativo de 2,400 tokens con dos prioridades.

context-token-budget.py Python 3.12.15 · duckdb 2.0.0 · pyyaml 6.0.3 · sqlglot 30.31.0 Superado
"""Fit one question's layered context to a token budget, and check the token estimate against the model's count. Theestimate runs high, the safe side for a budget, and the dropped lines show what each priority gives up."""from lib.context import LAYERS, build_context, estimate_tokensfrom lib.golden import load_goldenfrom lib.model import RecordedModelfrom lib.prompts import layered_sql_messagesfrom lib.warehouse import connectcon = connect()question = next(q for q in load_golden() if q.question_id == "gq-009").questionfull = build_context(con, question)for layer, tokens in full.layer_tokens().items():    print(f"{layer} tokens (estimated): {tokens}")messages = layered_sql_messages(full.text, question)call = RecordedModel("context-layers").call_for(messages)print(f"prompt tokens (estimated): {estimate_tokens(''.join(m['content'] for m in messages))}")   # every message, joinedprint(f"prompt tokens (model's tokenizer): {call['prompt_tokens']}")budget = 2400          # an illustrative budget, well inside the 8,192-token window the recordings usefor name, priority in (("default", LAYERS), ("examples first", ("schema", "columns", "verified", "glossary"))):   # verified ranks above glossary    fitted = build_context(con, question, layers=priority, budget=budget)    kept = {layer: len(items) for layer, items in fitted.items.items()}    print(f"{name} priority: {fitted.tokens} tokens, kept {kept}")    print(f"{name} dropped: {len(fitted.dropped)}")print(f"render order: {', '.join(layer for layer, items in fitted.items.items() if items)}, then the question")
Salidapython context-token-budget.py
schema tokens (estimated): 912
columns tokens (estimated): 1087
glossary tokens (estimated): 609
verified tokens (estimated): 200
prompt tokens (estimated): 2917
prompt tokens (model's tokenizer): 2364
default priority: 2393 tokens, kept {'schema': 1, 'columns': 33, 'glossary': 9, 'verified': 0}
default dropped: 6
examples first priority: 2384 tokens, kept {'schema': 1, 'columns': 33, 'glossary': 5, 'verified': 2}
examples first dropped: 8
render order: schema, columns, glossary, verified, then the question

La estimación para el prompt completo es de 2917 tokens, frente a los 2364 contados por el tokenizador del modelo. Una estimación que queda por encima del valor real es el lado seguro para un presupuesto: el contexto admitido siempre cabe. Con la prioridad por defecto, que sigue el orden en que se introdujeron las capas, el presupuesto conserva todas las notas de columnas y parte del glosario, y descarta las dos consultas recuperadas, 6 elementos en total. Si las consultas verificadas se colocan por encima del glosario, el mismo presupuesto conserva las dos consultas y descarta 8 entradas del glosario. Ninguno de los dos órdenes es correcto en general. La siguiente sección mide qué capas se ganan sus tokens en las preguntas de la tienda, y eso es lo que debería determinar el orden.

Regla práctica: Fija un presupuesto de tokens bien por debajo de la ventana del modelo, y ajusta el contexto de cada pregunta a él según una prioridad que hayas medido, no según el orden en que escribiste las capas. Nunca descartes el esquema cuando quepa. Genera primero las capas estables y la pregunta al final, sea lo que sea lo que se haya descartado. Compara tu estimación de tokens con el recuento real del modelo una vez, y mantenla por el lado alto.

Esta parte aún no está traducida, así que está en inglés.

Measure what each layer adds

Último cambio en la edición 2026.11.15

Each layer so far was judged by the questions it fixed and broke. To choose which layers to keep, and in what order a budget should drop them, you need all of them measured the same way. An ablation measures every layer the same way: it scores a series of configurations that differ by one layer each, on the same questions, under the same comparison rule. The questions are the dev split of the golden questions, because choosing among configurations is tuning, and tuning on the test split would leave nothing held out to check the choice.

The shop's dev split has 24 golden questions. This chapter scores the 17 of kind answer. The other 7 should be declined, because the warehouse can't answer them or the role asking may not see the answer, and a prompt that only writes SQL can't decline, so they would score as wrong under every layer set; chapters 8 and 9 score them. Each layer set adds one layer to the one before: the schema alone, then column notes, then the glossary, then up to three retrieved verified queries. The outcomes come from grade() in lib/golden.py, as in every chapter: correct when the result matches the golden result, wrong when it runs and doesn't, error when it doesn't run. The first layer set, the schema alone, isn't chapter 3's baseline either. That baseline used the schema-only prompt and scored every question in each split, while this run uses the layered prompt LAYERED_SQL with no other layer filled in and scores the answer questions only, so its numbers differ.

layer-ablation-dev.py Python 3.12.15 · duckdb 2.0.0 · pyyaml 6.0.3 · sqlglot 30.31.0 Superado
"""Score each layer set on the dev split's answer questions, replaying the recorded model's SQL for each, and print thetotals, the average token counts and the outcome grid. The correct totals and the fixed and broken lines are the numbers the prose rests on."""from collections import Counterfrom lib.context import LAYER_SETS, build_context, grade_with_context, layer_set_namefrom lib.golden import load_goldenfrom lib.model import RecordedModelfrom lib.warehouse import connectcon = connect()model = RecordedModel("context-layers")dev_split = [q for q in load_golden() if q.split == "dev"]dev = [q for q in dev_split if q.kind == "answer"]          # the other questions are declined, and are scored in chapters 8 and 9print(f"dev split questions: {len(dev_split)}")print(f"dev questions: {len(dev)}")print(f"dev questions to decline: {len(dev_split) - len(dev)}")grid = {}for layers in LAYER_SETS:    name = layer_set_name(layers)    grid[name] = [grade_with_context(con, model, q, layers) for q in dev]    counts = Counter(grid[name])    tokens = sum(build_context(con, q.question, layers).tokens for q in dev) // len(dev)   # mean over the dev questions, whole tokens    print(f"{name}: correct {counts['correct']}, wrong {counts['wrong']}, error {counts['error']}")    print(f"{name} correct: {counts['correct']}")    print(f"{name} error: {counts['error']}")    print(f"{name} tokens: {tokens}")print("question  " + "  ".join(f"{i + 1}" for i in range(len(LAYER_SETS))))for i, q in enumerate(dev):    print(f"{q.question_id}    " + "  ".join(grid[name][i][0] for name in grid))# What the verified queries changed, against the run with the glossary.before, after = grid[layer_set_name(LAYER_SETS[2])], grid[layer_set_name(LAYER_SETS[3])]   # the glossary run, then the glossary plus verified queriesfixed = [q.question_id for q, was, now in zip(dev, before, after) if was != "correct" and now == "correct"]broken = [q.question_id for q, was, now in zip(dev, before, after) if was == "correct" and now != "correct"]print(f"fixed by verified queries: {len(fixed)}")print(f"fixed by verified queries, IDs: {', '.join(fixed)}")print(f"broken by verified queries: {len(broken)}")
Salidapython layer-ablation-dev.py
dev split questions: 24
dev questions: 17
dev questions to decline: 7
schema: correct 3, wrong 9, error 5
schema correct: 3
schema error: 5
schema tokens: 912
schema+columns: correct 9, wrong 7, error 1
schema+columns correct: 9
schema+columns error: 1
schema+columns tokens: 1999
schema+columns+glossary: correct 9, wrong 8, error 0
schema+columns+glossary correct: 9
schema+columns+glossary error: 0
schema+columns+glossary tokens: 2608
schema+columns+glossary+verified: correct 14, wrong 2, error 1
schema+columns+glossary+verified correct: 14
schema+columns+glossary+verified error: 1
schema+columns+glossary+verified tokens: 2775
question  1  2  3  4
gq-001    e  c  c  c
gq-002    w  c  w  c
gq-003    e  c  c  c
gq-004    w  c  c  c
gq-006    e  e  w  w
gq-007    w  w  w  w
gq-008    w  c  c  c
gq-009    w  c  w  c
gq-010    e  w  w  c
gq-011    w  c  c  c
gq-012    w  w  c  c
gq-013    e  w  w  c
gq-014    c  c  c  c
gq-015    w  w  c  c
gq-016    c  c  w  c
gq-017    w  w  w  e
gq-018    c  w  c  c
fixed by verified queries: 5
fixed by verified queries, IDs: gq-002, gq-009, gq-010, gq-013, gq-016
broken by verified queries: 0

Column notes and verified queries raise the score; the glossary doesn't. The schema alone gets 3 of the dev questions right, column notes take it to 9, and the glossary leaves it at 9. Verified queries on top give 14, the best set on dev.

The grid shows where the change came from. Against the run with the glossary, retrieved examples fixed 5 questions and broke 0. Two of the fixes are new: the fiscal-quarter question gq-010 and the order count by region gq-013, whose nearest verified queries show exactly the join and the count they needed. The other three, gq-002, gq-009 and gq-016, are the questions the glossary run had lost, so there the examples won back what the glossary had cost.

The totals say how much each layer helped. The failures say what the next layer should be, so the next two examples read them, starting with the schema-only queries that don't run.

missing-columns-schema-only.py Python 3.12.15 · duckdb 2.0.0 · pyyaml 6.0.3 · sqlglot 30.31.0 Superado
"""Why the dev queries written with the schema alone don't run: the column each one names, and the analytics tables that have it.The per-table counts are the evidence the prose rests on: every failure is a column on the wrong table."""import refrom collections import Counterimport duckdbfrom lib.context import LAYER_SETS, build_contextfrom lib.golden import load_goldenfrom lib.model import RecordedModelfrom lib.prompts import layered_sql_messagesfrom lib.tools import extract_sql, run_sqlfrom lib.warehouse import connectcon, model = connect(), RecordedModel("context-layers")dev = [q for q in load_golden() if q.split == "dev" and q.kind == "answer"]where = Counter()for q in dev:    context = build_context(con, q.question, LAYER_SETS[0])          # the schema alone    sql = extract_sql(model.complete(layered_sql_messages(context.text, q.question)))    try:        run_sql(con, sql)        continue    except duckdb.Error as e:        column = re.search(r'column (?:named )?"(\w+)"', str(e)).group(1)   # DuckDB says either form; take the name    (tables,), = con.execute(        "SELECT string_agg(table_schema || '.' || table_name, ', ' ORDER BY 1) FROM information_schema.columns "   # sorted, so a column in two tables lists them in a fixed order        "WHERE table_schema = 'analytics' AND column_name = ?", [column]).fetchall()   # the marts only: where the model should have looked    where[tables] += 1    print(f"{q.question_id}: no column {column}; it is in {tables}")print(f"schema-only queries that don't run: {where.total()}")for tables, n in where.items():    print(f"missing column only {tables} has: {n}")
Salidapython missing-columns-schema-only.py
gq-001: no column fiscal_year; it is in analytics.dim_date
gq-003: no column calendar_year; it is in analytics.dim_date
gq-006: no column quantity; it is in analytics.fct_order_lines
gq-010: no column fiscal_quarter; it is in analytics.dim_date
gq-013: no column order_date; it is in analytics.orders_legacy
schema-only queries that don't run: 5
missing column only analytics.dim_date has: 3
missing column only analytics.fct_order_lines has: 1
missing column only analytics.orders_legacy has: 1

Every failure is a column the model put on the wrong table. Of the 5, 3 ask the orders mart for a fiscal or calendar column that only dim_date has, 1 asks it for quantity, which only the order lines have, and 1 uses the legacy table's order_date.

These are the mistakes column notes are for. With the notes, 1 query in the whole dev split fails to run.

A second script prints the query for each dev question that all four layers still don't get right.

what-stays-wrong.py Python 3.12.15 · duckdb 2.0.0 · pyyaml 6.0.3 · sqlglot 30.31.0 Superado
"""The dev questions that stay wrong with all four layers, each with its query, and a check of whether gq-007 fails only onthe spelling of 'tents'. The count line settles how many dev questions stay wrong, and the gq-007 line settles whether the spelling alone causes its failure."""import duckdbfrom lib.context import LAYERS, build_context, grade_with_contextfrom lib.golden import golden_result, grade, load_goldenfrom lib.model import RecordedModelfrom lib.prompts import layered_sql_messagesfrom lib.tools import extract_sql, run_sqlfrom lib.warehouse import connectcon, model = connect(), RecordedModel("context-layers")dev = [q for q in load_golden() if q.split == "dev" and q.kind == "answer"]def sql_with_all_layers(q):    return extract_sql(model.complete(layered_sql_messages(build_context(con, q.question, LAYERS).text, q.question)))unsolved = [(q, outcome) for q in dev if (outcome := grade_with_context(con, model, q, LAYERS)) != "correct"]   # LAYERS is all fourprint(f"not correct with all four layers: {len(unsolved)}")for q, outcome in unsolved:    sql = sql_with_all_layers(q)    print(f"{q.question_id} ({outcome}): {q.question}")    print("\n".join("    " + line for line in sql.strip().splitlines()))    if outcome == "error":        try:            run_sql(con, sql)        except duckdb.Error as e:            print(f"    -- {str(e).splitlines()[0]}")gq007 = next(q for q in dev if q.question_id == "gq-007")respelled = sql_with_all_layers(gq007).replace("'tents'", "'Tents'")   # change the spelling only, to see whether it is the whole failureprint(f"gq-007 with 'Tents' spelled as stored: {grade(gq007, run_sql(con, respelled), golden_result(con, gq007))}")
Salidapython what-stays-wrong.py
not correct with all four layers: 3
gq-006 (wrong): What were net revenue and the number of units sold in each region in 2025?
    SELECT
        o.region,
        SUM(o.net_revenue_eur) AS net_revenue,
        SUM(l.quantity) AS units_sold
    FROM analytics.fct_orders AS o
    JOIN analytics.fct_order_lines AS l USING (order_id)
    WHERE o.order_date_local >= '2025-01-01' AND o.order_date_local < '2026-01-01'
        AND o.status <> 'cancelled'
    GROUP BY o.region
    ORDER BY net_revenue DESC
gq-007 (wrong): How much net revenue came from orders that included a tent in 2025?
    SELECT
        SUM(fol.net_amount_eur) AS net_revenue
    FROM
        analytics.fct_order_lines fol
    JOIN
        analytics.fct_orders fo ON fol.order_id = fo.order_id
    JOIN
        analytics.dim_products dp ON fol.product_id = dp.product_id
    JOIN
        analytics.dim_date dd ON fo.order_date_local = dd.date_day
    WHERE
        dp.category = 'tents'
        AND dd.calendar_year = 2025
        AND fo.status != 'cancelled'
gq-017 (error): What was net revenue in each of our countries in 2025?
    SELECT d.country, round(sum(o.net_revenue_eur), 2) AS net_revenue
    FROM analytics.fct_orders AS o
    JOIN analytics.dim_date AS d ON d.date_day = o.order_date_local
    WHERE o.status <> 'cancelled'
      AND d.calendar_year = 2025
    GROUP BY d.country
    ORDER BY net_revenue DESC
    -- Binder Error: Table "d" does not have a column named "country"
gq-007 with 'Tents' spelled as stored: wrong

With all four layers, 3 questions are still not right. gq-006 asks for net revenue and units sold by region. The model joined the orders mart to its lines and summed net_revenue_eur over the joined rows, so an order's revenue counts once for every line it has. The glossary's entry for units sold says to sum the lines per order first for exactly this reason, so this is a rule the context stated and the model missed. gq-007 filters on the category 'tents' where the data says 'Tents', a value no layer lists. Spelled as stored, the answer is still wrong, because the query sums only the tent lines, where the question asks for the whole revenue of the orders that included one. gq-017 asks for "our countries", which depends on the region of the regional lead asking. No layer says who is asking, and the model's query, which takes country from dim_date, doesn't run.

This is the book's measurement on one synthetic warehouse and one 4-billion-parameter model, with one sampled reply per question, not a general result about layered context. We found no peer-reviewed study from 2025 or 2026 that measures retrieved verified queries separately from other context on business data, which is why the chapter measures it here.

Working rule: Measure context one layer at a time on the dev split, with the same questions, comparison rule and model, and keep the outcomes per question, not only the totals. Treat a difference of one or two questions on a set this size as noise until repeated runs say otherwise. Read the queries that fail, because each one names a layer you don't have yet or a rule the model still skips.

Verificar las capas elegidas en el conjunto de prueba

Último cambio en la edición 2026.11.12

La ablación eligió las cuatro capas en el conjunto de desarrollo (dev), las preguntas con las que el capítulo afinó. El conjunto que obtuvo la mejor puntuación allí aún debe verificarse una vez en el conjunto de prueba (test), que está reservado, junto con el esquema por sí solo, para que la comparación no quede favorecida por haber sido elegida con las mismas preguntas. El registro context-layers-test contiene el SQL del modelo para las 17 preguntas de prueba del tipo answer, solo con esos dos conjuntos de capas. Cada pregunta de prueba también se compara con su consulta verificada más parecida, para los párrafos que siguen.

best-layers-on-test.py Python 3.12.15 · duckdb 2.0.0 · pyyaml 6.0.3 · sqlglot 30.31.0 Superado
"""Choose the best layer set on the dev split, then score it once on the held-out test split beside the schema alone. Thecorrect counts are the numbers the prose rests on, and the retrieval scores measure how close each split sits to the verified queries."""from collections import Counterfrom lib.context import LAYER_SETS, QueryIndex, grade_with_context, layer_set_name, load_verified_queriesfrom lib.golden import load_goldenfrom lib.model import RecordedModelfrom lib.warehouse import connectcon = connect()questions = [q for q in load_golden() if q.kind == "answer"]   # the answer questions only, as in the ablationdev = [q for q in questions if q.split == "dev"]test = [q for q in questions if q.split == "test"]dev_model = RecordedModel("context-layers")          # dev and test replies come from separate recordings; the test one plays no part in the choicedev_correct = {layers: sum(grade_with_context(con, dev_model, q, layers) == "correct" for q in dev)               for layers in LAYER_SETS}best = max(LAYER_SETS, key=lambda layers: (dev_correct[layers], -len(layers)))   # ties go to fewer layersprint(f"best on dev: {layer_set_name(best)}")print(f"best on dev correct: {dev_correct[best]} of {len(dev)}")test_model = RecordedModel("context-layers-test")print(f"test questions: {len(test)}")print(f"test question IDs: {', '.join(q.question_id for q in test)}")for name, layers in (("schema", LAYER_SETS[0]), ("best", best)):    counts = Counter(grade_with_context(con, test_model, q, layers) for q in test)    print(f"test {name}: correct {counts['correct']}, wrong {counts['wrong']}, error {counts['error']}")    print(f"test {name} correct: {counts['correct']}")# How close each split's questions sit to the verified queries, by the retriever's own score.index = QueryIndex(load_verified_queries())for split, qs in (("dev", dev), ("test", test)):    best_match = [(index.search(q.question, k=1, min_score=0.0)[0], q) for q in qs]   # the best match, whatever its score    print(f"{split} mean top retrieval score: {sum(score for (score, _), _ in best_match) / len(qs):.2f}")    print(f"{split} questions with no verified query kept: {sum(not index.search(q.question) for q in qs)}")    (score, query), q = max(best_match, key=lambda pair: pair[0][0])    print(f"{split} closest pair: {q.question_id} and {query.name}")    print(f"{split} closest score: {score:.3f}")
Salidapython best-layers-on-test.py
best on dev: schema+columns+glossary+verified
best on dev correct: 14 of 17
test questions: 17
test question IDs: gq-025, gq-026, gq-027, gq-028, gq-029, gq-030, gq-031, gq-032, gq-033, gq-034, gq-035, gq-036, gq-037, gq-038, gq-039, gq-040, gq-041
test schema: correct 4, wrong 9, error 4
test schema correct: 4
test best: correct 8, wrong 7, error 2
test best correct: 8
dev mean top retrieval score: 0.57
dev questions with no verified query kept: 2
dev closest pair: gq-001 and net_revenue_by_fiscal_quarter
dev closest score: 0.862
test mean top retrieval score: 0.63
test questions with no verified query kept: 1
test closest pair: gq-029 and new_customers_by_region
test closest score: 0.933

Las capas siguen ayudando en preguntas con las que nadie las afinó, que es el resultado que cuenta. El mejor conjunto en dev, schema+columns+glossary+verified, obtuvo 14 of 17 aciertos en dev. En test, el esquema por sí solo obtiene 4 aciertos y las cuatro capas obtienen 8.

La diferencia entre las puntuaciones de dev y de test es el otro resultado, y este capítulo no midió su causa. Una explicación es que las consultas verificadas se ajustan mejor a dev, y las propias puntuaciones del recuperador la contradicen: la puntuación media de la mejor coincidencia de cada pregunta es 0.57 en dev y 0.63 en test.

El par más cercano de todos, una pregunta de referencia y su coincidencia verificada, también está en test: gq-029 and new_customers_by_region, con 0.933, la misma pregunta planteada sobre el año natural 2024. El más cercano en dev puntúa 0.862.

Hay dos cosas conocidas. Primero, dev se usó para algo más que elegir el conjunto. Como decía la sección del glosario, el glosario se editó para que coincidiera con el SQL de referencia mientras se registraban las primeras respuestas en dev, y una edición así ajusta el contexto a las preguntas con las que se va a puntuar, lo quiera alguien o no. Así es como dev se sobreajusta, y por qué una puntuación en dev no es el resultado que debe informarse. Segundo, los conjuntos preguntan cosas distintas: el conjunto de prueba pregunta por el margen bruto, las sesiones y el gasto en marketing, que ninguna pregunta de dev toca. Distinguir esas causas requeriría más preguntas y ejecuciones repetidas de las que tiene este capítulo.

Regla práctica: Elige el conjunto en dev y luego puntúalo una sola vez en el conjunto de prueba junto a la línea base, e informa el número de prueba. Anota cada cambio que hayas hecho en el contexto después de leer un resultado de dev, porque cada uno es una forma en que la puntuación de dev puede exagerar. Cuando dev y test no coincidan, comprueba lo que puedas medir, como la cercanía de cada conjunto a tus ejemplos, antes de explicar la diferencia.

Qué sale mal en producción

Último cambio en la edición 2026.11.12

Las capas elevaron la puntuación del entorno de desarrollo muy por encima de lo que logra el esquema por sí solo, y solo siguen siendo correctas mientras el almacén de datos y sus definiciones no cambien. Una vez que la gente depende de ellas, los fallos son silenciosos: una nota o una consulta verificada sigue enseñando una regla que su almacén de datos ya ha cambiado.

Las notas se desvían de los data marts. Una nota de columna es verdadera sobre el data mart tal como estaba construido cuando alguien la escribió. Cuando los ingenieros de analítica cambian fct_orders, por ejemplo para excluir del data mart los pedidos cancelados, la nota que indica filtrar por status queda obsoleta, y nada falla. Lo detectas si mantienes las notas en el mismo repositorio que el SQL que construye los data marts, las revisas en las mismas solicitudes de cambio y compruebas en CI que cada tabla y columna que menciona una nota todavía existe.

Un valor que el contexto nunca enumera. El modelo escribió 'tents' para una categoría almacenada como 'Tents', y la consulta se ejecutó sin error pero no devolvió nada. Las preguntas nombran valores, como categorías, canales, regiones y nombres de campañas, y un modelo adivina su ortografía. Lo detectas al leer las respuestas incorrectas cuyo resultado está vacío o es cero. Para las columnas con unas pocas decenas de valores distintos, enumera esos valores en la nota de la columna. Para las columnas con muchos, recupera los valores que coinciden con las palabras de la pregunta, de la misma forma en que recuperas las consultas verificadas.

Una regla expresada en el contexto se sigue ignorando. Con el glosario, el modelo omitió el filtro de pedidos cancelados en gq-002 y dejó fuera los pedidos cancelados del recuento de clientes de gq-016. Con las notas de columna, y de nuevo con el glosario, los dejó fuera del recuento de pedidos de gq-013. Ambos casos van en contra de lo que indican la nota sobre status y el glosario. Con las cuatro capas, sumó los ingresos por pedido sobre las líneas de pedido en gq-006, algo que la entrada del glosario sobre unidades vendidas advierte que no se debe hacer. El contexto pone una regla a disposición del modelo, pero no hace que el modelo la siga. Solo las puntuaciones de referencia por pregunta lo revelan. Cuando una regla tiene que cumplirse siempre, sácala del prompt y llévala a código que el modelo no pueda saltarse, que es lo que hace la capa semántica del capítulo 5.

La recuperación ofrece el ejemplo equivocado. La recuperación léxica coincide con palabras, así que una pregunta que comparte solo "most" con una consulta verificada recibe esa consulta como ejemplo, a menos que un umbral lo impida, y una pregunta que significa lo mismo con otras palabras no recibe ninguno. Registra qué consulta verificada usó cada respuesta, tal como la informa el campo confidence de la API de Cortex Analyst de Snowflake (Snowflake), y luego revisa los registros en busca de respuestas incorrectas.

Una consulta verificada se confunde con una respuesta verificada. Una consulta recuperada guía al modelo, pero el SQL que escribe es nuevo y nadie lo ha comprobado. Databricks marca una respuesta de Genie como verificada solo cuando se usa el texto exacto de una consulta de ejemplo parametrizada (Databricks), y esa es la distinción que hay que mantener. Nunca muestres una respuesta como verificada porque lo era una consulta parecida. Lo detectas comprobando lo que dice tu interfaz sobre las respuestas construidas a partir de ejemplos.

Las consultas verificadas quedan obsoletas. Una consulta comprobada en octubre usa las definiciones de octubre. Cuando finanzas cambia una definición, las consultas antiguas enseñan la regla antigua a cada pregunta parecida. Conserva una fecha verified_at en cada consulta, vuelve a ejecutar cada una contra su equivalente de referencia cuando cambie una definición y elimina las consultas que nadie haya comprobado desde el cambio.

El contexto crece hasta costar más de lo que aporta. Cada capa es barata de añadir, y nadie quita una que dejó de ayudar. En una CPU, cada token del prompt cuesta tiempo, y con un modelo alojado cuesta dinero en cada llamada. Registrar los tokens de cada llamada junto con su resultado, como hacen las grabaciones de aquí, y repetir el análisis de ablación cuando una capa crece, muestra qué capa ha dejado de justificar sus tokens.

Regla práctica: Versiona el contexto junto con el almacén de datos: mantenlo al lado de los data marts, compruébalo en CI frente a las tablas que menciona, puntúalo con preguntas de referencia en cada cambio y regístralo por respuesta junto con las consultas verificadas que usó. Saca del contexto cualquier regla que deba cumplirse siempre y llévala a una definición que el modelo no pueda saltarse.

Decisiones que tomó este capítulo

  • El contexto llega en cuatro capas: el esquema completo, notas sobre las columnas cuyos nombres inducen a error, un glosario de negocio y consultas verificadas recuperadas para cada pregunta. Cada una vive en un archivo dentro de examples/data/shop/, y build_context() en lib/context.py las ensambla.
  • El esquema nunca se recorta mientras quepa; el presupuesto se aplica a las demás capas.
  • Las consultas verificadas se recuperan léxicamente, mediante TF-IDF sobre las palabras de la pregunta, con los números, los meses y los códigos de región enmascarados, como máximo tres, cada una con una puntuación de al menos 0.4. Ninguna de ellas es una pregunta dorada (golden), y cada una nombra al equipo que la verificó, nunca a una persona.
  • El contexto se presenta con las capas estables primero y la pregunta al final, de modo que la pregunta quede junto a sus ejemplos y el ejecutor pueda reutilizar el prefijo compartido.
  • Los tokens se estiman a partir de caracteres, a razón de 3.5 caracteres por token; esto sobrestimó el recuento del modelo en el prompt revisado, así que un contexto que cabe en la estimación debería caber en el modelo. Compruébalo con tus propios prompts.
  • Las capas se miden por ablación sobre las preguntas de respuesta del conjunto de desarrollo, una capa a la vez, y el mejor conjunto se verifica una sola vez en el conjunto de prueba junto al esquema solo.
  • El libro conserva las cuatro capas para los capítulos que escriben SQL, porque ese conjunto obtuvo la mejor puntuación en desarrollo. El glosario se mantiene aunque no mejoró la puntuación por sí solo: la capa de consultas verificadas se midió encima del glosario, y no medimos las consultas verificadas sin él.

Este capítulo, en esta edición

Edición 2026.11.15Ejemplos verificados el 19 de nov de 2026Superan la prueba 9 de 9 ejemplos ejemplos de este capítulo

Traducido del libro en inglés, edición 2026.11.15.

Cita esta sección

Formato

Cada formato cita el libro en inglés, que esta traducción sigue.

Cada formato nombra esta edición. Su enlace abre la edición actual, donde el encabezado de cada sección indica cuándo cambió su texto por última vez. Las fechas de acceso y de recuperación son el día en que lo abres.

En esta página

Capítulo 4 de 11: Give the agent layered context

  1. Qué dice el esquema y qué no dice
  2. Describe columns where names mislead
  3. Add the business glossary
  4. Retrieve verified queries as examples
  5. Ajusta el contexto a un presupuesto
  6. Measure what each layer adds
  7. Verificar las capas elegidas en el conjunto de prueba
  8. Qué sale mal en producción
  9. Decisiones que tomó este capítulo

Volver arribaTodos los capítulos