Business Intelligence with Agents

Contents

11 chapters, about 4 hours 35 minutes to read or 6 hours 11 minutes to listen to, with 78 examples.

1. 1A data agent end to end

   By the end of this chapter, you can build the shop's warehouse, have a recorded model write SQL for a business question, run that SQL and catch the wrong number it returns, answer the same question from one defined metric, and score both approaches on five golden questions.

   21 minutes6 examples

   8 sections

   • Meet the shop and its warehouse
   • Ask a question, get SQL from a model
   • Run the SQL and check the number
   • Answer from one defined metric
   • Score both on five golden questions
   • Read the trace of one run
   • What goes wrong in production
   • Decisions this chapter made
2. 2Where raw text-to-SQL gets the number wrong

   By the end of this chapter, you can name the five ways schema-only text-to-SQL returns a plausible wrong number on a real warehouse, show each one on the shop's warehouse, count them across a set of golden questions, and tell which a better prompt can fix and which need definitions the schema doesn't hold.

   23 minutes6 examples

   8 sections

   • Joins that fan out and inflate a sum
   • Definitions the schema doesn't hold
   • Time zones and the fiscal calendar
   • The table with the tempting name
   • Filters nobody wrote down
   • Count the failures by kind
   • What goes wrong in production
   • Decisions this chapter made
3. 3Build golden questions from your own work

   By the end of this chapter, you can write golden questions from the questions people in your business ask, each with its golden SQL, the role that asks it and the kind of answer it needs.

   23 minutes8 examples

   9 sections

   • Why a public benchmark isn't your test
   • Write questions from the work people do
   • Compare results, not SQL text
   • Order, rounding and column names
   • Check the gold before you trust it
   • Cover declines and roles, not only answers
   • Hold out a test split
   • What goes wrong in production
   • Decisions this chapter made
4. 4Give the agent layered context

   By the end of this chapter, you can give a model that writes SQL more than the schema, in layers: notes on columns whose names mislead, a glossary of the shop's business terms, and verified queries retrieved for each question.

   27 minutes9 examples

   9 sections

   • What the schema says, and what it doesn't
   • Describe columns where names mislead
   • Add the business glossary
   • Retrieve verified queries as examples
   • Fit the context to a budget
   • Measure what each layer adds
   • Check the chosen layers on test
   • What goes wrong in production
   • Decisions this chapter made
5. 5Define metrics once in a semantic layer

   By the end of this chapter, you can write the shop's semantic models and metrics in MetricFlow's YAML spec, compile a metric request to SQL with the book's own compiler, show that its entity joins can't fan out, group by the fiscal calendar, and confirm that MetricFlow compiles the same definitions to the same results.

   24 minutes8 examples

   9 sections

   • Describe each table as a semantic model
   • Compile a metric request to SQL
   • Join through entities so sums can't fan out
   • Build ratio and derived metrics from simple ones
   • Put the fiscal calendar in the time spine
   • Check the definitions against MetricFlow
   • One definition, any warehouse's dialect
   • What goes wrong in production
   • Decisions this chapter made
6. 6Route questions to governed metrics first

   By the end of this chapter, you can give the agent the semantic layer's metrics as tools instead of the warehouse's tables, have it return a structured metric request that's validated before anything runs, have it ask the person back when a question could mean several metrics, and measure the change on the golden questions.

   25 minutes6 examples

   8 sections

   • Give the agent metrics, not tables
   • Ask for a request, not SQL
   • Validate the request against the layer
   • Ask back when a question is ambiguous
   • Measure metrics-first on the golden questions
   • Read the runs that weren't correct
   • What goes wrong in production
   • Decisions this chapter made
7. 7Let the agent write SQL inside guards

   By the end of this chapter, you can handle the questions no metric covers by letting the agent write SQL that runs read-only on a database holding only the rows and columns its role may read, and that is parsed, checked against an allowlist of tables, bounded in rows, size, cost and time, repaired once from an error, and transpiled to your warehouse's dialect.

   32 minutes11 examples

   12 sections

   • Give the agent a database with only what it may read
   • Give each role a database of views
   • Parse the SQL before it runs
   • Allow only read-only queries on known tables
   • Bound the rows and the cost
   • Enforce the deadline by killing a process
   • Replay the attacks on the guard
   • Repair once from the error
   • Transpile to the warehouse's dialect
   • When to prefer a new metric over SQL
   • What goes wrong in production
   • Decisions this chapter made
8. 8Say when the answer can't be given

   By the end of this chapter, you can make declining a first-class answer: tell the questions the warehouse can't answer from the ones it can, decline with a reason of a known kind, check that the data for the period asked about has arrived, show the provenance behind every answer, and measure declines with precision and recall, so the agent neither guesses nor refuses too often.

   24 minutes5 examples

   7 sections

   • Kinds of questions the warehouse can't answer
   • Decline with a reason
   • Check freshness before answering
   • Show the provenance with every answer
   • Measure declines, not only accuracy
   • What goes wrong in production
   • Decisions this chapter made
9. 9Keep the agent inside who may see what

   By the end of this chapter, you can enforce each role's row filters, column masks and minimum group size below the agent, in the role's own database, with the rewrite in the semantic layer and the SQL guard as defence in depth, so that no prompt can widen access.

   26 minutes8 examples

   10 sections

   • A rule in the prompt is not a control
   • Apply row filters in the compiler
   • Mask columns in every path
   • Suppress groups small enough to identify someone
   • Subtracting two answers recovers a group
   • Treat text in the data as data
   • Measure access on the golden questions
   • Log who asked what, and what ran
   • What goes wrong in production
   • Decisions this chapter made
10. 10Judge answers and catch regressions

    By the end of this chapter, you can score a run of the agent on several measures at once, use a model as a judge only where results can't be compared directly, calibrate that judge against reference labels anyone can recompute, see how much a score moves from one run to the next, and gate a change to prompts, context or metrics in CI on recorded runs.

    26 minutes6 examples

    8 sections

    • Score a run on more than accuracy
    • Judge only what can't be compared
    • Calibrate the judge against reference labels
    • Repeat runs to see the variance
    • Gate a change in CI
    • Catch a schema change before users do
    • What goes wrong in production
    • Decisions this chapter made
11. 11Serve the semantic layer over MCP

    By the end of this chapter, you can serve the shop's governed metrics as MCP tools that answer for the role the server was built for, return each answer with its provenance as structured data, test the server in memory, and run the book's agent through it as a host.

    24 minutes5 examples

    7 sections

    • Turn metrics into tools
    • Carry the caller's role into every query
    • Return answers with their provenance
    • Test the server in memory
    • Answer questions through a host
    • What goes wrong in production
    • Decisions this chapter made