Time Series Forecasting

Teaching kit

From Edition 2026.11.11, the book's current edition. It changes when the book does. Use and adapt it freely, with credit to the book.

Syllabus

Reading times are for the chapter's text. Students also run its examples, in the Lab or on their own machine.

A chapter a week: 10 weeks

In 5 weeks

Objectives and outlines

Each chapter's objectives are its first sentence. Each outline follows the chapter's sections, with the terms it defines, the predictions it asks for before an example runs, and the rule each section ends on.

1. Forecast tomorrow's demand

About 20 minutes to read, 3 examples.

This chapter builds the first forecast of tomorrow's demand, and the simple baseline that every later forecast has to beat.

Lecture outline: 7 sections
  1. The case and its data
    • Working rule Check the cut against the data card before any forecast is made. The file should hold 30 clients, and the example counts how many of them have a full two years of hourly rows. Those counts should match the card. Confirm that MT_223 is missing and explained. A cut that does not match the card is different data, and every number computed from it describes that data, not this one.
  2. Load one client's hourly demand
    • Run load-lead-client.py
    • Working rule Choose the client and the window first, then count the quirks in that window's history before any model sees it. If a history holds a clock-change label or a before_first_reading row, do not hand it to a model unchanged, because the model will read the label's value as demand. Chapter 3 and Chapter 4 give the rules for those rows. A window that holds neither can use its rows as they are, as this chapter does.
  3. Forecast the next day with seasonal naive
    • Run seasonal-naive-next-day.py
    • Working rule Keep seasonal naive in every comparison for an hourly series with a daily cycle. It needs no fitting, so it runs on every window at no extra cost. A model that cannot beat it on the same windows has not earned its extra work. Check the forecasts for missing values before you read their score, because a missing reading in the last season becomes a missing forecast. Chapter 3 gives the rule for filling those hours before any model runs.
  4. Forecast the next day with one model
    • Run autoets-next-day.py
    • Working rule Add a model only with a baseline it has to beat on the same windows. Refit the model at each cutoff, using history from before that cutoff only. A model fitted once on the whole series and then scored on that series has seen the answers, and its score is not a forecast score.
  5. Score both on a held-out week
    • Working rule Score each model on forecasts it made without the scored hours in its history, over more than one day, and report the number of windows and hours behind each score. Compare models only on the same windows. Read MAE and RMSE together. If they disagree about which model is better, look at the largest misses in the windows before you choose, because a few large misses can decide the result.
  6. What goes wrong in production
    • Working rule Before you trust a score, check three things in this order: each window's first forecast hour is its cutoff, each history ends before its cutoff and holds the full history length, and no forecast is missing. The helpers raise an error for the first two, and the examples print the third. A score counts as a finding only after all three pass.
  7. Decisions this chapter made

2. Set the baselines a model must beat

About 21 minutes to read, 2 examples.

In this chapter you compute three baselines for a client's day-ahead forecast, naive, seasonal naive and average of weeks, and scale a client's error with MASE, using a scale that reads only the readings before the forecast week.

Lecture outline: 7 sections
  1. Why a baseline comes first
    • Working rule Score every baseline on the same windows, histories and scoring code as the model it will be compared with, and count each baseline's missing forecasts before you read its score. A model that beats a baseline scored on different hours has not beaten it.
  2. Three baselines to beat
    • Run naive-baselines-held-out-week.py
    • Working rule Start from naive and seasonal naive, both from the library, and add average of weeks when the series has a weekly rhythm worth testing. Check one forecast by hand against its formula before you rely on a baseline: pick one cutoff, compute one hour from the readings in its history, and compare the result with the forecast.
  3. Scale errors with MASE
    • Run mase-scale-leak-check.py
    • Working rule Report MASE beside MAE whenever the clients you compare differ in size, and say in the same sentence which history the scale comes from. Read MASE as a ratio of two mean errors, one from the scored hours and one from the history. It is not a percentage gain over seasonal naive, even when the MASE is well below 1.
  4. Take the MASE scale from history
    • Working rule Pass a scale only the readings before the first test day, mask its clock-change readings, and print the last reading it uses. Check that this reading is the hour before the first test day. A small difference between the two scales does not show the procedure is right. The row check does.
  5. Read a baseline's score
    • Working rule Read each baseline's MASE against 1 and against the other baselines, and do not name a winner on one week. A model counts as better only when it beats the best baseline on more than one set of windows, and when the margin is larger than the rounding of the printed scores.
  6. What goes wrong in production
    • Working rule Check a baseline's inputs, and its scale's, in this order: the same windows for every method, a scale that reads only earlier readings with its last reading printed, an unbroken hourly history for the seasonal lag, and a season that matches the data's cycle. Each check is one line of output or one count, and a failed check stops the comparison until it is fixed.
  7. Decisions this chapter made

3. Backtest without looking ahead

About 22 minutes to read, 7 examples.

This chapter teaches you to score a day-ahead forecast over a year of cutoffs without letting the future into its history, to handle the two clock-change hours in the data, and to run a shift test that catches a feature which reads past its cutoff.

Lecture outline: 10 sections
  1. Why one test week misleads
    • Working rule Prefer a year of cutoffs before you trust a difference between two methods. When cost forces fewer, say so beside each score, as Chapters 4 to 9 do. Make sure the sample reaches the odd days of the year. Report how many windows and how many scored hours stand behind each number. A figure from one week is one sample, and a reader should be able to see that.
  2. Roll the cutoff forward
    • Run backtest-sundays-2014.py
    • Working rule Set the history length explicitly, in hours, and check that it is the same for every window. When a library loops over windows for you, read how it sets the training slice before you trust its output, because its default may be an expanding window rather than the length you chose.
  3. Keep the future out of the history
    • Run history-cut-check.py
    • Working rule Slice each window's history by its cutoff, never by the start of its test window, and pass a library only the rows it may use. Check the first and last timestamps of every history, and make sure the last one is an hour before the cutoff.
  4. Meet the clock changes
  5. The March hour that reads zero
    • Run clock-change-census.py
    • Run clock-gap-daily-cutoffs.py
    • Working rule Treat a 01:00 reading of zero on a March clock-change day as a missing hour, not as zero demand. Keep it out of a model's history and out of the scores, and do not let a seasonal method copy it into the next day.
  6. The October hour that holds two
    • Run clock-repeat-daily-cutoffs.py
    • Working rule Treat the October 01:00 label as two hours of energy in one slot, and never as an ordinary hour. Do not halve it to make it look like one, for the reason the next section gives.
  7. Mark clock-change hours as missing
    • Run clock-masked-backtest.py
    • Working rule Mark both clock-change hours missing in every history and every score, count the masked hours in your report, and fill them from the history before the cutoff only when the model cannot take missing values. If the count of masked hours changes from one run to the next, the sample or the file's flags have changed, and you should find out why before you rerun the scores.
  8. Check a backtest for leakage
    • Run leakage-shift-check.py
    • Working rule Keep every lag at least the horizon, 24 hours, so that the reading a lag uses for any target hour was known at the cutoff. Before you trust a backtest with features, run both halves of the shift test on every feature, at several cutoffs spread through the year, and require zero differing rows for every feature the model reads. A feature that differs is reading the future, so fix or remove it before you score anything.
  9. What goes wrong in production
    • Working rule Run the clock and shift checks again on every data refresh, and before any change to the sample's dates, the history length or the scale. Keep the masked-hour count, the census counts and the shift-test counts in the run log next to the scores, so that a later change in either can be traced to its cause.
  10. Decisions this chapter made

4. Fit a statistical model for each client

About 22 minutes to read, 6 examples.

Here you fit AutoETS and a fixed seasonal ARIMA for each client over the same backtest, score both against seasonal naive, and learn to start a late-starting client at its first reading.

Lecture outline: 10 sections
  1. Why a statistical model comes next
    • Working rule Fit one model per client only after a baseline and a backtest are in place. Count the fits before you choose the number of windows: clients, times windows, times refits if you run the model every day. Use the same cutoffs and history for every model in a comparison. Otherwise the comparison measures the windows, not the models.
  2. Fit exponential smoothing
    • Run autoets-subset-backtest.py
    • Run flat-form-check.py
    • Run autoets-forced-season-backtest.py
    • Working rule Start with AutoETS for hourly demand with season_length=24, and print the chosen forms in every backtest. When a client takes a flat form, check that its seasonal form is not failing. If the seasonal fit has the lower likelihood, force the season with model="ZZA", and score both forms on earlier windows before you keep either: cutoffs before the held-out week that the report does not score, such as the four ordinary days in late November and early December 2014, which Chapter 5 sets aside for choices. The standard cutoffs show the comparison here, but they do not decide it. When a forecast gets worse, look first for a form that changed between cutoffs.
  3. Fit a fixed seasonal ARIMA
    • Run arima-summary-look.py
    • Working rule When a search is too slow to run at every cutoff, fix the order and write it in the call. Print the summary once, and check that the terms it lists are the ones you meant. Refit the fixed order at every cutoff, as you would refit any model. The order stays fixed. The coefficients do not.
  4. Compare models on the same cutoffs
    • Run arima-subset-backtest.py
    • Working rule Score every candidate on the same cutoffs, with the same history and the same scale, and report MAE and MASE side by side. When the two disagree, say which one the decision needs. We recommend more windows before choosing between two models that differ by less than about a tenth of MASE. That threshold is a judgement from this sample, not a measurement.
  5. Meet the late starters
  6. A history that starts with zeros
  7. Start each series at its first reading
    • Working rule When a client's history holds zeros, check whether its first non-zero reading falls inside the history. If it does, apply the start rule before any model sees the history, whatever the zero share, because leading zeros are not use. Run check_start_rule on the whole series first, and if it fails, find out why before you trim anything. Print the history length at each cutoff, because a short history is the first sign that the rule has taken too much.
  8. Why the book avoids automatic ARIMA at season 24
    • Working rule Do not run an automatic search with a season of 24 on hourly data in a schedule that refits every day. Time the search on one client before you commit to it. If it is too slow, fix the order and test that order in the backtest.
  9. What goes wrong in production
  10. Decisions this chapter made

5. Add calendar and lag features

About 23 minutes to read, 2 examples.

After this chapter, you can turn one client's hourly history into a feature table, add lags and calendar and holiday features that a day-ahead forecast is allowed to see, measure what each feature adds on the same windows, and catch a lag that reads the day being forecast.

Lecture outline: 8 sections
  1. Turn history into features
    • Working rule Build every feature row as if the cutoff were the last reading you have. A feature for a forecast hour must be computable from readings at or before the cutoff of its window. Before any shift test, ask whether the feature is known at the cutoff. Then check it with the shift test from Chapter 3, and do not train on a table you have not checked.
  2. Add lags the forecast can see
    • Working rule Use only lags at least as long as the horizon. With a 24-hour horizon that means 24 hours or more, and this book uses 24 and 168. A shorter lag needs a reason, and a model that uses one must pass the shift test before anyone trusts its score. Keep the clock rule on the lags, and count the rows whose lags read a masked hour, and the hours filled from history.
  3. Add calendar and holiday features
    • Working rule Add the calendar features the case's rhythm needs, and keep each one only if it helps on the same windows, as the next section measures. For holidays, use a calendar that covers every year in your data and every year you will forecast, and print how many holidays it holds for those years. A count of zero for a forecast year means the list is missing that year, not that the year had no holidays.
  4. Choose on decision windows
    • Working rule Choose a feature on windows that come before the held-out week and hold the kind of day the feature is for: a holiday feature needs holiday windows. The standard cutoffs are reported beside the choice and never used to make it.
  5. Measure what each feature adds
    • Run lgbm-feature-sets.py
    • Working rule Keep a feature set only when it lowers the mean per-client MASE against the set it extends, on the same windows. Report the pooled MAE beside it. Do not tune the model to make a disagreement between the two views go away.
  6. Catch a lag that leaks the future
    • Run lag-leak-check.py
    • Working rule Run the shift test on every lag and every window feature before a model trains on it, and treat any differing row as a leak until you have shown otherwise. When an error falls sharply after you add a feature, check that feature first, before you credit it.
  7. What goes wrong in production
    • Working rule Recheck the holiday count for each forecast year, the masked hours and the rows they reach, and the lag lengths whenever the data or a library changes. Run the shift test on every lag, and read the per-client MASE beside the pooled MAE. A check that changes is a reason to stop and read the cause before the next forecast is published.
  8. Decisions this chapter made

6. Forecast many clients with one model

About 21 minutes to read, 4 examples.

You finish this chapter with one global model for all thirty clients, a start rule that keeps the late starters' leading zeros out of it, and a score that sets it beside seasonal naive for each client on the same cutoffs.

Lecture outline: 7 sections
  1. One model for many clients
    • Working rule Fit one global model when your series share a shape and you must forecast many of them on each window. Scale each series' target with statistics from that window's history only, use only lags at least as long as the horizon, and fit once per window, not once per series. Then score the model per series against a baseline on the same windows, as the sections below do, before you trust its pooled error.
  2. Identify each client
    • Run client-identifier-check.py
    • Working rule Treat a client identifier as one more feature under Chapter 5's rule: keep it only if it lowers the mean per-client MASE on the decision windows, with the pooled MAE beside it. If the decision is close, say so, and measure it again on other windows. If the identifier wins, write down its cost: a new client has no learned split until it has history, and the model must be refitted before it can forecast that client.
  3. Apply the start rule to every client
    • Run start-rule-leak-check.py
    • Run global-model-raw-starts.py
    • Working rule Apply the start rule to every series that has leading zeros, and find each first reading from the rows before the cutoff. When the rule refuses a cutoff, stop the run and name the client and the cutoff. Do not fill the zeros, and do not publish a panel with a client missing. Judge the rule by the scale and by the windows where zeros enter the history, and expect the MAE effect to be small and mixed.
  4. Score thirty clients on the same cutoffs
    • Run global-model-thirty-clients.py
    • Working rule Score a global model per client on the same cutoffs as the baseline, with the MASE scale fixed before the first scored cutoff. Report the mean per-client MASE beside seasonal naive, and print the counts of clients and scored hours so a reader can check them. State any client left out of MASE and why. Read the pooled MAE and RMSE as the cost in kilowatt hours, not as the measure of model quality.
  5. Budget the run time
    • Working rule Before you run a backtest, count its fits: clients times windows times methods for a per-client method, and windows times methods for a global one. If the count is over budget, cut the subset or the window count, never the scored windows, and write the cut down beside the score. Then a reader can compare like with like.
  6. What goes wrong in production
    • Working rule Print and check the counts (clients, windows, scored hours, refused cutoffs, clients left out of MASE) on every refresh, before any score is read. Then report the mean per-client MASE beside seasonal naive. Stop the run on any refusal, and do not publish a total built from a short panel.
  7. Decisions this chapter made

7. Make the total add up

About 19 minutes to read, 2 examples.

You'll forecast the total demand of thirty clients two ways, choose between the two on decision windows, and split that total into client shares using only the history before each cutoff.

Lecture outline: 7 sections
  1. Why the total needs a check
    • Working rule Forecast the total both ways whenever a decision depends on it. Score both on the same windows, against the total and against each client. Keep the method that wins on the decision windows, and never accept a total because its parts add up.
  2. Sum the client forecasts
    • Run total-bottom-up-and-direct.py
    • Working rule Sum the client forecasts hour by hour, after checking that the total series equals the sum of its readings in every hour without a clock-change flag. Apply the start rule to each client before the sum, so no client's model is fitted to its zeros. Treat the sum as a point forecast, and do not take the sum of the clients' medians as the median of the total.
  3. Forecast the total directly
    • Working rule Forecast the total directly with the clients' features, from the total's own history before each cutoff, and choose the target's scaling on the decision windows. Use the same windows for both methods, and score each against seasonal naive on the total. Report both methods, and not only the one you built.
  4. Compare the two totals
    • Working rule Choose a method on the decision windows, by the measure you will report. Report the same measure on the standard cutoffs beside seasonal naive, with the scored hours and the scale. When two window sets disagree, report both, say how far apart they are, and do not pick the one that reads better.
  5. Split a total by share
    • Run total-split-by-share.py
    • Working rule Split a total by share only as a baseline, or when the clients' daily profiles are alike. Compute each share from the history before its cutoff, check that no share moves when readings after its cutoff change, and score each client against bottom-up on the same scale. Keep the split for a client only when it wins on the decision windows.
  6. What goes wrong in production
    • Working rule In production, check the total against the sum of its readings on every refit, stop when a client is missing from a total, and run the poisoning check on each share and on the scale for each window. Log the total's error with and without the identifier on the same windows. When the windows disagree, report both and do not choose the one that flatters the model.
  7. Decisions this chapter made

8. Quantify forecast uncertainty

About 23 minutes to read, 4 examples.

This chapter puts a range around each day-ahead forecast, so you can say whether a purchase can rely on it.

Lecture outline: 7 sections
  1. Why one number is not enough
    • Working rule Ask for a range wherever the costs of the two errors differ, and choose its level from those costs rather than from a number that looks reassuring. Report coverage as the share of hours inside the band over many windows, never as the chance that one day falls inside.
  2. Build intervals from residuals
    • Run conformal-three-windows.py
    • Working rule Build each band from the residuals of windows that come before its cutoff, and check the coverage it reaches on windows it did not use before you trust it. Read the level a design can reach before you read the coverage. Three residuals per client and hour cannot reach either label, so the fix is a larger calibration set, not a different reading of the output.
  3. Calibrate on earlier windows
    • Run conformal-decision-windows.py
    • Run conformal-earlier-windows.py
    • Working rule Calibrate on windows that come before the scored ones, with no holiday in them, and scale each client before pooling its residuals. Make each design choice on windows that are not the ones you report. Report how many windows and residuals went into the bands and the coverage they gave on the scored days, even when that coverage falls short of the level.
  4. Measure coverage on held-out days
    • Working rule Report coverage on windows that the calibration never used, with the number of windows and the coverage of each. Keep the same measure in a running log once the forecast runs every day, as Chapter 10 does. Read three scored days as a warning, not as a measurement of the level you were promised.
  5. Choose the level from the cost of error
    • Run buy-ahead-level-by-cost.py
    • Working rule Set the buy level from the two costs, as tau = cu / (cu + co), fix it before you score the days, and calibrate the total's bands on the total's own residuals. Show the realised cost at other levels as a check, but never move the level to whichever one would have won on the days you score.
  6. What goes wrong in production
  7. Decisions this chapter made

9. Compare Prophet and a foundation model

About 22 minutes to read, 6 examples.

Once you finish this chapter, you can fit Prophet on the book's three standard cutoffs, load a recorded foundation model (Chronos-2) and a recorded neural model (NHITS), and score all of them on one MASE scale against the same readings.

Lecture outline: 7 sections
  1. Where Prophet fits
    • Working rule Use Prophet per client when a reader needs its trend and seasonal settings as parameters they can read, and when its scored forecasts beat seasonal naive on the decision windows you chose before scoring. Set its interval width in code, and read its default 80% interval as an 80% interval, never as a 95% range. Check its coverage against the level it claims.
  2. Run Prophet on the same cutoffs
    • Run prophet-holiday-decision.py
    • Run prophet-thirty-clients-backtest.py
    • Working rule Set Prophet's seasonality, holidays and interval width in code before you score it, and choose any of them on decision windows, never on the standard cutoffs. Keep the holiday table only if it lowers the MASE on the holiday decision windows. Compare Prophet with seasonal naive and the global model on the same windows and the same scale, and list the clients it loses. Set the seed before each predict, and check that a second run prints the same lines.
  3. Load a recorded foundation model's forecasts
    • Run recorded-chronos2-check.py
    • Working rule Record a foundation model only with its script, its pinned revision, its package versions and a manifest that hashes every output. Read the recorded outputs in every example, and check the hashes before any score reads them. Check the licence of the weights, not only the licence of the package, and name the model version in any licence sentence.
  4. Score the recordings on the same cutoffs
    • Run recorded-chronos2-score.py
    • Run recorded-nhits-score.py
    • Working rule Score a recorded forecast on the same windows, the same scale and the same readings as every other model, and name any interpolated quantile wherever its range is reported. Before you read a lead, check what the model was trained on, because a model that saw the scored readings cannot show skill on them. Do not choose a model or a setting on these windows. A recorded forecast with no forecasts on the decision windows cannot be chosen for production until it has them.
  5. Why the neural model is recorded
    • Run recorded-nhits-check.py
    • Working rule Keep a model out of the examples' runtime when its dependencies would break the runtime budget, and record it instead, with its pins in a separate requirements file. Before you add a dependency, install it in a clean environment and measure what it adds in size and in packages. A resolve that moves a pinned package is a reason to record the model, not a reason to change the pin.
  6. What goes wrong in production
    • Working rule Before any Prophet or recorded forecast is reported, check that the holiday table covers the forecast's years when the table is on, that the interval level is printed beside its coverage, that the recording has a window for every cutoff in use, that its quantiles are in order, and that the model revision matches the one scored. A failed check stops the report, and the report says which check failed.
  7. Decisions this chapter made

10. Keep the forecast running

About 23 minutes to read, 5 examples.

Once a forecast runs every day, the question changes from whether it scored well to whether it still holds on the day it's used.

Lecture outline: 7 sections
  1. Refit on a schedule
    • Run refit-cadence-decision.py
    • Working rule Choose the refit cadence on ordinary days that come before the held-out week, scored on one scale that was read before the first of those days. Take the cadence with the lower mean MASE, and when the two are close, take the one with fewer refits and say so. Never refit on readings from the day you are forecasting.
  2. Log every forecast before its actuals arrive
    • Run rolling-refit-seven-days.py
    • Working rule Write each forecast's rows at its cutoff, from the readings that have arrived by then, with the refit day and both arrival times beside them. Never rebuild a past log from a newer model or from the actuals. If a day's rows are lost, mark that day as missing in the log, and do not fill it in afterwards.
  3. Score the log as actuals arrive
    • Run late-fill-decision.py
    • Working rule Score each day when its last reading arrives, against the forecast logged at its cutoff, on the chapter's one scale. When a reading arrives late, score it then, count it in the log, and leave the logged forecast alone. Never change a forecast after its cutoff, even when the change would improve the score.
  4. Flag drift against the backtest
    • Run drift-flag-rolling.py
    • Working rule Flag a day when its daily MASE is above twice the median of the earlier calibration windows, and then ask whether a calendar cause explains it. Record the cause beside the flag when there is one. Treat two unexplained flags in a row as the start of a retirement check, never as proof that the model is broken.
  5. Know when to retire a model
    • Run drift-after-a-change.py
    • Working rule Refit early when flags run on several days in a row with no calendar cause, or when the 80% band covers far below 80% of readings on several days, and record the trigger with the log. Retire the model when an early refit does not clear the flags. Re-run this chapter's checks when a pinned library is released, and keep the pins until the checks pass on the new version.
  6. What goes wrong in production
    • Working rule Keep three checks in every run: the history rule that keeps the future out of each refit and forecast, the two arrival checks on each log row, and the leak checks from Chapters 3 and 7. When one of them fails, stop the loop and fix the cause before the next forecast. Do not repair a history or a log by hand, because a hand repair is the kind of change the checks exist to catch.
  7. Decisions this chapter made

Quiz bank

Questions on every chapter, with answer keys, for verified educators with the Educator plan. Each answer is tied to its section. The Educator plan