添加日历特征与滞后特征
阅读约需 76分钟 分钟,8节中含2个示例。
学完本章后,你可以把一位客户的逐小时历史数据整理成特征表,加入提前一天预测允许使用的滞后特征、日历特征和节假日特征,在相同的窗口上衡量每个特征带来的增益,并识别出读取了待预测日当天数据的滞后特征。
第 4 章中的统计模型只看到一条序列。它们只了解序列的水平和日内节律,除此之外一无所知:它们不知道 2014 年 12 月 25 日是公共假日,不知道星期一与星期日有何不同,也不知道需求会随季节变化。第 1 章已经说明了这样做的代价。其季节性朴素基线表现最差的一天是 12 月 26 日,即圣诞节次日,因为它把节日当天的读数原样向后复制了。梯度提升模型可以把这类事实作为列接收进来;而由于供应商在一天开始之前就要购电,只有在截止时刻已知的列才能进入它的预测。本章为八位客户的子集添加日历列和节假日列,并在每一列计入之前逐一检验。节假日标志并不能修复 12 月 26 日的失误,因为 12 月 26 日并非节假日。能够修复这一失误的特征是节后一天标志,它同样可以提前知道,但本章不构建它,而且本章的标准截止时刻也不对节假日打分。
这部分尚未翻译,因此以英文显示。
Turn history into features
The table comes first, because every feature in this chapter is a column of it. A feature table has one row for each hour a model must predict. Each row of a feature table holds the value to predict, which the book calls the target, and the features: the readings and facts the model may use for the row's hour. A gradient-boosting model learns a mapping from the columns to the target, one row at a time, so the table is all the model can know.
A row's features must be known at the cutoff of the window that forecasts it, because the forecast is made there. The target is the reading at the hour. The model needs two kinds of row. Training rows are the hours of the history before the cutoff, each with its own features and its own reading as the target. Forecast rows are the 24 hours after the cutoff, with features but no target. The model is fitted on the first kind and predicts the second.
Take the cutoff at 00:00 on Dec 30, 2014. Its forecast rows are the 24 hours of Dec 30, from 00:00 to 23:00, and each has a blank target. Its training rows run back eight weeks, to Nov 4, 2014, one row per client and hour, with the reading as the target. Take the row for Dec 15 at 18:00: its target is the reading at that hour, and its features are readings from earlier hours, such as the reading 24 hours before. No row is a daily total.
Each window's training table is built from the 1,344 hours of history before its cutoff, eight weeks, for each client in SUBSET8. The eight clients are stacked in one table, and the same construction repeats for every decision window and for the three standard cutoffs, Dec 29 to Dec 31, 2014. The first week of history has no reading a week earlier, so it leaves the table: 1,344 hours less 168 hours leaves 1,176 rows per client in each window. One model is fitted per window on that window's history alone, so no training row comes from the days being forecast.
For the lags-only set the columns are the two lags and the target y; the hour of the day arrives with the other calendar columns later in the chapter, and the holiday column after them. MLForecast in mlforecast 1.1.0 takes the series and produces the training rows in fit and the forecast rows in predict. Two of its defaults matter here. A training row whose features are missing is dropped (dropna=True, in mlforecast/core.py), and the first hours of each history drop out because they have no reading a week earlier. Nothing in the library stops a feature from reading the hour it predicts. The rule that stops it is the one you write and check.
Working rule: Build every feature row as if the cutoff were the last reading you have. A feature for a forecast hour must be computable from readings at or before the cutoff of its window. Before any shift test, ask whether the feature is known at the cutoff. Then check it with the shift test from Chapter 3, and do not train on a table you have not checked.
添加预测时可见的滞后特征
滞后特征是要加入那张表的第一批列:同一客户更早的读数。小时 t 的 k 小时滞后,就是 t 减 k 时刻的读数。滞后特征是让模型获得序列节律的最低成本方式:滞后 24 是昨天同一小时的读数,滞后 168 是一周前同一小时的读数。模型会学习对每一个滞后应给予多大的信任。滞后 168 向前回溯整整一周,落在同一个星期几上,因此无需星期列,它就带有周模式。
某个滞后是否已知,取决于预测范围。在 00:00 做出的预测覆盖截止点之后的 24 个小时。把当天的各小时记为 0 到 23,从截止点起算。滞后 k 读取的是第 j 减 k 小时,所以对于目标小时 j,只有当 j 减 k 为负时,它读到的才是截止点之前的读数。这必须对 0 到 23 的每个 j 都成立,也就是说 k 必须不小于 24。滞后 1 对第一个小时是已知的,因为它读取的是截止点之前的最后一个读数。对其余 23 个小时,它读取的是预测所没有的读数。
下表列出了截止点为 00:00 时,这一次日预测中三类目标小时的情况。小时 0 是截止点之后的第一个小时,小时 23 是最后一个小时。
| 目标小时 | 滞后 1 读取 | 滞后 1 在截止点时是否已知 | 滞后 24 读取 | 滞后 24 在截止点时是否已知 |
|---|---|---|---|---|
| 00:00(小时 0) | 前一天 23:00 | 是 | 前一天 00:00 | 是 |
| 01:00(小时 1) | 预测当天 00:00 | 否 | 前一天 01:00 | 是 |
| 23:00(小时 23) | 预测当天 22:00 | 否 | 前一天 23:00 | 是 |
供应商的两个滞后 24 和 168,在次日预测的每个小时都是已知的,所以模型从不需要尚不存在的读数。lgbm-feature-sets 把这两个滞后传给 MLForecast,其余的由该库完成。
第 3 章的时钟规则同样适用于滞后特征:2014 年 3 月 30 日的缺口小时被设为缺失,因此任何读取它的特征同样缺失。在截止点为 2014 年 4 月 1 日的历史中,lgbm-feature-sets 找到 1 个被屏蔽的小时,有 1 行的滞后读取了它:即缺口之后 24 小时的那一行。在拟合每个窗口的模型之前,该窗口中被屏蔽的小时会按第 3 章的方式填充,依据的是该窗口自身的历史,填充时不读取截止点之后的任何内容。填充之后,该小时就成为一个读数,读取它的那一行因此保留其值,LightGBM 也就不会从滞后特征中收到缺失值。这条规则是明确声明的选择,而不是实测得出的最优解。旁边打印的计数,才是读者应当核对的内容。
工作准则: 只使用不短于预测范围的滞后。对 24 小时的预测范围,这意味着 24 小时或更长,本书使用 24 和 168。更短的滞后需要给出理由,使用它的模型在任何人信任其得分之前,必须先通过平移检验。对滞后特征保持时钟规则,并统计滞后读取了被屏蔽小时的行数,以及由历史填充的小时数。
这部分尚未翻译,因此以英文显示。
Add calendar and holiday features
Lags are built from readings. Calendar features, unlike lags, are facts about the target hour itself: its hour of day, its day of the week and its month. They are known in advance, because the clock states them before any reading exists. They need no lag and no cutoff rule. Their value at a target hour cannot depend on any reading, so the shift test has nothing to catch.
The supplier's demand follows the day. The hour of day repeats what the lags already carry: the daily rhythm. The weekday adds what they do not. It tells the model that Saturday and Sunday differ from the working days. The month is the third column, and it is a weak one with this history. Eight weeks of history span two or three calendar months, so month can only tell a few weeks of the history from the rest, and on the standard cutoffs every forecast hour falls in December. The book uses three mlforecast date features, named by their attribute: hour, dayofweek and month. The weekday column already holds the weekend, so the book adds no separate weekend flag. In pandas, dayofweek counts Monday as 0 and Sunday as 6, so Saturday and Sunday are the two values a tree must separate from the rest. A tree can split on that in one step, which is why the weekday is cheap to test.
A public holiday does not follow the weekday. A model with only weekday features reads Dec 25, 2014 as an ordinary Thursday, and it reads a holiday on a Sunday as an ordinary Sunday. The holiday feature answers this. It is a function of the timestamp, is_holiday in the shared module examples/lib/features.py, which returns 1 on a Portuguese national holiday and 0 otherwise. mlforecast accepts such a function as a date feature and names its column after the function. The holidays come from the holidays package 0.110.dev20261118063242, under the MIT licence, through holidays.country_holidays("PT", years=[2013, 2014]). For the two years in the data, the package lists 18 dates. The book does not ship that list. Each run builds it from the package's calendar.
The standard cutoffs are Dec 29 to Dec 31, 2014, and none of their 72 hours per client is a holiday. The held-out week's Dec 25 is outside them, so the feature's effect on that day is not scored in this chapter. On the standard cutoffs, it reaches the scores only through the training history, which contains the holidays of Dec 8 and Dec 25, 2014.
The holiday list is a fact known in advance, like the weekday. It comes from a published calendar and not from the readings, so it cannot leak a reading from the target day. Calendar features answer the question in the working rule at every target hour, by construction, and a lag passes only when it is at least as long as the horizon.
Working rule: Add the calendar features the case's rhythm needs, and keep each one only if it helps on the same windows, as the next section measures. For holidays, use a calendar that covers every year in your data and every year you will forecast, and print how many holidays it holds for those years. A count of zero for a forecast year means the list is missing that year, not that the year had no holidays.
这部分尚未翻译,因此以英文显示。
Choose on decision windows
Decision windows are the windows the feature set is chosen on, never the standard cutoffs, which are the windows the book reports. There are two kinds. The first is four ordinary cutoffs, Nov 25, Nov 26, Dec 2 and Dec 3, 2014. The second is seven holiday cutoffs: the Portuguese national holidays of 2014 before Dec 25 whose eight-week history is complete, Apr 18, Apr 20, Apr 25, May 1, Jun 10, Aug 15 and Dec 8. The ordinary windows hold no holiday, so a holiday feature could never win on them, and the holiday windows are where it can. The choice therefore uses all the decision windows, because one set holds a holiday feature. The standard cutoffs are scored after the choice and printed beside it. They choose nothing.
The holiday flag acts in two ways. On the decision windows it is on for scored hours, so it can change a forecast. On the standard cutoffs no holiday falls in a scored hour, so the flag changes those scores only through training.
Working rule: Choose a feature on windows that come before the held-out week and hold the kind of day the feature is for: a holiday feature needs holiday windows. The standard cutoffs are reported beside the choice and never used to make it.
这部分尚未翻译,因此以英文显示。
Measure what each feature adds
A feature is worth keeping when it lowers the error on windows the model did not train on. To measure one, hold everything else fixed: the same windows, the same history, the same settings. Three feature sets are fitted, one at a time. The first has lags only. The second adds the calendar features. The third adds the holiday feature to them. Each set is fitted once per window, with one model for all eight clients of SUBSET8, and each is compared with seasonal naive on the same windows.
Each set gets two scores. The pooled MAE is the MAE over every client-hour in the windows, the single number for the whole supplier. The mean per-client MASE is MASE for each client, averaged over the eight clients. MASE divides each client's error by a scale: the mean absolute error that seasonal naive makes in sample on that client's own readings. The book uses one scale function everywhere, scoring.book_scale. The scale reads every reading before the first scored window of the set, Apr 18, 2014 at 00:00, with clock-change hours masked, so it does not depend on the windows scored and cannot leak. One client needs care. MT_115 starts reading on Jan 14, 2014, and the zeros before its first reading are left out of its scale, because it is counted from its first reading. Chapter 4 teaches that rule for late starters.
1"""Compare three feature sets for the global LightGBM forecast of SUBSET8 on the decision windows (B-5, B-9, B-10): four2ordinary cutoffs and seven 2014 holidays before Dec 25, each fitted scaled and unscaled. The mean MASE lines carry the3choice; the standard-cutoff lines are reported beside it and choose nothing."""4import pandas as pd56from lib.backtest import history_before7from lib.electricity import load8from lib.features import CALENDAR, PT_HOLIDAYS, is_holiday, rows_reading_missing, score_feature_sets9from lib.panel import DECISION_CUTOFFS, HOLIDAY_DECISION_CUTOFFS10from lib.quirks import mask_clock_changes11from lib.settings import EVAL_CUTOFFS, LEAD, SUBSET8, show1213LAGS = [24, 168] # both at least the 24-hour horizon, so each is known at every forecast hour14# CALENDAR is hour, dayofweek and month, as mlforecast date features; the holiday set adds is_holiday to them15FEATURE_SETS = {"lags_only": [], "with_calendar": CALENDAR, "with_holidays": CALENDAR + [is_holiday]}16MODELS = ["SeasonalNaive", *FEATURE_SETS, *[f"unscaled_{name}" for name in FEATURE_SETS]]17DECISION = sorted([*DECISION_CUTOFFS, *HOLIDAY_DECISION_CUTOFFS]) # the decision windows; standard cutoffs are not here18raw = load(SUBSET8)1920# DECISION[0] is the first decision window, Apr 18, 2014: the MASE scale reads only the readings before it21forecasts, scored, per_client = score_feature_sets(raw, DECISION, LAGS, FEATURE_SETS, DECISION[0])22show("clients", forecasts["unique_id"].nunique())23show("decision windows", len(DECISION))24show("scored hours", len(scored))25show("Portuguese holidays in 2013 and 2014", len(PT_HOLIDAYS))26show("holiday hours in the decision windows", int(is_holiday(scored["ds"]).sum()))27for model in MODELS:28 show(f"MAE {model.replace('unscaled_', 'unscaled ')}", (scored["y"] - scored[model]).abs().mean())29for model in MODELS:30 show(f"mean MASE {model.replace('unscaled_', 'unscaled ')}", per_client[model].mean(), digits=3)31for model in MODELS:32 # the prose does not quote this client, so this line is for reference only33 show(f"MASE MT_268 {model.replace('unscaled_', 'unscaled ')}", per_client.loc["MT_268", model], digits=3)3435# The standard cutoffs, reported beside the choice above. Nothing is chosen from them.36# The scale here reads the readings before the first standard cutoff, Dec 29, 2014.37_, _, standard = score_feature_sets(raw, EVAL_CUTOFFS, LAGS, FEATURE_SETS, EVAL_CUTOFFS[0])38for model in MODELS:39 show(f"standard cutoffs, mean MASE {model.replace('unscaled_', 'unscaled ')}", standard[model].mean(), digits=3)4041# Clock-change check: the history of a 1 April 2014 cutoff holds the 30 March gap, which the feature rules must reach.42march = history_before(mask_clock_changes(load([LEAD])), pd.Timestamp("2014-04-01")) # the Chapter 3 mask, then the cut43show("masked hours in the history", int(march["y"].isna().sum()))44# a row counts when either lag, 24 or 168, reads a masked hour45show("rows whose lag reads a masked hour", rows_reading_missing(march.set_index("ds")["y"], LAGS))
python lgbm-feature-sets.pyclients: 8
decision windows: 11
scored hours: 2112
Portuguese holidays in 2013 and 2014: 18
holiday hours in the decision windows: 1344
MAE SeasonalNaive: 305.21
MAE lags_only: 280.57
MAE with_calendar: 270.57
MAE with_holidays: 267.40
MAE unscaled lags_only: 304.03
MAE unscaled with_calendar: 271.63
MAE unscaled with_holidays: 280.75
mean MASE SeasonalNaive: 0.934
mean MASE lags_only: 0.886
mean MASE with_calendar: 0.894
mean MASE with_holidays: 0.893
mean MASE unscaled lags_only: 1.115
mean MASE unscaled with_calendar: 1.119
mean MASE unscaled with_holidays: 1.139
MASE MT_268 SeasonalNaive: 0.840
MASE MT_268 lags_only: 0.797
MASE MT_268 with_calendar: 0.758
MASE MT_268 with_holidays: 0.789
MASE MT_268 unscaled lags_only: 0.855
MASE MT_268 unscaled with_calendar: 0.871
MASE MT_268 unscaled with_holidays: 0.936
standard cutoffs, mean MASE SeasonalNaive: 1.162
standard cutoffs, mean MASE lags_only: 1.057
standard cutoffs, mean MASE with_calendar: 1.165
standard cutoffs, mean MASE with_holidays: 1.134
standard cutoffs, mean MASE unscaled lags_only: 1.357
standard cutoffs, mean MASE unscaled with_calendar: 1.494
standard cutoffs, mean MASE unscaled with_holidays: 1.486
masked hours in the history: 1
rows whose lag reads a masked hour: 1
Seasonal naive is the bar here. Its MAE, pooled over 2112 scored hours (eight clients in 11 windows of 24 hours each), is 305.21 kWh.
Without scaling, the lags-only model trails seasonal naive on the per-client view. Its mean per-client MASE is 1.115, worse than seasonal naive's 0.934. The clients differ a great deal in size, so the largest clients' readings are many times the smallest's. A global model fits one loss over every client's rows, and here that loss is squared error in kWh, so the largest clients' errors set the splits and the small clients carry little weight. We think the cause is the loss, not the range of any one client. The scaled runs below remove the difference, which supports that reading. It is a judgement from these numbers, not a proof.
The scaled runs standardise each client's target with mlforecast's LocalStandardScaler (mlforecast/target_transforms.py), which subtracts the client's mean and divides by its standard deviation. Both come from the window's history before the cutoff, fitted inside fit, and predict only inverts them. No reading after the cutoff sets them, so the scaling passes the same test as the features. make_forecaster takes scaled=True by default, and Chapter 6 reuses that default.
Scaled, the comparison changes, and lags alone beat seasonal naive on the per-client view: a mean MASE of 0.886, against 0.934. Its pooled MAE is 280.57 kWh.
The calendar set does not beat lags alone. It still beats seasonal naive: its mean MASE is 0.894, and its pooled MAE is 270.57 kWh.
The holiday flag, added to the calendar set, narrows the gap without closing it: the mean is 0.893, and the pooled MAE is 267.40 kWh.
Chapter 4 set AutoETS and ARIMA as the bar a gradient-boosting model must beat. On the standard cutoffs, lags alone also come in under that bar: 1.057 against ARIMA's 1.09, on the same clients, cutoffs and scale.
By the working rule, only the lags stay, and the per-client view decides it, because it is the view that matters to a client's forecast. The calendar set's mean MASE is above lags alone's, so it is not kept. The pooled view points the other way here: the calendar set's pooled MAE is below lags alone's, and the holiday set's is lower still. The chapter reports both, as the rule asks, and does not tune the model to make them agree.
The rule keeps a feature only when it lowers the error of the set it extends. The holiday flag lowers the calendar set's mean by a hair, too little to count as a gain, and the set that holds it is still above lags alone. The flag is on for 1344 client-hours, a full day on each holiday window, for every client, so it had its chance, and it does not earn a place. The book therefore hands Chapter 6 the lags-only set. That result comes from 11 windows, which is a small test.
Working rule: Keep a feature set only when it lowers the mean per-client MASE against the set it extends, on the same windows. Report the pooled MAE beside it. Do not tune the model to make a disagreement between the two views go away.
这部分尚未翻译,因此以英文显示。
Catch a lag that leaks the future
The measurements above assume that each feature is known at its cutoff. A leak is a feature that uses a reading the forecast cannot have at its cutoff. In the book's view, a leak is a likely reason a backtest looks better than the model will be, and it can be built by accident. One pattern builds the feature table once from the whole series, so each target hour carries its lag, and then scores the table's forecast rows as if they were out of sample. Nothing fails. The score is too good, and nothing in it says so.
The library adds a second trap of its own. MLForecast predicts recursively by default. In recursive prediction, for a 24-hour horizon, MLForecast forecasts one hour, writes that forecast back as the reading for the next hour, and repeats (_predict_recursive in mlforecast/core.py). A lag of 1 therefore reads a forecast for every hour after the first, while training read actual readings for that lag. Calling that mismatch a leak is the book's judgement. It is one more reason to choose lags of 24 or more. With lags of 24 and 168, every step of the 24 reads actual readings, so the recursion never feeds this model its own forecasts. With max_horizon set, the library trains one model per horizon instead, and predicts with them in _predict_multi. The rule does not change with it: a feature must still be known at the cutoff of the window it forecasts.
The shift test from Chapter 3 carries over with one change: compare the lag at the target hours after the cutoff, not at the history rows. Compute the lag for each target hour twice: once from the history cut at the cutoff, and once from the whole series. Count the hours where the two differ. A lag of 24 agrees on every target hour, because each of the 24 hours reads a reading before the cutoff. A lag of 1 agrees only on the first hour.
Which do you expect to have the lower error, the model with a lag of 1 or the one without it? Both are fitted on the same training rows with the same settings, for each of the three standard cutoffs. The honest model uses lags 24 and 168. The leaky model adds a lag of 1, read from the whole series for the target hours, which is what the table built once would do.
1"""Shows which lag leaks: a lag of 1 reads the hour before each target hour, which a 24-hour forecast does not have at its2cutoff. The rows-differing lines are the shift test, and the two MAE lines show how much the leak flatters the score."""3import numpy as np4import pandas as pd56from lib.electricity import load7from lib.features import fit_window_mae, lag_values8from lib.settings import EVAL_CUTOFFS, HORIZON, LEAD, show910y = load([LEAD]).set_index("ds")["y"].asfreq("h") # asfreq gives every hour a row, so a missing hour is NaN, not skipped11HONEST, LEAKY = [24, 168], [1, 24, 168] # HONEST lags are both at least the horizon; LEAKY adds lag 112CUTOFFS = pd.to_datetime(EVAL_CUTOFFS)131415def shift_test(lag):16 """Target hours where the lag differs between the history cut at each cutoff and the whole series, and how many."""17 compared = differing = 018 for c in CUTOFFS:19 targets = pd.date_range(c, periods=HORIZON, freq="h") # the 24 hours each forecast must predict20 honest = lag_values(y[y.index < c], lag, targets) # readings before the cutoff only, as the forecast has them21 whole = lag_values(y, lag, targets) # the same lag read from the whole series, as a table built once would read it22 same = (honest == whole) | (np.isnan(honest) & np.isnan(whole)) # NaN never equals NaN, so the second term counts two missing readings as the same23 compared += len(targets)24 differing += int((~same).sum())25 return compared, differing262728show("cutoffs", len(CUTOFFS))29show("rows compared", shift_test(1)[0]) # the first value is the hours compared, the same for every lag30for lag in (1, 24, 168):31 show(f"rows differing for lag {lag}", shift_test(lag)[1])32show("MAE honest, lags 24 and 168", fit_window_mae(y, HONEST, CUTOFFS, leaky=False)) # the table holds the lags and the hour of day; leaky=False reads the target lags from the cut history33show("MAE leaky, lags 1, 24 and 168", fit_window_mae(y, LEAKY, CUTOFFS, leaky=True)) # leaky=True reads the whole series
python lag-leak-check.pycutoffs: 3
rows compared: 72
rows differing for lag 1: 69
rows differing for lag 24: 0
rows differing for lag 168: 0
MAE honest, lags 24 and 168: 5.36
MAE leaky, lags 1, 24 and 168: 2.87
Only the lag of 1 fails the shift test. Of the 72 target hours compared, 69 read a different value from the cut history than from the whole series: every hour of each window but the first. The lags of 24 and 168 differ on 0 and 0 rows. The lag of 1 agrees only on the first hour of each window, because that hour reads the last reading before the cutoff, which the forecast does have.
The leaky model looks almost twice as good: its MAE is 2.87 kWh, against 5.36 kWh for the honest model. The gap is the leak. Its lag of 1 reads the hour before the target, which for most target hours is an hour of the forecast window itself. The honest model never sees that hour, so its error is the one that production will show.
Working rule: Run the shift test on every lag and every window feature before a model trains on it, and treat any differing row as a leak until you have shown otherwise. When an error falls sharply after you add a feature, check that feature first, before you credit it.
生产环境中的常见问题
如果节假日列表在数据结束处截止,之后的每个节假日都会读到零。这样,预测年份的每一天标志都是关闭的,模型会把节假日当作普通日来预测。第一个节假日到来时,仪表盘上就会出现一次很大的偏差。示例打印的节假日数量就是检查手段:新一年的列表若打印出零个节假日,就说明缺少该年的数据。holidays 的后续版本也可能改变其日历,因此应像本书一样固定版本,并在每次升级后重新运行计数。
如果把为小时级模型设计的、短于预测范围的滞后项沿用到日前预测中,它在回测中得分很好,因为回测是用整条序列构建的。在生产环境中,靠后的那些小时读取的是模型自己的预测值。应当按日前预测的小时,把线上误差与回测自身在各小时的误差作比较。没有泄漏的模型,在每个小时上的误差应与回测显示的误差大致相当。若误差随预测范围内小时数的增加而增大,超出这一水平,就是模型自身预测反馈的迹象。
时钟规则在流水线中仍可能出错。被屏蔽的小时会使部分预测特征缺失,而流水线可能不作任何说明就丢弃这些行,或填充这些小时。每次历史数据发生变化时,都应打印被屏蔽的小时及其影响的行。
汇总得分可能掩盖典型客户的情况。在 lgbm-feature-sets 中,未缩放的仅含滞后项的模型,其汇总 MAE 为 304.03 kWh,低于季节性朴素方法的 305.21 kWh,但它的各客户平均 MASE 为 1.115,而季节性朴素方法为 0.934。供应商为每个客户分别购电,因此决定某个客户预测好坏的是按客户计算的数字。只报告汇总 MAE 的供应商,会把错误的预测卖给那些被汇总数字掩盖的客户。应在汇总数字旁边报告各客户的 MASE,并先查看最差的客户。
结果还可能依赖于机器。LightGBM 的参数参考指出,在 CPU 设备类型上,对相同的数据和相同的参数,deterministic 能给出稳定的结果,并警告说,结果可能因 LightGBM 版本不同、由其他编译器编译的二进制文件不同或系统不同而有差异。本书在 LightGBM 4.7.0 下设置了 deterministic、按行直方图、单线程和固定种子。这些设置使输出在同一台机器、同一线程数下可重复:在记录机器上,三次运行的输出完全一致。它们并不保证不同机器之间一致,本书也没有对此进行测试。文中的得分是四舍五入后引用的,而四舍五入并不能使两台机器的结果一致。如果发布机器打印出不同的值,记录的输出需要重新记录,正文也需要作同样的检查。
工作准则: 每当数据或某个库发生变化时,都要重新核对各预测年份的节假日数量、被屏蔽的小时及其波及的行,以及各滞后项的长度。对每个滞后项运行平移检验,并结合汇总 MAE 阅读各客户的 MASE。某项检查的结果发生变化,就应当停下来,先查明原因,再发布下一次预测。
这部分尚未翻译,因此以英文显示。
Decisions this chapter made
The choices below are the ones a reader would have to make differently to change the results. Each one is the book's choice, open to revision when a later edition measures something else.
- The handed-on model uses lags 24 and 168 only. Lag 1 appears only in the leak check, never in a forecast.
- The calendar set is
hour,dayofweekandmonth. It is measured and not kept: on the per-client view it is worse than lags alone. There is no separate weekend flag, because the weekday carries the weekend. - The holiday flag is measured only on top of the calendar set, on all the decision windows. It is not kept, because the set that holds it is still worse than lags alone. Chapter 6 gets the lags-only set.
- Holidays come from the
holidayspackage for Portugal, for the two years in the data. Theis_holidayfunction lives inexamples/lib/features.py. - Each client's target is scaled by default, with
make_forecaster(scaled=True). The scaler is fitted on each window's history only. The unscaled run is printed once, to show the failure that scaling corrects. - Each window's masked clock-change hours are filled from that window's history before the fit, so no row is dropped for them and LightGBM never receives a missing value.
- The LightGBM settings are
LGBM_PARAMSinexamples/lib/features.py: deterministic, row-wise histograms, one thread and seed 0. They sit there, not insettings.py, which this chapter leaves unchanged. - The MASE scale is
scoring.book_scale. It starts at each client's first reading and reads the readings before the first decision window, Apr 18, 2014. - On the decision windows, the lags-only model beats seasonal naive on mean per-client MASE. The chapter does not tune the model to change that result.
- The feature sets are chosen on the decision windows, not on the standard cutoffs or the held-out week. The holiday windows end on Dec 8, so the holiday's effect on Dec 25 is not measured in this chapter. Whether calendar features pay on a longer history is also not measured here.
- The leak check uses
MT_014, one client, because the leak is a property of the lag and not of the client mix, so its target is not scaled. The feature sets useSUBSET8. - Chapter 6 reuses
make_forecaster,mlforecast_windows,fit_window_mae,lag_tableandis_holidayfromexamples/lib/features.py.make_forecasterscales by default. - The lesson is an order of tests: the shift test runs before any score, and the per-client MASE decides which features stay.
本章,在本版次中
版次 2026.11.11示例于 2026年11月20日 通过本章 2个示例 个示例中有 2 个通过
译自英文版图书,版次 2026.11.11。