달력 특성과 지연 특성 추가하기
읽는 데 약 49분분 걸린다, 8절에 2개 예제이(가) 있음.
AI로 번역함. 영어 원문을 사람이 검증함. 우리가 책을 만드는 방식
영어판 2026.11.11판을 번역한 것이다. 원문 읽기
이 장을 마치면 한 고객의 시간별 이력을 특성 표로 바꾸고, 하루 앞 예측이 볼 수 있는 지연 특성과 달력 및 공휴일 특성을 추가하고, 같은 구간에서 각 특성이 더하는 효과를 측정하고, 예측 대상 날짜를 읽는 지연 특성을 찾아낼 수 있다.
4장의 통계 모델은 시계열만 본다. 이 모델은 시계열의 수준과 하루 주기만 알 뿐 다른 것은 알지 못한다. 2014년 12월 25일이 공휴일이었다는 것, 월요일이 일요일과 다르다는 것, 수요가 계절에 따라 움직인다는 것을 모른다. 1장은 그 대가가 무엇인지 보여 주었다. 계절 나이브 기준 모델은 공휴일의 측정값을 그대로 앞으로 복사했기 때문에 크리스마스 다음 날인 12월 26일에 가장 나쁜 하루를 기록했다. 그래디언트 부스팅 모델은 이런 사실을 열로 받아들일 수 있다. 공급자는 하루가 시작되기 전에 전력을 구매하므로, 기준 시점에 알려진 열만 예측에 들어갈 수 있다. 이 장에서는 고객 8곳의 부분 집합에 달력 열과 공휴일 열을 추가하고, 각 열을 반영하기 전에 하나씩 검사한다. 공휴일 플래그는 12월 26일의 실패를 바로잡지 못한다. 12월 26일은 공휴일이 아니기 때문이다. 이를 바로잡을 수 있는 특성인 공휴일 다음 날 플래그도 미리 알 수 있지만, 이 장에서는 만들지 않으며 표준 기준 시점들은 공휴일을 평가하지 않는다.
이 부분은 아직 번역되지 않았으므로 영어로 표시된다.
Turn history into features
The table comes first, because every feature in this chapter is a column of it. A feature table has one row for each hour a model must predict. Each row of a feature table holds the value to predict, which the book calls the target, and the features: the readings and facts the model may use for the row's hour. A gradient-boosting model learns a mapping from the columns to the target, one row at a time, so the table is all the model can know.
A row's features must be known at the cutoff of the window that forecasts it, because the forecast is made there. The target is the reading at the hour. The model needs two kinds of row. Training rows are the hours of the history before the cutoff, each with its own features and its own reading as the target. Forecast rows are the 24 hours after the cutoff, with features but no target. The model is fitted on the first kind and predicts the second.
Take the cutoff at 00:00 on Dec 30, 2014. Its forecast rows are the 24 hours of Dec 30, from 00:00 to 23:00, and each has a blank target. Its training rows run back eight weeks, to Nov 4, 2014, one row per client and hour, with the reading as the target. Take the row for Dec 15 at 18:00: its target is the reading at that hour, and its features are readings from earlier hours, such as the reading 24 hours before. No row is a daily total.
Each window's training table is built from the 1,344 hours of history before its cutoff, eight weeks, for each client in SUBSET8. The eight clients are stacked in one table, and the same construction repeats for every decision window and for the three standard cutoffs, Dec 29 to Dec 31, 2014. The first week of history has no reading a week earlier, so it leaves the table: 1,344 hours less 168 hours leaves 1,176 rows per client in each window. One model is fitted per window on that window's history alone, so no training row comes from the days being forecast.
For the lags-only set the columns are the two lags and the target y; the hour of the day arrives with the other calendar columns later in the chapter, and the holiday column after them. MLForecast in mlforecast 1.1.0 takes the series and produces the training rows in fit and the forecast rows in predict. Two of its defaults matter here. A training row whose features are missing is dropped (dropna=True, in mlforecast/core.py), and the first hours of each history drop out because they have no reading a week earlier. Nothing in the library stops a feature from reading the hour it predicts. The rule that stops it is the one you write and check.
Working rule: Build every feature row as if the cutoff were the last reading you have. A feature for a forecast hour must be computable from readings at or before the cutoff of its window. Before any shift test, ask whether the feature is known at the cutoff. Then check it with the shift test from Chapter 3, and do not train on a table you have not checked.
예측이 볼 수 있는 래그 추가
그 표에 가장 먼저 추가할 열은 래그, 즉 같은 고객의 이전 측정값이다. 시각 t에 대한 k시간 래그는 t에서 k를 뺀 시각의 측정값이다. 래그는 모델에 시계열의 리듬을 알려 주는 가장 값싼 방법이다. 24 래그는 어제 같은 시각의 측정값이고, 168 래그는 일주일 전 같은 시각의 측정값이다. 모델은 각 래그를 얼마나 신뢰할지 학습한다. 168 래그는 일주일 전을 읽으므로 같은 요일의 값이 되고, 따라서 요일 열이 없어도 주간 패턴을 담는다.
래그를 알 수 있는지는 예측 기간에 달려 있다. 00:00에 만든 예측은 기준 시점부터 24시간을 다룬다. 그날의 시각을 기준 시점부터 세어 0부터 23까지라고 하자. k 래그는 j에서 k를 뺀 시각을 읽으므로, 목표 시각 j에 대해 이 값이 음수일 때만 기준 시점 이전의 측정값을 읽는다. 이 조건이 0부터 23까지 모든 j에서 성립해야 하며, 그러려면 k는 24 이상이어야 한다. 1 래그는 첫 번째 시각에서는 알 수 있다(0시간). 기준 시점 이전의 마지막 측정값을 읽기 때문이다. 나머지 23개 시각에서는 예측이 가지고 있지 않은 측정값을 읽는다.
아래 표는 기준 시점이 00:00일 때 이 하루 앞 예측에서 목표 시각의 세 가지 경우를 보여 준다. 0시간은 기준 시점 직후의 첫 시각이고, 23시간은 마지막 시각이다.
| 목표 시각 | 1 래그가 읽는 값 | 기준 시점에 1 래그를 아는가 | 24 래그가 읽는 값 | 기준 시점에 24 래그를 아는가 |
|---|---|---|---|---|
| 00:00 (0시간) | 전날 23:00 | 예 | 전날 00:00 | 예 |
| 01:00 (1시간) | 예측일 00:00 | 아니요 | 전날 01:00 | 예 |
| 23:00 (23시간) | 예측일 22:00 | 아니요 | 전날 23:00 | 예 |
공급자의 두 래그인 24와 168은 하루 앞 예측의 모든 시각에서 알 수 있으므로, 모델은 아직 존재하지 않는 측정값을 필요로 하지 않는다. lgbm-feature-sets는 이 두 래그를 MLForecast에 전달하고, 나머지는 라이브러리가 처리한다.
3장의 시계 규칙은 래그에도 적용된다. 2014년 3월 30일의 공백 시각은 결측으로 설정되므로, 그 시각을 읽는 모든 피처도 결측이 된다. 2014년 4월 1일 기준 시점의 이력에서 lgbm-feature-sets는 마스킹된 시각 1개를 찾으며, 이를 읽는 래그를 가진 행은 1개다. 공백으로부터 24시간 뒤의 행이다. 각 윈도의 모델을 적합하기 전에, 그 윈도의 마스킹된 시각은 3장에서 채운 방식대로 해당 윈도 자체의 이력으로 채우며, 이 채우기는 기준 시점 이후의 값을 읽지 않는다. 채워진 시각은 하나의 측정값이 되므로, 그 시각을 읽는 행은 값을 그대로 유지하고 LightGBM은 래그로부터 결측값을 받지 않는다. 이 규칙은 측정으로 확인한 최적값이 아니라 명시한 선택이다. 독자가 확인해야 할 것은 규칙 옆에 출력된 개수다.
작업 규칙: 예측 기간 이상의 길이를 가진 래그만 사용한다. 예측 기간이 24시간이면 24시간 이상을 뜻하며, 이 책은 24와 168을 사용한다. 더 짧은 래그에는 근거가 필요하고, 그런 래그를 쓰는 모델은 점수를 신뢰하기 전에 시프트 테스트를 통과해야 한다. 래그에는 시계 규칙을 유지하고, 래그가 마스킹된 시각을 읽는 행의 수와 이력으로 채운 시각의 수를 센다.
이 부분은 아직 번역되지 않았으므로 영어로 표시된다.
Add calendar and holiday features
Lags are built from readings. Calendar features, unlike lags, are facts about the target hour itself: its hour of day, its day of the week and its month. They are known in advance, because the clock states them before any reading exists. They need no lag and no cutoff rule. Their value at a target hour cannot depend on any reading, so the shift test has nothing to catch.
The supplier's demand follows the day. The hour of day repeats what the lags already carry: the daily rhythm. The weekday adds what they do not. It tells the model that Saturday and Sunday differ from the working days. The month is the third column, and it is a weak one with this history. Eight weeks of history span two or three calendar months, so month can only tell a few weeks of the history from the rest, and on the standard cutoffs every forecast hour falls in December. The book uses three mlforecast date features, named by their attribute: hour, dayofweek and month. The weekday column already holds the weekend, so the book adds no separate weekend flag. In pandas, dayofweek counts Monday as 0 and Sunday as 6, so Saturday and Sunday are the two values a tree must separate from the rest. A tree can split on that in one step, which is why the weekday is cheap to test.
A public holiday does not follow the weekday. A model with only weekday features reads Dec 25, 2014 as an ordinary Thursday, and it reads a holiday on a Sunday as an ordinary Sunday. The holiday feature answers this. It is a function of the timestamp, is_holiday in the shared module examples/lib/features.py, which returns 1 on a Portuguese national holiday and 0 otherwise. mlforecast accepts such a function as a date feature and names its column after the function. The holidays come from the holidays package 0.110.dev20261118063242, under the MIT licence, through holidays.country_holidays("PT", years=[2013, 2014]). For the two years in the data, the package lists 18 dates. The book does not ship that list. Each run builds it from the package's calendar.
The standard cutoffs are Dec 29 to Dec 31, 2014, and none of their 72 hours per client is a holiday. The held-out week's Dec 25 is outside them, so the feature's effect on that day is not scored in this chapter. On the standard cutoffs, it reaches the scores only through the training history, which contains the holidays of Dec 8 and Dec 25, 2014.
The holiday list is a fact known in advance, like the weekday. It comes from a published calendar and not from the readings, so it cannot leak a reading from the target day. Calendar features answer the question in the working rule at every target hour, by construction, and a lag passes only when it is at least as long as the horizon.
Working rule: Add the calendar features the case's rhythm needs, and keep each one only if it helps on the same windows, as the next section measures. For holidays, use a calendar that covers every year in your data and every year you will forecast, and print how many holidays it holds for those years. A count of zero for a forecast year means the list is missing that year, not that the year had no holidays.
이 부분은 아직 번역되지 않았으므로 영어로 표시된다.
Choose on decision windows
Decision windows are the windows the feature set is chosen on, never the standard cutoffs, which are the windows the book reports. There are two kinds. The first is four ordinary cutoffs, Nov 25, Nov 26, Dec 2 and Dec 3, 2014. The second is seven holiday cutoffs: the Portuguese national holidays of 2014 before Dec 25 whose eight-week history is complete, Apr 18, Apr 20, Apr 25, May 1, Jun 10, Aug 15 and Dec 8. The ordinary windows hold no holiday, so a holiday feature could never win on them, and the holiday windows are where it can. The choice therefore uses all the decision windows, because one set holds a holiday feature. The standard cutoffs are scored after the choice and printed beside it. They choose nothing.
The holiday flag acts in two ways. On the decision windows it is on for scored hours, so it can change a forecast. On the standard cutoffs no holiday falls in a scored hour, so the flag changes those scores only through training.
Working rule: Choose a feature on windows that come before the held-out week and hold the kind of day the feature is for: a holiday feature needs holiday windows. The standard cutoffs are reported beside the choice and never used to make it.
이 부분은 아직 번역되지 않았으므로 영어로 표시된다.
Measure what each feature adds
A feature is worth keeping when it lowers the error on windows the model did not train on. To measure one, hold everything else fixed: the same windows, the same history, the same settings. Three feature sets are fitted, one at a time. The first has lags only. The second adds the calendar features. The third adds the holiday feature to them. Each set is fitted once per window, with one model for all eight clients of SUBSET8, and each is compared with seasonal naive on the same windows.
Each set gets two scores. The pooled MAE is the MAE over every client-hour in the windows, the single number for the whole supplier. The mean per-client MASE is MASE for each client, averaged over the eight clients. MASE divides each client's error by a scale: the mean absolute error that seasonal naive makes in sample on that client's own readings. The book uses one scale function everywhere, scoring.book_scale. The scale reads every reading before the first scored window of the set, Apr 18, 2014 at 00:00, with clock-change hours masked, so it does not depend on the windows scored and cannot leak. One client needs care. MT_115 starts reading on Jan 14, 2014, and the zeros before its first reading are left out of its scale, because it is counted from its first reading. Chapter 4 teaches that rule for late starters.
1"""Compare three feature sets for the global LightGBM forecast of SUBSET8 on the decision windows (B-5, B-9, B-10): four2ordinary cutoffs and seven 2014 holidays before Dec 25, each fitted scaled and unscaled. The mean MASE lines carry the3choice; the standard-cutoff lines are reported beside it and choose nothing."""4import pandas as pd56from lib.backtest import history_before7from lib.electricity import load8from lib.features import CALENDAR, PT_HOLIDAYS, is_holiday, rows_reading_missing, score_feature_sets9from lib.panel import DECISION_CUTOFFS, HOLIDAY_DECISION_CUTOFFS10from lib.quirks import mask_clock_changes11from lib.settings import EVAL_CUTOFFS, LEAD, SUBSET8, show1213LAGS = [24, 168] # both at least the 24-hour horizon, so each is known at every forecast hour14# CALENDAR is hour, dayofweek and month, as mlforecast date features; the holiday set adds is_holiday to them15FEATURE_SETS = {"lags_only": [], "with_calendar": CALENDAR, "with_holidays": CALENDAR + [is_holiday]}16MODELS = ["SeasonalNaive", *FEATURE_SETS, *[f"unscaled_{name}" for name in FEATURE_SETS]]17DECISION = sorted([*DECISION_CUTOFFS, *HOLIDAY_DECISION_CUTOFFS]) # the decision windows; standard cutoffs are not here18raw = load(SUBSET8)1920# DECISION[0] is the first decision window, Apr 18, 2014: the MASE scale reads only the readings before it21forecasts, scored, per_client = score_feature_sets(raw, DECISION, LAGS, FEATURE_SETS, DECISION[0])22show("clients", forecasts["unique_id"].nunique())23show("decision windows", len(DECISION))24show("scored hours", len(scored))25show("Portuguese holidays in 2013 and 2014", len(PT_HOLIDAYS))26show("holiday hours in the decision windows", int(is_holiday(scored["ds"]).sum()))27for model in MODELS:28 show(f"MAE {model.replace('unscaled_', 'unscaled ')}", (scored["y"] - scored[model]).abs().mean())29for model in MODELS:30 show(f"mean MASE {model.replace('unscaled_', 'unscaled ')}", per_client[model].mean(), digits=3)31for model in MODELS:32 # the prose does not quote this client, so this line is for reference only33 show(f"MASE MT_268 {model.replace('unscaled_', 'unscaled ')}", per_client.loc["MT_268", model], digits=3)3435# The standard cutoffs, reported beside the choice above. Nothing is chosen from them.36# The scale here reads the readings before the first standard cutoff, Dec 29, 2014.37_, _, standard = score_feature_sets(raw, EVAL_CUTOFFS, LAGS, FEATURE_SETS, EVAL_CUTOFFS[0])38for model in MODELS:39 show(f"standard cutoffs, mean MASE {model.replace('unscaled_', 'unscaled ')}", standard[model].mean(), digits=3)4041# Clock-change check: the history of a 1 April 2014 cutoff holds the 30 March gap, which the feature rules must reach.42march = history_before(mask_clock_changes(load([LEAD])), pd.Timestamp("2014-04-01")) # the Chapter 3 mask, then the cut43show("masked hours in the history", int(march["y"].isna().sum()))44# a row counts when either lag, 24 or 168, reads a masked hour45show("rows whose lag reads a masked hour", rows_reading_missing(march.set_index("ds")["y"], LAGS))
python lgbm-feature-sets.pyclients: 8
decision windows: 11
scored hours: 2112
Portuguese holidays in 2013 and 2014: 18
holiday hours in the decision windows: 1344
MAE SeasonalNaive: 305.21
MAE lags_only: 280.57
MAE with_calendar: 270.57
MAE with_holidays: 267.40
MAE unscaled lags_only: 304.03
MAE unscaled with_calendar: 271.63
MAE unscaled with_holidays: 280.75
mean MASE SeasonalNaive: 0.934
mean MASE lags_only: 0.886
mean MASE with_calendar: 0.894
mean MASE with_holidays: 0.893
mean MASE unscaled lags_only: 1.115
mean MASE unscaled with_calendar: 1.119
mean MASE unscaled with_holidays: 1.139
MASE MT_268 SeasonalNaive: 0.840
MASE MT_268 lags_only: 0.797
MASE MT_268 with_calendar: 0.758
MASE MT_268 with_holidays: 0.789
MASE MT_268 unscaled lags_only: 0.855
MASE MT_268 unscaled with_calendar: 0.871
MASE MT_268 unscaled with_holidays: 0.936
standard cutoffs, mean MASE SeasonalNaive: 1.162
standard cutoffs, mean MASE lags_only: 1.057
standard cutoffs, mean MASE with_calendar: 1.165
standard cutoffs, mean MASE with_holidays: 1.134
standard cutoffs, mean MASE unscaled lags_only: 1.357
standard cutoffs, mean MASE unscaled with_calendar: 1.494
standard cutoffs, mean MASE unscaled with_holidays: 1.486
masked hours in the history: 1
rows whose lag reads a masked hour: 1
Seasonal naive is the bar here. Its MAE, pooled over 2112 scored hours (eight clients in 11 windows of 24 hours each), is 305.21 kWh.
Without scaling, the lags-only model trails seasonal naive on the per-client view. Its mean per-client MASE is 1.115, worse than seasonal naive's 0.934. The clients differ a great deal in size, so the largest clients' readings are many times the smallest's. A global model fits one loss over every client's rows, and here that loss is squared error in kWh, so the largest clients' errors set the splits and the small clients carry little weight. We think the cause is the loss, not the range of any one client. The scaled runs below remove the difference, which supports that reading. It is a judgement from these numbers, not a proof.
The scaled runs standardise each client's target with mlforecast's LocalStandardScaler (mlforecast/target_transforms.py), which subtracts the client's mean and divides by its standard deviation. Both come from the window's history before the cutoff, fitted inside fit, and predict only inverts them. No reading after the cutoff sets them, so the scaling passes the same test as the features. make_forecaster takes scaled=True by default, and Chapter 6 reuses that default.
Scaled, the comparison changes, and lags alone beat seasonal naive on the per-client view: a mean MASE of 0.886, against 0.934. Its pooled MAE is 280.57 kWh.
The calendar set does not beat lags alone. It still beats seasonal naive: its mean MASE is 0.894, and its pooled MAE is 270.57 kWh.
The holiday flag, added to the calendar set, narrows the gap without closing it: the mean is 0.893, and the pooled MAE is 267.40 kWh.
Chapter 4 set AutoETS and ARIMA as the bar a gradient-boosting model must beat. On the standard cutoffs, lags alone also come in under that bar: 1.057 against ARIMA's 1.09, on the same clients, cutoffs and scale.
By the working rule, only the lags stay, and the per-client view decides it, because it is the view that matters to a client's forecast. The calendar set's mean MASE is above lags alone's, so it is not kept. The pooled view points the other way here: the calendar set's pooled MAE is below lags alone's, and the holiday set's is lower still. The chapter reports both, as the rule asks, and does not tune the model to make them agree.
The rule keeps a feature only when it lowers the error of the set it extends. The holiday flag lowers the calendar set's mean by a hair, too little to count as a gain, and the set that holds it is still above lags alone. The flag is on for 1344 client-hours, a full day on each holiday window, for every client, so it had its chance, and it does not earn a place. The book therefore hands Chapter 6 the lags-only set. That result comes from 11 windows, which is a small test.
Working rule: Keep a feature set only when it lowers the mean per-client MASE against the set it extends, on the same windows. Report the pooled MAE beside it. Do not tune the model to make a disagreement between the two views go away.
이 부분은 아직 번역되지 않았으므로 영어로 표시된다.
Catch a lag that leaks the future
The measurements above assume that each feature is known at its cutoff. A leak is a feature that uses a reading the forecast cannot have at its cutoff. In the book's view, a leak is a likely reason a backtest looks better than the model will be, and it can be built by accident. One pattern builds the feature table once from the whole series, so each target hour carries its lag, and then scores the table's forecast rows as if they were out of sample. Nothing fails. The score is too good, and nothing in it says so.
The library adds a second trap of its own. MLForecast predicts recursively by default. In recursive prediction, for a 24-hour horizon, MLForecast forecasts one hour, writes that forecast back as the reading for the next hour, and repeats (_predict_recursive in mlforecast/core.py). A lag of 1 therefore reads a forecast for every hour after the first, while training read actual readings for that lag. Calling that mismatch a leak is the book's judgement. It is one more reason to choose lags of 24 or more. With lags of 24 and 168, every step of the 24 reads actual readings, so the recursion never feeds this model its own forecasts. With max_horizon set, the library trains one model per horizon instead, and predicts with them in _predict_multi. The rule does not change with it: a feature must still be known at the cutoff of the window it forecasts.
The shift test from Chapter 3 carries over with one change: compare the lag at the target hours after the cutoff, not at the history rows. Compute the lag for each target hour twice: once from the history cut at the cutoff, and once from the whole series. Count the hours where the two differ. A lag of 24 agrees on every target hour, because each of the 24 hours reads a reading before the cutoff. A lag of 1 agrees only on the first hour.
Which do you expect to have the lower error, the model with a lag of 1 or the one without it? Both are fitted on the same training rows with the same settings, for each of the three standard cutoffs. The honest model uses lags 24 and 168. The leaky model adds a lag of 1, read from the whole series for the target hours, which is what the table built once would do.
1"""Shows which lag leaks: a lag of 1 reads the hour before each target hour, which a 24-hour forecast does not have at its2cutoff. The rows-differing lines are the shift test, and the two MAE lines show how much the leak flatters the score."""3import numpy as np4import pandas as pd56from lib.electricity import load7from lib.features import fit_window_mae, lag_values8from lib.settings import EVAL_CUTOFFS, HORIZON, LEAD, show910y = load([LEAD]).set_index("ds")["y"].asfreq("h") # asfreq gives every hour a row, so a missing hour is NaN, not skipped11HONEST, LEAKY = [24, 168], [1, 24, 168] # HONEST lags are both at least the horizon; LEAKY adds lag 112CUTOFFS = pd.to_datetime(EVAL_CUTOFFS)131415def shift_test(lag):16 """Target hours where the lag differs between the history cut at each cutoff and the whole series, and how many."""17 compared = differing = 018 for c in CUTOFFS:19 targets = pd.date_range(c, periods=HORIZON, freq="h") # the 24 hours each forecast must predict20 honest = lag_values(y[y.index < c], lag, targets) # readings before the cutoff only, as the forecast has them21 whole = lag_values(y, lag, targets) # the same lag read from the whole series, as a table built once would read it22 same = (honest == whole) | (np.isnan(honest) & np.isnan(whole)) # NaN never equals NaN, so the second term counts two missing readings as the same23 compared += len(targets)24 differing += int((~same).sum())25 return compared, differing262728show("cutoffs", len(CUTOFFS))29show("rows compared", shift_test(1)[0]) # the first value is the hours compared, the same for every lag30for lag in (1, 24, 168):31 show(f"rows differing for lag {lag}", shift_test(lag)[1])32show("MAE honest, lags 24 and 168", fit_window_mae(y, HONEST, CUTOFFS, leaky=False)) # the table holds the lags and the hour of day; leaky=False reads the target lags from the cut history33show("MAE leaky, lags 1, 24 and 168", fit_window_mae(y, LEAKY, CUTOFFS, leaky=True)) # leaky=True reads the whole series
python lag-leak-check.pycutoffs: 3
rows compared: 72
rows differing for lag 1: 69
rows differing for lag 24: 0
rows differing for lag 168: 0
MAE honest, lags 24 and 168: 5.36
MAE leaky, lags 1, 24 and 168: 2.87
Only the lag of 1 fails the shift test. Of the 72 target hours compared, 69 read a different value from the cut history than from the whole series: every hour of each window but the first. The lags of 24 and 168 differ on 0 and 0 rows. The lag of 1 agrees only on the first hour of each window, because that hour reads the last reading before the cutoff, which the forecast does have.
The leaky model looks almost twice as good: its MAE is 2.87 kWh, against 5.36 kWh for the honest model. The gap is the leak. Its lag of 1 reads the hour before the target, which for most target hours is an hour of the forecast window itself. The honest model never sees that hour, so its error is the one that production will show.
Working rule: Run the shift test on every lag and every window feature before a model trains on it, and treat any differing row as a leak until you have shown otherwise. When an error falls sharply after you add a feature, check that feature first, before you credit it.
운영에서 무엇이 잘못되는가
데이터 끝에서 멈춘 휴일 목록은 그 뒤의 모든 휴일을 0으로 읽는다. 그러면 예측 연도의 모든 날에 휴일 플래그가 꺼진 채로 남고, 모델은 휴일을 평범한 날로 예측한다. 첫 휴일은 대시보드에서 큰 오차로 나타난다. 예제가 출력하는 휴일 수가 점검 수단이다. 새 연도의 목록에서 휴일 수가 0으로 출력되면 그 연도가 목록에서 빠진 것이다. holidays의 이후 버전이 달력을 바꿀 수도 있으므로, 이 책처럼 버전을 고정하고 업그레이드할 때마다 휴일 수를 다시 확인한다.
시간 단위 모델에서 가져온, 예측 구간보다 짧은 래그를 하루 앞 예측에 쓰면 백테스트에서는 점수가 좋게 나온다. 백테스트는 전체 시계열로 만들어졌기 때문이다. 운영에서는 뒤쪽 시간대가 모델 자신의 예측값을 읽는다. 하루 앞 예측의 실제 오차를 시간대별로 구하여 백테스트 자체의 시간대별 오차와 비교한다. 누수가 없는 모델이라면 각 시간대에서 백테스트가 보인 오차와 비슷한 수준이어야 한다. 이를 넘어 예측 구간의 시간이 뒤로 갈수록 오차가 커진다면, 모델 자신의 예측이 되먹임되고 있다는 신호이다.
시계 규칙도 파이프라인에서는 여전히 어긋날 수 있다. 가려진 시간대 때문에 일부 예측 피처가 결측이 되는데, 파이프라인은 그 행을 알리지 않고 버리거나 그 시간대를 채울 수 있다. 이력이 바뀔 때마다 가려진 시간대와 그 영향을 받는 행을 출력한다. 이 점검은 조용히 지나가는 일이 없어야 한다.
통합 점수는 전형적인 고객을 가릴 수 있다. lgbm-feature-sets에서 스케일링하지 않은 래그 전용 모델의 통합 MAE는 304.03 kWh로 계절 단순 예측의 305.21 kWh보다 낮지만, 고객별 MASE의 평균은 1.115이고 계절 단순 예측은 0.934이다. 공급자는 고객마다 전력을 구매하므로, 한 고객의 예측을 좌우하는 것은 고객별 수치이다. 통합 MAE만 보고하는 공급자는 그 통합 수치에 가려진 고객들에게 잘못된 예측을 제공하게 된다. 통합 수치 옆에 고객별 MASE를 함께 보고하고, 가장 나쁜 고객부터 살핀다.
결과는 기계에 따라 달라지기도 한다. LightGBM의 매개변수 레퍼런스에 따르면 deterministic는 CPU 디바이스 유형에서 같은 데이터와 같은 매개변수에 대해 안정적인 결과를 내며, 다른 LightGBM 버전, 다른 컴파일러로 만든 바이너리, 다른 시스템에서는 결과가 달라질 수 있다고 경고한다. 이 책은 LightGBM 4.7.0에서 deterministic, 행 단위 히스토그램, 단일 스레드, 고정 시드를 설정한다. 이 설정은 같은 기계에서 같은 스레드 수로 실행할 때 출력을 재현 가능하게 만든다. 기록용 기계에서 세 번 실행한 출력은 모두 같았다. 그러나 기계 사이의 일치까지 보장하지는 않으며, 책은 이를 검증하지 않았다. 점수는 반올림하여 인용했지만, 반올림이 두 기계의 일치를 보장하지는 않는다. 배포용 기계가 다른 값을 출력하면 기록된 출력을 새로 기록해야 하고, 본문도 같은 방식으로 다시 확인해야 한다.
작업 규칙: 데이터나 라이브러리가 바뀔 때마다 예측 연도별 휴일 수, 가려진 시간대와 그 영향을 받는 행, 래그 길이를 다시 확인한다. 모든 래그에 이동 검정을 실행하고, 통합 MAE와 함께 고객별 MASE를 읽는다. 점검 결과가 달라졌다면 다음 예측을 공표하기 전에 멈추고 원인을 읽어야 한다.
이 부분은 아직 번역되지 않았으므로 영어로 표시된다.
Decisions this chapter made
The choices below are the ones a reader would have to make differently to change the results. Each one is the book's choice, open to revision when a later edition measures something else.
- The handed-on model uses lags 24 and 168 only. Lag 1 appears only in the leak check, never in a forecast.
- The calendar set is
hour,dayofweekandmonth. It is measured and not kept: on the per-client view it is worse than lags alone. There is no separate weekend flag, because the weekday carries the weekend. - The holiday flag is measured only on top of the calendar set, on all the decision windows. It is not kept, because the set that holds it is still worse than lags alone. Chapter 6 gets the lags-only set.
- Holidays come from the
holidayspackage for Portugal, for the two years in the data. Theis_holidayfunction lives inexamples/lib/features.py. - Each client's target is scaled by default, with
make_forecaster(scaled=True). The scaler is fitted on each window's history only. The unscaled run is printed once, to show the failure that scaling corrects. - Each window's masked clock-change hours are filled from that window's history before the fit, so no row is dropped for them and LightGBM never receives a missing value.
- The LightGBM settings are
LGBM_PARAMSinexamples/lib/features.py: deterministic, row-wise histograms, one thread and seed 0. They sit there, not insettings.py, which this chapter leaves unchanged. - The MASE scale is
scoring.book_scale. It starts at each client's first reading and reads the readings before the first decision window, Apr 18, 2014. - On the decision windows, the lags-only model beats seasonal naive on mean per-client MASE. The chapter does not tune the model to change that result.
- The feature sets are chosen on the decision windows, not on the standard cutoffs or the held-out week. The holiday windows end on Dec 8, so the holiday's effect on Dec 25 is not measured in this chapter. Whether calendar features pay on a longer history is also not measured here.
- The leak check uses
MT_014, one client, because the leak is a property of the lag and not of the client mix, so its target is not scaled. The feature sets useSUBSET8. - Chapter 6 reuses
make_forecaster,mlforecast_windows,fit_window_mae,lag_tableandis_holidayfromexamples/lib/features.py.make_forecasterscales by default. - The lesson is an order of tests: the shift test runs before any score, and the per-client MASE decides which features stay.
이 판의 이 장
2026.11.11판예제 통과 2026년 11월 20이 장에서 2개 예제개 중 2개가 통과한다
영어판 2026.11.11판을 번역한 것이다.