設定模型必須超越的基準線

閱讀約需 66 分鐘 分鐘,7 節 中共有 2 個範例。

版次 2026.11.11 範例於 2026年11 月 20 通過測試 41 個範例中有 41 個通過 變更內容

由 AI 翻譯。經人工以英文驗證。 我們如何製作書籍

譯自英文書,版次 2026.11.11。 閱讀原文

在本章中,你會為某位客戶的隔日預測計算三種基準線,分別是天真法(naive)、季節性天真法(seasonal naive)與週平均法(average of weeks),並以 MASE 衡量該客戶的誤差,其尺度只取用預測週之前的讀數。

第 1 章結束時,留下了保留週上的兩個分數,但目前還無法評斷,因為以 kWh 表示的誤差本身並沒有意義。供應商的採購取決於每個小時的預測,因此必須將各模型的誤差,與最簡單的預測在相同時段上的得分相比,也要與該客戶的需求逐日變動的幅度相比。本章建立這兩種比較。供應商必須先有這兩項比較,才能說某個模型值得其成本。

每個誤差都是某一小時以千瓦小時計的落差,而這裡的分數是這些落差的逐小時平均。基準線的分數也是同一類數值。它說明最簡單的規則在一天中的平均每個小時會留下多少千瓦小時的誤差,而無法勝過這個數字的模型,就不值得多付出的成本。因此,每個分數除了與其他各列對照,也要與主要客戶的平均需求對照。

此部分尚未翻譯,因此以英文呈現。

Why a baseline comes first

最近一次變動於第 2026.11.11 版

A baseline is a forecast that needs no fitting. It copies a value the history already holds, so it costs almost nothing to run, and it sets a bar that any model has to clear. A model earns its place only by beating a forecast that cost nothing. Fitting time, tuning and keeping a model running are all costs, and the baseline's score measures what those costs buy.

Three terms matter here. The cutoff is the hour at which a forecast is made. The cutoff is the first hour the forecast covers, and the history is every reading before the cutoff. The horizon is how far ahead a forecast reaches, here 24 hours. A window is one cutoff together with the horizon it covers, and a backtest is the score of many windows, each made as it would have been made on its day.

The comparison has to be fair, or it measures the setup and not the model. A baseline scored on different hours, with a different history or a different scoring routine, says nothing about the model beside it. So every method in this chapter takes its cutoffs from held_out_cutoffs(), the seven daily cutoffs at 00:00 of the held-out week, reads its history through history_before(), which takes the eight weeks of hours before a cutoff, and is scored by the same code. These are the seven windows Chapter 1 used.

Working rule: Score every baseline on the same windows, histories and scoring code as the model it will be compared with, and count each baseline's missing forecasts before you read its score. A model that beats a baseline scored on different hours has not beaten it.

此部分尚未翻譯,因此以英文呈現。

Three baselines to beat

最近一次變動於第 2026.11.11 版

Each baseline copies the history in a different way. Naive carries the last hour it has seen forward across the whole day. Seasonal naive copies the day before, hour for hour. Average of weeks takes the mean of that hour on that weekday over the four weeks before.

Every rule covers the 24 hours after a cutoff. Call the cutoff T, the hour at which the forecast is made, and let k run from 0 to 23 across the hours the forecast covers. The three rules are these:

  • Naive repeats the last reading before the cutoff: forecast(T + k) = y(T - 1).
  • Seasonal naive repeats the reading one day earlier: forecast(T + k) = y(T + k - 24).
  • Average of weeks takes the mean of the same hour on the same weekday over the four weeks before: forecast(T + k) = (y(T + k - 168) + y(T + k - 336) + y(T + k - 504) + y(T + k - 672)) / 4.

The lags are in hours, so 168 is one week. Every reading a rule uses comes from before the cutoff: naive's is the hour before it, seasonal naive's are the 24 hours before it, and average of weeks reaches back one to four weeks. Take the window at 00:00 on Dec 25, 2014 as a worked case. Naive gives every hour of Dec 25 the reading at 23:00 on Dec 24, so the forecast is flat. Seasonal naive gives 00:00 on Dec 25 the reading at 00:00 on Dec 24, and so copies the whole previous day's shape. Average of weeks gives 00:00 on Dec 25 the mean of the 00:00 readings on Dec 18, Dec 11, Dec 4 and Nov 27. That mean smooths out one unusual day, but it also averages in any level change across the four weeks.

Each rule fails in a known way. Naive gives every hour of the day the same value, so it cannot follow the daily peak at all. Seasonal naive copies one day, so an unusual day is copied into the forecast for the next day, as the per-day errors later in this section show on the held-out week. Average of weeks needs four clean weeks, and a holiday in any of them enters the mean with the same weight as an ordinary day. A baseline's score therefore describes its rule on the windows it was run on, and these failures show up only on the windows where they happen.

Naive and seasonal naive come from statsforecast (2.1.1), whose classes Naive and SeasonalNaive are in statsforecast/models.py. The seasonal naive rule is the one Hyndman and Athanasopoulos give for seasonal data: the forecast is the last observed value from the same season. Average of weeks is not a library method, so examples/lib/baselines.py computes it in pandas, where the rule can be read, and each window reads only its own history. A lag missing from that history raises an error instead of being skipped. One script scores all three rules through those seven windows, and its first lines count the windows, scored hours and missing forecasts, so you can check them before you read a score.

naive-baselines-held-out-week.py Python 3.13.16 · numpy 2.5.5 · pandas 2.3.3 · statsforecast 2.1.1 · utilsforecast 0.2.17 通過
"""Score the three baselines for the lead client over the held-out week, on the same seven windows. The point rests on the missing-forecasts line, which must be zero for the scores to compare, and on the three MAE lines."""import pandas as pdfrom lib.backtest import held_out_cutoffs, score_windows, window_maefrom lib.baselines import BASELINES, baseline_forecastsfrom lib.electricity import loadfrom lib.settings import LEAD, showcutoffs = held_out_cutoffs()                                 # seven daily cutoffs at 00:00; every baseline uses these same windowsdf = load([LEAD])[["unique_id", "ds", "y"]]                  # unique_id, ds and y only: the baselines read no other columnforecasts = baseline_forecasts(df, cutoffs)                  # one row per forecast hour, one column per baseline; each window reads its own historylabels = ["naive", "seasonal naive", "average of weeks"]     # printed names, in the order of BASELINES, which the loops below zip withshow("windows", len(cutoffs))show("scored hours", len(forecasts))show("missing forecasts", int(forecasts[BASELINES].isna().sum().sum()))   # NaN marks a forecast that could not be made; the two sums count them over every columnshow("mean demand", df[df["ds"].isin(forecasts["ds"])]["y"].mean())       # the mean over the scored hours only, not over the whole historyfor model, label in zip(BASELINES, labels):    show(f"{label} MAE", score_windows(df, forecasts, model)["mae"])      # pooled over every scored hour; the result is keyed by metric namefor model, label in zip(BASELINES, labels):    per_window = window_mae(df, forecasts, model)                         # one MAE per window, keyed by the day its forecast hours fall on    show(f"{label} worst window MAE", per_window.max())    show(f"{label} worst window", per_window.idxmax().strftime("%b %d, %Y"))    for day in ["2014-12-25", "2014-12-26"]:                              # Dec 25 is the holiday and Dec 26 the day after: the two windows the prose reads        show(f"{label} MAE {pd.Timestamp(day):%b %d} window", per_window[pd.Timestamp(day)])   # windows are keyed by midnight of their day, so a Timestamp at 00:00 finds one
輸出python naive-baselines-held-out-week.py
windows: 7
scored hours: 168
missing forecasts: 0
mean demand: 46.62
naive MAE: 8.16
seasonal naive MAE: 8.36
average of weeks MAE: 8.96
naive worst window MAE: 11.01
naive worst window: Dec 30, 2014
naive MAE Dec 25 window: 5.84
naive MAE Dec 26 window: 10.36
seasonal naive worst window MAE: 12.16
seasonal naive worst window: Dec 26, 2014
seasonal naive MAE Dec 25 window: 9.41
seasonal naive MAE Dec 26 window: 12.16
average of weeks worst window MAE: 22.09
average of weeks worst window: Dec 25, 2014
average of weeks MAE Dec 25 window: 22.09
average of weeks MAE Dec 26 window: 12.33

The three scores are comparable only because they cover the same hours: 168 scored hours in 7 windows, one for each hour of the seven days, and 0 missing forecasts across the three rules. That count is the one the working rule in the first section asks for, and zero is the only value that lets the three be compared on the same hours.

Read each error below against the lead client's mean demand of 46.62 kWh. Each score below is a MAE, the mean absolute miss in kWh that Chapter 1 defines. The three baselines score:

  • naive: 8.16 kWh
  • seasonal naive: 8.36 kWh
  • average of weeks: 8.96 kWh

The simplest rule has the lowest error on this week, and average of weeks, which averages four weeks, is the weakest. Nothing here explains that order, and this is one client and one week, so it is a result for this week only. Chapter 3 is where the book backtests seasonal naive across many cutoffs.

The totals hide where the errors sit. The week begins on Dec 25, 2014, a Portuguese national holiday, and its first window is where the rules part company. Naive has the lowest error on it, at 5.84 kWh, against 9.41 kWh for seasonal naive. Average of weeks is worse still, and its largest error of the week falls on that window, at 22.09 kWh, because its four lags are ordinary days and the holiday has no match among them.

The day after the holiday is where copying costs seasonal naive. Its largest error of the week falls on Dec 26, 2014, at 12.16 kWh, because that window copies the holiday's pattern. On that day naive is still the lowest, at 10.36 kWh.

We recommend reporting all three baselines every time. Use seasonal naive as the main reference for MASE, because it carries the daily cycle that makes hourly demand predictable, and keep naive beside it, because on some weeks the flat rule is the stronger one.

Working rule: Start from naive and seasonal naive, both from the library, and add average of weeks when the series has a weekly rhythm worth testing. Check one forecast by hand against its formula before you rely on a baseline: pick one cutoff, compute one hour from the readings in its history, and compare the result with the forecast.

此部分尚未翻譯,因此以英文呈現。

Scale errors with MASE

最近一次變動於第 2026.11.11 版

A kilowatt-hour error means different things for different clients. A miss of 8.36 kWh an hour is small for a client that uses 46.62 kWh an hour, and the same miss would be large for a hypothetical client that uses 4 kWh an hour. Scaling the error removes that difference. A scaled error divides each absolute error by a scale, the typical error of a simple method on the same client's own history. MASE, the mean absolute scaled error, is the mean of the scaled errors. Hyndman and Koehler introduced it in a 2006 paper in the International Journal of Forecasting. The link is the authors' preprint, dated 2005. They noted that the scaled error does not depend on the units of the data, so the same number can be read across clients of very different size. This chapter scores one client, so that comparison across clients is the reason for the scale, not a result shown here.

The 2006 paper scales by the one-step naive error: the mean absolute difference between each reading and the one before it. That is the natural scale for non-seasonal series. For hourly demand, this book uses the seasonal form that Hyndman and Athanasopoulos give for seasonal data, because the comparison a reader wants is with seasonal naive: under the seasonal scale, the denominator is seasonal naive's error on the client's own history. A MASE below 1 then says the forecast's error on the scored hours is smaller than seasonal naive's error on that history. A one-step scale would compare it with a method this book does not use for hourly demand.

The two formulas are these:

  • scale = mean of |y(t) - y(t - 24)| over every reading t before the first test day, with the pairs that touch a masked clock-change hour left out
  • MASE = mean of |actual(t) - forecast(t)| / scale over the scored hours

The scale reads every reading before the first test day, 17352 hours, about two years. That is longer than each window's eight-week history, and a longer history gives a steadier estimate of the client's usual swing. One scale for the week also means every window is read against the same number. The function seasonal_scale in examples/lib/baselines.py computes it for one client or for many, and returns one value per client. To see whether the cut matters, score the three baselines on the held-out week twice: with the scale from history, and with the scale from the whole series, both with the clock-change readings masked.

mase-scale-leak-check.py Python 3.13.16 · numpy 2.5.5 · pandas 2.3.3 · statsforecast 2.1.1 · utilsforecast 0.2.17 通過
"""Score the lead client's held-out week in MASE with a scale from history and with one from the whole series, both with the clock-change readings masked. The point rests on the two last-reading lines, which show where each scale stops, and on the MASE ratio lines."""import pandas as pdfrom lib.backtest import held_out_cutoffsfrom lib.baselines import BASELINES, baseline_forecasts, mase_scores, scale_history, seasonal_scalefrom lib.electricity import loadfrom lib.quirks import mask_clock_changesfrom lib.settings import HELD_OUT_WEEK, LEAD, showLABELS = ["naive", "seasonal naive", "average of weeks"]    # printed names, in the order of BASELINESfirst_day = pd.Timestamp(HELD_OUT_WEEK[0])                   # the first test day: the history scale may read only readings before itfull = load([LEAD])                                          # every reading of the lead client, with the quirk column that marks clock-change hoursdf = full[["unique_id", "ds", "y"]]                          # the three columns the baselines and mase_scores read; quirk is for masking onlyforecasts = baseline_forecasts(df, held_out_cutoffs())       # the same windows and history rule as naive-baselines-held-out-week.pyhistory = scale_history(full, first_day)                     # keeps only rows strictly before the first test day, so no test hour reaches this scalemasked_history = mask_clock_changes(history)                 # clock-change hours set to NaN, so the pairs that touch them drop out of the meanmasked_whole = mask_clock_changes(full)                      # the same masking on the whole series, so the test week is the only differencetest_rows = len(full[full["ds"] >= first_day])               # readings from the first test day on: the test week's rowsshow("scale rows from history", len(history))show("last scale reading from history", history["ds"].max().strftime("%b %d, %Y %H:%M"))   # this must be the hour before the first test day; the line is the checkshow("scale rows from whole series", len(full))show("last reading of whole series", full["ds"].max().strftime("%b %d, %Y %H:%M"))show("test week share of whole-series rows in percent", 100 * test_rows / len(full), digits=3)show("clock-change rows in scale history", int(history["quirk"].ne("normal").sum()))   # every quirk but normal; for the lead client these are the clock-change hours, as it has no late startshow("test week own mean day-to-day difference", seasonal_scale(full[full["ds"] >= first_day - pd.Timedelta(hours=24)]).loc[LEAD], digits=4)   # starts 24 hours early, so its first pair is the first test day against the day beforeshow("scale from history as recorded", seasonal_scale(history).loc[LEAD], digits=4)          # seasonal_scale returns one value per client, so .loc picks the lead client; the clock-change pairs are kept hereshow("scale from history clock changes masked", seasonal_scale(masked_history).loc[LEAD], digits=4)   # pairs with a NaN are left out of the meanshow("scale from whole series clock changes masked", seasonal_scale(masked_whole).loc[LEAD], digits=4)hist = mase_scores(df, forecasts, BASELINES, masked_history).loc[LEAD]   # the scale comes from the rows passed as train, not from df; that choice is what this example testswhole = mase_scores(df, forecasts, BASELINES, masked_whole).loc[LEAD]for model, label in zip(BASELINES, LABELS):    show(f"{label} MASE from history scale", hist[model])    show(f"{label} MASE from whole-series scale", whole[model])    show(f"{label} MASE ratio whole-series over history", whole[model] / hist[model], digits=4)   # both MASEs share the same errors, so this ratio is the history scale over the whole-series scaleshow("seasonal naive MASE from history scale at four decimals", hist["SeasonalNaive"], digits=4)   # two decimals hide the leak, so the last two lines print fourshow("seasonal naive MASE from whole-series scale at four decimals", whole["SeasonalNaive"], digits=4)
輸出python mase-scale-leak-check.py
scale rows from history: 17352
last scale reading from history: Dec 24, 2014 23:00
scale rows from whole series: 17520
last reading of whole series: Dec 31, 2014 23:00
test week share of whole-series rows in percent: 0.959
clock-change rows in scale history: 4
test week own mean day-to-day difference: 8.3607
scale from history as recorded: 7.1121
scale from history clock changes masked: 7.1005
scale from whole series clock changes masked: 7.1126
naive MASE from history scale: 1.15
naive MASE from whole-series scale: 1.15
naive MASE ratio whole-series over history: 0.9983
seasonal naive MASE from history scale: 1.18
seasonal naive MASE from whole-series scale: 1.18
seasonal naive MASE ratio whole-series over history: 0.9983
average of weeks MASE from history scale: 1.26
average of weeks MASE from whole-series scale: 1.26
average of weeks MASE ratio whole-series over history: 0.9983
seasonal naive MASE from history scale at four decimals: 1.1775
seasonal naive MASE from whole-series scale at four decimals: 1.1755

The scale from history is 7.1005 kWh, the average gap between each hour and the same hour the day before, and it reads against the mean demand the way an MAE does. A scale that is a large share of the mean says the client's own day-to-day swing is large, and every error for that client is read against that swing. Dividing seasonal naive's MAE of 8.36 kWh by the scale gives its MASE, 1.18. utilsforecast's mase, reached through the helper mase_scores, computes this MASE by dividing the same MAE by the same seasonal scale (utilsforecast 0.2.17, utilsforecast/losses.py, function mase), so the division by hand gives the same figure.

utilsforecast (0.2.17) computes MASE with its mase function in utilsforecast/losses.py. The function takes its scale from the train_df it is given, and it shifts that frame by rows, not by time, so a missing hour moves every later lag and nothing reports it. It also needs sorted input. Its time filter runs only when the scored frame has a cutoff column (function _create_train_with_cutoffs). So the library does not decide which readings the scale may read. The caller does, and the next section is about that choice.

Working rule: Report MASE beside MAE whenever the clients you compare differ in size, and say in the same sentence which history the scale comes from. Read MASE as a ratio of two mean errors, one from the scored hours and one from the history. It is not a percentage gain over seasonal naive, even when the MASE is well below 1.

此部分尚未翻譯,因此以英文呈現。

Take the MASE scale from history

最近一次變動於第 2026.11.11 版

A scale describes the history before the first test day, not the series as a whole. If the scale reads the whole series, it also reads the hours the forecast is scored on, so the held-out week enters the denominator of its own error. That is a leak, and it raises no error. The MASE it produces looks like any other. The rule in this book is that a scale reads only the readings before the first test day of the week it scores, which for the held-out week is Dec 25, 2014.

A whole-series scale looks like the safer choice, because it uses more readings. It uses more readings from the wrong period. The extra readings are the ones the forecast is judged on, so the scale is no longer a measure of what was known at the cutoff. More data does not fix that. Only the cut does.

The script in the previous section prints the rows each scale reads, and those lines are the check. The history scale reads 17352 readings, and its last reading is Dec 24, 2014 23:00, the hour before the first test day. A later date there would mean test hours reached the scale.

The whole-series scale reads 17520 readings, through Dec 31, 2014 23:00, which covers the whole held-out week. Both scales have their clock-change readings masked, so the held-out week is the only difference between them.

The whole-series scale is the larger one, so every MASE computed with it is smaller. At two decimals both scales give seasonal naive 1.18, so the leak does not show. At four decimals it does, with seasonal naive's MASE at 1.1775 against 1.1755, and the factor below is the lesson, not the rounding.

The scales are 7.1005 kWh from history and 7.1126 kWh from the whole series, and the same factor, 0.9983, shrinks each of the three rules.

The leak is small on this week because the held-out week is a small share of the series. It makes up 0.959 percent of the whole-series readings, so the whole-series scale is roughly the history scale moved toward the held-out week's own swing by that share. The held-out week's own mean day-to-day difference is 8.3607 kWh, against 7.1005 kWh for the history, so the week swings more than the history does. The share times that gap is the leak, and the ratio above is its size. The leak grows in two ways. It grows when more of the scale's rows are test hours, as in a test period much longer than one week, such as the year of cutoffs in Chapter 3. It also grows when the held-out week's swing sits further from the history's. A quieter week would pull the whole-series scale down and make every MASE larger, so the same leak can flatter a model in one week and penalise it in another.

The clock changes are a second, smaller point, and this chapter handles them as Chapter 3 does. The history holds 4 clock-change readings: the two gap hours and the two repeat hours of the March and October changes in 2013 and 2014. The gap hours read zero, and the repeat hours hold the energy of two, as the data card describes; Chapter 3 shows the effect. Kept as recorded, those pairs give a scale of 7.1121 kWh. Masked, the scale is 7.1005 kWh, because the pairs that touch a masked reading drop out of the mean. A masked reading is neither zero nor a copy; it is missing. The chapter's scale masks them, and Chapter 3 gives the reason, which sets them to missing in every training set and every score.

Working rule: Pass a scale only the readings before the first test day, mask its clock-change readings, and print the last reading it uses. Check that this reading is the hour before the first test day. A small difference between the two scales does not show the procedure is right. The row check does.

解讀基準模型的分數

最近一次變動於第 2026.11.5 版

MASE 將預測值與客戶自身歷史資料上季節性樸素法(seasonal naive)的誤差相比較。低於 1 時,預測在被評分時段的誤差小於季節性朴素法在該歷史資料上的誤差;高於 1 時則較大。Hyndman 與 Koehler在論文中針對一步尺度說明了這種解讀方式,而季節尺度的算術是相同的。本書不提供好分數的門檻。何謂好,取決於預測失準對供應商造成的成本,這一點供應商清楚,本書則不得而知。

下表並列三個基準模型,每個一列。MAE 的單位為 kWh,MASE 使用已遮蔽時鐘變更的歷史尺度。上一節所述的洩漏在小數點後兩位並不會顯現,因此表中只有一欄 MASE。

基準模型 MAE(kWh) MASE
朴素法 8.16 1.15
季節性朴素法 8.36 1.18
週平均法 8.96 1.26

三個 MASE 值皆高於 1,因此每個基準模型的誤差都超過季節性朴素法在歷史資料上的一般誤差。就季節性朴素法本身而言,該週的逐日波動大於歷史資料的波動。其 MAE 等於保留週本身的逐日差異 8.3607 kWh,而歷史資料為 7.1005 kWh。至於朴素法與週平均法,尺度是拿它們與季節性朴素法比較,而不是與它們自己的過去比較,所以這張表並未說明這兩種規則在歷史資料上的表現。這一週以國定假日開始,而前面的一節顯示假日期間是週平均法誤差最大的時段。本書在此不檢驗其餘各日。

兩種指標下的順序相同:朴素法第一、季節性朴素法第二、週平均法第三。這種一致並不是一項發現。MASE 將每個 MAE 除以相同的尺度,因此對同一個客戶而言,MAE 與 MASE 的順序不可能不同。

這張表也帶出供應商的問題:最簡單的規則是否好到足以作為採購依據?這取決於每小時誤差的成本,而 MASE 並未替成本定價。MASE 給供應商的是一種可重複進行的比較。之後的模型可以用相同的輔助函式評分,使其 MASE 與這些列並排,而不必轉換單位。

閱讀此表請依下列順序。第一,由前面的一節確認每一列評分的是相同的時段,且沒有缺漏的預測。第二,將 MASE 欄與 1 對照閱讀。第三,將每一列與其他列相比,而不是與教科書上的某個數字相比。只與最佳一列相當的擬合模型並未證明什麼,而優於最佳一列的幅度小於所印出的四捨五入誤差者,則什麼也沒有證明。

實用原則: 將各基準模型的 MASE 與 1 對照,也與其他基準模型對照,不要依據單一週就指定優勝者。模型必須在不只一組時間窗上勝過最佳基準模型,且差距大於所印分數的四捨五入誤差,才算較佳。

正式環境中會出什麼問題

最近一次變動於第 2026.11.5 版

若在其他時段,或以不同的歷史資料為基準線評分,比較結果就不再代表它所宣稱的意義。團隊可能以一整年的截止點替模型評分,卻只用一週替基準線評分,或是給模型比基準線更長的歷史資料。這樣的比較衡量的是設定,而印出來的數字看起來卻像證據。檢查方式就是本章所採用的做法:每種方法的截止點都取自 held_out_cutoffs,歷史資料都取自 history_before,並且每次執行都印出缺漏預測的數量。

尺度中的洩漏會使 MASE 看起來與其他 MASE 無異,也不會引發任何錯誤。在本章中,它把每個季節性樸素預測(seasonal naive)的 MASE 都乘上 0.9983,這個變動被兩位小數掩蓋了。換成另一週出現更大的洩漏,在紙面上看起來也會一樣。檢查方式是印出的列數,以及尺度所使用的最後一筆讀數,兩者都在同一行輸出中,審閱者不必重新執行任何程式就能讀到。

季節性落後期(seasonal lag)中若有缺口,會產生錯誤的尺度,而且沒有任何輸出會標示出來。少了一個小時時,24 列的落後期會回溯到 25 小時之前,於是隔天的每一組配對都在比較錯誤的小時。函式庫的 mase 本身沒有缺口檢查。輔助函式 seasonal_scale 會檢查每位客戶的小時資料是否連續,而 mase_scores 在函式庫讀取尺度所用的列之前,會對這些列執行相同的檢查。這項檢查是引發錯誤,而不是使用斷言,因為 Python 在 -O 下會移除斷言。受評分的小時只檢查是否有讀數以及是否重複。正式執行應在計算任何尺度之前,先清點資料來源中的小時數,因為輔助函式內部的檢查是在問題已經進入尺度之後才回報。

季節性落後期也可能選錯週期。24 的落後期適用於具有日週期的每小時需求。日資料具有週週期,而帶有週節奏的每小時資料可能需要改用 168 的落後期。以錯誤季節計算出的尺度仍然是一個數字,所以輸出中看不出錯誤。檢查方式是在文字敘述中把季節標示在尺度旁邊,並在使用尺度之前對照資料的週期加以確認。

在保留週(held-out week)上挑選設定,會破壞比較。週平均法(Average of weeks)含有一項選擇:要平均幾週。如果這個數字是因為在保留週上得分最佳而選出,基準線就等於擬合了用來評判它的那一週,與模型的比較也就不再公平。本書在印出任何分數之前就固定為四週,本章對每個視窗都沿用這個值。若要更動,應列入本章末尾的決策清單,並在看到分數之前完成。

實用原則: 依下列順序檢查基準線的輸入及其尺度的輸入:每種方法使用相同的視窗、尺度只讀取較早的讀數並印出最後一筆讀數、季節性落後期使用連續不中斷的每小時歷史資料,以及季節與資料的週期相符。每項檢查都是一行輸出或一個計數,檢查未通過時,比較就應停止,直到問題修正為止。

本章的決定

最近一次變動於第 2026.11.5 版

本章做了七項選擇,最後一項是交接給第 3 章的部分。多數選擇都在examples/lib/baselines.py這個基準線輔助程式中,因此在那裡修正一次,就會影響所有呼叫它們的章。歷史長度與預測範圍是examples/lib/settings.py中的設定,與第 1 章所用的相同,所以這裡沒有任何基準線讀取的歷史,會與一旁的預測所用的不同。

  • MASE 的尺度採季節性,延遲為 24 小時。2006 年那篇論文所用的單步尺度則不採用。
  • 尺度是每位客戶一個數值,用於留出的那一週,取自 2014 年 12 月 25 日之前的每一筆讀數。如前文所述,它比每個視窗的歷史更長。當資料框帶有截止點欄位時,utilsforecast 可以改為每個截止點各取一個尺度,但本章不使用這個選項。若你要加入該選項,須知它的篩選條件保留ds <= cutoff,因此截止點那一小時本身也包含在內。本書中的截止點,是預測所涵蓋的第一個小時,該小時的讀數並不屬於歷史。因此,以本書的截止點建立的截止點欄位,必須先往前移一小時。
  • 週平均使用每個預測日之前的四週,以 pandas 計算,並且只讀取它自己的八週歷史。
  • Naive 與 seasonal naive 使用 statsforecast 的類別。分數是對該週每個被評分的小時合併計算,並印出至小數點後兩位。
  • 依照第 3 章的規則,尺度會遮蔽其中4筆時鐘變更的讀數。印出的歷史尺度隨後為7.1005 kWh。
  • 基準線不產生隨機數,因此範例不設定種子。
  • 第 3 章沿用 seasonal naive 與這些輔助程式,並在一年的星期日截止點上執行,每位客戶的尺度取自第一個截止點之前的每一筆讀數,也就是本章所訂的規則。只有一週的資料無法進行該項檢驗。

本章,於本版次中

版次 2026.11.11範例於 2026年11 月 20 通過測試本章 2 個範例 個範例中有 2 個通過

譯自英文書,版次 2026.11.11。

引用本節

格式

各種格式皆引用英文原書,本譯本即依循該原書。

每種格式都會標示本版。其連結會開啟目前版次,節標題會說明該節內文最後一次變動的時間。存取日期與擷取日期即為你開啟此頁的當日。

本頁內容

第 2 章,共 10 章: 設定模型必須超越的基準線

  1. Why a baseline comes first
  2. Three baselines to beat
  3. Scale errors with MASE
  4. Take the MASE scale from history
  5. 解讀基準模型的分數
  6. 正式環境中會出什麼問題
  7. 本章的決定

返回頂端所有章節