Comparación de Prophet y un modelo fundacional

Unos 24 minutos de lectura, con 6 ejemplos en 7 secciones.

Edición 2026.11.11 Ejemplos verificados el 20 de nov de 2026 41 de 41 ejemplos pasan Qué ha cambiado

Traducido con IA. Verificado por personas en inglés. Cómo hacemos los libros

Traducido del libro en inglés, edición 2026.11.11. Lee el original

Al terminar este capítulo, podrás ajustar Prophet con los tres cortes estándar del libro, cargar un modelo fundacional registrado (Chronos-2) y un modelo neuronal registrado (NHITS), y evaluar todos ellos en una misma escala MASE frente a las mismas lecturas.

Los lectores se encuentran con Prophet y los modelos neuronales de pronóstico, cada uno con sus propias afirmaciones. Este capítulo pregunta qué hace cada uno con los clientes del proveedor, en los cortes estándar, del 29 al 31 de diciembre de 2014. El proveedor necesita algo más acotado de cualquier modelo: un pronóstico a un día vista para cada cliente y un rango sobre el que pueda comprar. El capítulo 8 calibró esos rangos en ventanas anteriores. Los rangos aquí no se calibran de esa manera, así que este capítulo comprueba su cobertura, pero no la corrige. Cada modelo se compara con el naive estacional del capítulo 2 y con el modelo global del capítulo 6, sobre las mismas horas y en una misma escala MASE. Dos de los modelos se presentan registrados en lugar de ejecutarse.

Esta parte aún no está traducida, así que está en inglés.

Where Prophet fits

Último cambio en la edición 2026.11.11

Prophet is the first model a reader tries beyond the baselines, so we test it first, and its assumptions decide where it can help.

Prophet writes a series as a trend, plus Fourier seasonal terms, plus holiday effects, plus an error term. The authors describe it as a modular regression model whose parameters an analyst can read and adjust (Taylor and Letham (2018)). By default Prophet's fit is a maximum a posteriori estimate, an optimisation rather than a set of sampled draws. The code of version 1.5.0 sets that default in its optimisation call (models.py, lines 165 and 172), and its mcmc_samples argument defaults to 0 (forecaster.py, line 144). Both are in the Prophet source distribution for the pinned version. Each client gets its own fit, so Prophet does not learn from the other clients.

Prophet does several things well. It needs no interpolation of missing observations: the code drops rows whose y is missing before the fit (forecaster.py, line 1335). The clock-change hours are missing in the history by the rule in Chapter 3, so Prophet leaves them out without any extra code. Prophet takes holidays as a table of dates and effects, and the table this chapter tests holds the Portuguese national holidays for 2013 and 2014. The authors report that, on their own series with default settings, Prophet had lower error than auto.arima, ETS, seasonal naive and TBATS (Taylor and Letham (2018)). That result rests on their data, so we cite it as the authors' result and do not use it as evidence about this case.

Several defaults get in the way here. Yearly seasonality turns on only with at least 730 days of history (forecaster.py, line 1187), so it stays off with eight weeks. The default interval is 80%, not 95% (forecaster.py, line 145). It covers trend uncertainty and observation noise, not the seasonal pattern, unless the model is fitted with full sampling (Prophet documentation). Its coverage therefore makes a narrower claim than the word "interval" suggests. Prophet also assumes that the future trend changes as often, and by as much, as it did in the history (Prophet documentation). With eight weeks of history there is little to learn trend changes from, so in our reading that assumption carries most of the forecast.

Working rule: Use Prophet per client when a reader needs its trend and seasonal settings as parameters they can read, and when its scored forecasts beat seasonal naive on the decision windows you chose before scoring. Set its interval width in code, and read its default 80% interval as an 80% interval, never as a 95% range. Check its coverage against the level it claims.

Esta parte aún no está traducida, así que está en inglés.

Run Prophet on the same cutoffs

Último cambio en la edición 2026.11.11

The last section named Prophet's assumptions. This one fixes its settings and runs it on the same hours as every other model.

The comparison uses the three standard cutoffs, at 00:00 on Dec 29, Dec 30 and Dec 31, 2014. These are the windows Chapters 4 to 8 score. Each window's history is the 1,344 hours, eight weeks, before its cutoff, with the start rule applied, so a late starter's history begins at its first reading. Prophet is fitted once for each client and cutoff, which makes 90 fits.

We chose the holiday table on decision windows, not on the standard cutoffs. The ordinary decision windows hold no holiday in their scored hours, so they cannot test a holiday table at all, while the holiday decision windows can. So the choice uses both kinds. A run with the table and one without it shows what the table does on those windows for SUBSET8: the four ordinary days and seven holidays, Apr 18, Apr 20, Apr 25, May 1, Jun 10, Aug 15 and Dec 8, 2014. All of them come before the held-out week, and none has Christmas Day in its history.

prophet-holiday-decision.py Python 3.13.16 · holidays 0.110.dev20261118063242 · lightgbm 4.7.0 · mlforecast 1.1.0 · numpy 2.5.5 · pandas 2.3.3 · prophet 1.5.0 · statsforecast 2.1.1 · utilsforecast 0.2.17 Superado
"""Prophet with and without the Portuguese holiday table, on the eleven decision windows for SUBSET8 (B-9, B-10),before any standard cutoff is scored. The point that the table stays off rests on the two mean MASE lines and onholiday table kept, which says whether the table lowers the mean per-client MASE."""from lib.compare import prophet_windows, seasonal_naive, scorefrom lib.electricity import loadfrom lib.features import is_holidayfrom lib.panel import DECISION_CUTOFFS, HOLIDAY_DECISION_CUTOFFSfrom lib.quirks import mask_clock_changesfrom lib.settings import SUBSET8, show# the four ordinary and seven holiday windows, in date orderDECISION = sorted([*DECISION_CUTOFFS, *HOLIDAY_DECISION_CUTOFFS])raw = load(SUBSET8)# clock-change hours set to missing, so Prophet fits without themdf = mask_clock_changes(raw)[["unique_id", "ds", "y"]]keys = ["unique_id", "ds"]# one fit per client and window, with the Portuguese holidays as Prophet's holiday table; two processes, and the same# fits at any countwith_holidays = prophet_windows(df, DECISION, use_holidays=True, workers=2)[keys + ["Prophet"]].rename(    columns={"Prophet": "WithHolidays"})without = prophet_windows(df, DECISION, use_holidays=False, workers=2)[keys + ["Prophet"]].rename(    columns={"Prophet": "NoHolidays"})forecasts = with_holidays.merge(without, on=keys, validate="one_to_one").merge(    seasonal_naive(df, DECISION), on=keys, validate="one_to_one")models = ["WithHolidays", "NoHolidays", "SeasonalNaive"]# per_client: one MASE per client and model; scored: forecasts beside readingsper_client, scored = score(raw, forecasts, models, SUBSET8, DECISION)show("decision windows", len(DECISION))show("holiday decision windows", len(HOLIDAY_DECISION_CUTOFFS))show("clients", len(SUBSET8))show("scored hours", len(scored))# 1 for each hour on a Portuguese national holidayshow("holiday hours in the decision windows", int(is_holiday(scored["ds"]).sum()))for model in models:    # the mean of the per-client MASEs, not a pooled error    show(f"mean MASE {model}", per_client[model].mean(), digits=3)show("hours where the forecasts differ", int((scored["WithHolidays"] != scored["NoHolidays"]).sum()))show("clients where holidays help", int((per_client["WithHolidays"] < per_client["NoHolidays"]).sum()))# the table is kept only on a strictly lower mean; a tie keeps the simpler model, with the table offshow("holiday table kept", "yes" if per_client["WithHolidays"].mean() < per_client["NoHolidays"].mean() else "no")
Salidapython prophet-holiday-decision.py
decision windows: 11
holiday decision windows: 7
clients: 8
scored hours: 2112
holiday hours in the decision windows: 1344
mean MASE WithHolidays: 1.070
mean MASE NoHolidays: 1.061
mean MASE SeasonalNaive: 0.934
hours where the forecasts differ: 768
clients where holidays help: 2
holiday table kept: no

On the decision windows the table makes Prophet worse. Over the eleven of them, its mean per-client MASE is 1.070 with the table and 1.061 without it, against 0.934 for seasonal naive.

A setting is kept only if it lowers the decision-window MASE, and a tie keeps the simpler model. The example's verdict on keeping the table is no, so the table stays off.

The holiday windows hold 1344 client-hours of holiday, and they are the only place the table can change a forecast. 768 of the 2112 scored hours differ between the two variants.

The standard cutoffs tell a different story, which we report without using it to choose. Their histories include Christmas Day, and the holiday windows' histories do not. That is a likely reason for the gap, but we did not test which holiday causes it. With the table on, Prophet's mean per-client MASE on the standard cutoffs would be 1.27, against 1.35 with it off. A team that wants the table should test it on its own decision windows before it scores anything.

Daily and weekly seasonality are on, and yearly is off. These are the choices Prophet makes for itself on eight weeks of hourly data: yearly needs at least 730 days, weekly at least two weeks, and daily at least two days of sub-daily data (forecaster.py, lines 1187, 1209 and 1223). We set them explicitly and did not tune them. The interval is 95%, set in code and printed as Prophet interval level, so coverage sits beside the recordings' 95% ranges. Prophet draws its interval samples from numpy's global generator, so the code sets the seed (SEED) before each predict. The fit itself is deterministic.

Every model is scored on the book's one MASE scale: the seasonal naive error in sample, over every reading before Dec 29, 2014, with the clock-change hours masked. A MASE below 1 means the model's error on the scored hours is smaller than seasonal naive's in-sample error. Each client has its own scale, so a large client does not outweigh a small one. The next run fits Prophet for all thirty clients and scores it beside seasonal naive and the global model on this scale, which is the comparison every later result in the chapter uses.

prophet-thirty-clients-backtest.py Python 3.13.16 · holidays 0.110.dev20261118063242 · lightgbm 4.7.0 · mlforecast 1.1.0 · numpy 2.5.5 · pandas 2.3.3 · prophet 1.5.0 · statsforecast 2.1.1 · utilsforecast 0.2.17 Superado
"""Prophet for each of the thirty clients on the three standard cutoffs, with a 95% interval, scored on the book's MASEscale beside seasonal naive and the Chapter 6 global model. The point that Prophet's mean MASE is higher rests onthe mean MASE lines, and the coverage line says whether its 95% interval holds."""import pandas as pdfrom lib.compare import (PROPHET_HOLIDAYS, PROPHET_LEVEL, global_forecasts, prophet_windows, report_range,                         report_subset, score, seasonal_naive)from lib.electricity import loadfrom lib.panel import ALL30from lib.quirks import mask_clock_changesfrom lib.scoring import report_scoresfrom lib.settings import EVAL_CUTOFFS, showraw = load(ALL30)# clock-change hours set to missing, as chapter 3 sets themdf = mask_clock_changes(raw)[["unique_id", "ds", "y"]]cutoffs = pd.DatetimeIndex(EVAL_CUTOFFS)# Prophet at the book's settings from lib.compare: holiday table off, 95% interval, one fit per client and cutoffforecasts = prophet_windows(df, cutoffs).merge(    seasonal_naive(df, cutoffs), on=["unique_id", "ds"], validate="one_to_one").merge(    global_forecasts(df, cutoffs), on=["unique_id", "ds"], validate="one_to_one")models = ["Prophet", "SeasonalNaive", "Global"]# per_client: one MASE per client and model; scored: forecasts beside readingsper_client, scored = score(raw, forecasts, models, ALL30)show("Prophet interval level", PROPHET_LEVEL)show("Prophet holiday table", "on" if PROPHET_HOLIDAYS else "off")show("clients scored", scored["unique_id"].nunique())show("scored hours", len(scored))show("clients where Prophet beats seasonal naive", int((per_client["Prophet"] < per_client["SeasonalNaive"]).sum()))show("clients where Prophet beats the global model", int((per_client["Prophet"] < per_client["Global"]).sum()))report_scores(scored, per_client, models)report_subset(per_client, models)# the 95% range from Prophet's lower and upper interval columnsreport_range(scored, "Prophet", "ProphetLo", "ProphetHi", PROPHET_LEVEL)# Sensitivity only, not used to choose: the holiday table on the same cutoffs, which the decision windows# did not favour.# two processes; each fit is the same at any worker counton = prophet_windows(df, cutoffs, use_holidays=True, workers=2)[["unique_id", "ds", "Prophet"]].rename(    columns={"Prophet": "ProphetHolidays"})per_holiday, _ = score(raw, forecasts.merge(on, on=["unique_id", "ds"], validate="one_to_one"),                       ["ProphetHolidays"], ALL30)show("mean MASE Prophet with holidays, not chosen", per_holiday["ProphetHolidays"].mean())
Salidapython prophet-thirty-clients-backtest.py
Prophet interval level: 95
Prophet holiday table: off
clients scored: 30
scored hours: 2160
clients where Prophet beats seasonal naive: 9
clients where Prophet beats the global model: 6
MAE Prophet: 182.44
MAE SeasonalNaive: 148.26
MAE Global: 122.81
RMSE Prophet: 764.88
RMSE SeasonalNaive: 780.48
RMSE Global: 678.81
mean MASE Prophet: 1.35
mean MASE SeasonalNaive: 1.18
mean MASE Global: 0.91
mean MASE Prophet SUBSET8: 1.39
mean MASE SeasonalNaive SUBSET8: 1.16
mean MASE Global SUBSET8: 1.03
coverage 95 Prophet: 0.918
interval score 95 Prophet: 932.66
mean MASE Prophet with holidays, not chosen: 1.27

On these cutoffs Prophet has a higher mean per-client MASE than seasonal naive: 1.35 over the thirty clients, against 1.18 for seasonal naive and 0.91 for the global model.

The loss is on most clients too, not only on average: Prophet beats seasonal naive on only 9 of the thirty clients, and the global model on 6.

Pooled over every scored hour, Prophet's RMSE is 764.88 kWh, against 780.48 for seasonal naive, so it makes fewer very large misses. It loses to seasonal naive on MAE and MASE, and to the global model on every measure. Its 95% interval covers 0.918 of the readings, and its interval score is 932.66. Coverage below 0.95 means the interval is too narrow for its stated level on these hours.

Working rule: Set Prophet's seasonality, holidays and interval width in code before you score it, and choose any of them on decision windows, never on the standard cutoffs. Keep the holiday table only if it lowers the MASE on the holiday decision windows. Compare Prophet with seasonal naive and the global model on the same windows and the same scale, and list the clients it loses. Set the seed before each predict, and check that a second run prints the same lines.

Esta parte aún no está traducida, así que está en inglés.

Load a recorded foundation model's forecasts

Último cambio en la edición 2026.11.11

Prophet fits each client from its own eight weeks. A foundation model brings what it learned from other series, and this section shows how the book holds its forecasts.

A foundation model is trained once on many series and then applied to a context, with no fitting on that series. Chronos-2 is one such model. Chronos-2 is an encoder-only transformer that takes a context of past values and outputs quantiles for the next steps (Ansari et al. (2025); the earlier Chronos model is described in Ansari et al. (2024)). Its weights are published as amazon/chronos-2 on Hugging Face under the Apache-2.0 licence, not gated (model card).

Its pretraining covers this dataset, which limits how far its score can be read. The Chronos-2 report lists the electricity dataset in its pretraining corpus. Appendix A, Table 6 gives "Electricity", with 370 series in the energy domain (Ansari et al. (2025)). Those series match the UCI ElectricityLoadDiagrams20112014 dataset the book reads, and the book's thirty clients are drawn from them. The report notes partial overlap with some GIFT-Eval training portions. For strictly zero-shot results it points to a variant trained only on synthetic data, in its Section 5.4. We have not run that variant. The report does not show which readings of these clients the model saw, so the scores below are upper bounds, not out-of-sample results.

We do not run Chronos-2 in an example. Its forecasts are recorded: the model ran once, outside the examples, on a CPU, with the revision pinned to 29ec3766d36d6f73f0696f85560a422f50e8498c, and the examples read the saved output. The recording is dated Nov 10, 2026. Examples must run offline and give the same output every time, and a model call does neither. A downloaded model can change, and a reader's machine may not hold the files at all.

The input for each client and cutoff is the 1,344 hours before the cutoff, with the start rule applied. None of the standard cutoffs needs that step. Their histories begin on Nov 3, 2014 or later, after the autumn clock change, and the three late starters had their first readings before Nov 3, 2014. The manifest counts no missing values in the contexts, so the clock-change handling changes no scored number here. The package builds a mask from missing values and scales around them (chronos-forecasting source, chronos2/model.py, line 398). The 90 contexts went in one batch, with each series kept separate. Cross-learning, which would let one series inform another, is off by default, and the recording does not turn it on (chronos2/pipeline.py, line 833). So the Dec 30 context does not inform the Dec 29 forecast.

The output is, for each client, cutoff and hour, five quantiles for the next 24 hours: 0.025, 0.1, 0.5, 0.9 and 0.975. That is 2160 rows in all. The manifest lists the SHA-256 of the output and of inputs.csv.gz, which keeps the exact context each window was given, along with the package versions and the machine, described only as a 4-vCPU Linux container with no GPU.

Not every requested quantile is trained. The package is trained on 21 levels, from 0.01 to 0.99 (Ansari et al. (2025)). The levels 0.1, 0.5 and 0.9 are among them, so they are read directly. The levels 0.025 and 0.975 are not, so the package interpolates each one linearly between the two nearest trained levels: 0.01 and 0.05 for the lower, and 0.95 and 0.99 for the upper. The code is interpolate_quantiles in chronos/utils.py, called from predict_quantiles in chronos2/pipeline.py (chronos-forecasting source). The manifest names the interpolated levels, and a 95% range built from them is partly the package's interpolation.

Before any score reads the recording, run the check. It hashes each output and compares it with manifest.json, counts the rows, clients and cutoffs in the forecast file, and looks for quantiles that run out of order.

recorded-chronos2-check.py Python 3.13.16 · holidays 0.110.dev20261118063242 · lightgbm 4.7.0 · mlforecast 1.1.0 · numpy 2.5.5 · pandas 2.3.3 · prophet 1.5.0 · statsforecast 2.1.1 · utilsforecast 0.2.17 Superado
"""Checks the recorded Chronos-2 outputs before any score reads them: each file's SHA-256 against manifest.json, theshape of the forecast file, and any quantiles out of order, without running the model. The point that the recordingis intact rests on the hash mismatches line and the rows with quantiles out of order line."""from lib.compare import check_recordingfrom lib.settings import showfor label, value in check_recording().items():   # the default name is the chronos-2 folder in examples/data/recorded/    show(label, value)
Salidapython recorded-chronos2-check.py
outputs checked: 2
hash mismatches: 0
rows: 2160
clients: 30
windows: 3
interpolated levels: 2
rows with quantiles out of order: 0
largest quantile crossing, kWh: 0.00
inverted 80 or 95 ranges: 0

The recording is intact: 0 of its 2 outputs fail their hashes, and 0 rows have quantiles out of order. The check verifies the output files, not the prompt hash, which covers the input and the configuration.

The licence decides which weights the book can run or record: this book uses only Apache-2.0 or MIT weights. Chronos-2's weights and the chronos-forecasting package are Apache-2.0 (model card; package metadata). Two other families were considered and are not used. The TimesFM 3.0 weights carry a non-commercial licence, which allows non-commercial and non-production use only (TimesFM 3.0 licence). TimesFM 2.5 is Apache-2.0 (TimesFM 2.5 card), so the licence depends on the version named. The Moirai R-series checkpoints are tagged CC-BY-NC-4.0, which is also non-commercial (Moirai 2.0 card).

A later edition can run Chronos-2 live once its weights and package fit the release service's limits. Until then the recording stands in for the live run.

Working rule: Record a foundation model only with its script, its pinned revision, its package versions and a manifest that hashes every output. Read the recorded outputs in every example, and check the hashes before any score reads them. Check the licence of the weights, not only the licence of the package, and name the model version in any licence sentence.

Esta parte aún no está traducida, así que está en inglés.

Score the recordings on the same cutoffs

Último cambio en la edición 2026.11.11

The recordings mean something only when they are scored on the same hours as Prophet and the baselines.

A forecast with five quantiles can be scored in two ways. The median, the 0.5 quantile, is a point forecast, scored with MASE on the book's scale. The 0.1 and 0.9 quantiles form an 80% range, and the 0.025 and 0.975 quantiles form a 95% range. Each range is scored by coverage, the share of readings inside it, and by the interval score, which adds the width to a penalty for each reading outside it. The penalty is 2 divided by the nominal miss rate, 0.05 for a 95% range, times the distance outside. Chapter 8 defines the interval score with its source. Each quantile is also scored by its quantile loss, which charges tau for each kWh a reading falls above the quantile and one minus tau for each kWh below it (Chapter 8).

Two examples score the recordings, one for each model, on the same thirty clients, cutoffs and MASE scale as the Prophet run. Each rebuilds the global model and seasonal naive instead of copying them from Chapter 6, so every model goes through one scoring path.

recorded-chronos2-score.py Python 3.13.16 · holidays 0.110.dev20261118063242 · lightgbm 4.7.0 · mlforecast 1.1.0 · numpy 2.5.5 · pandas 2.3.3 · prophet 1.5.0 · statsforecast 2.1.1 · utilsforecast 0.2.17 Superado
"""Scores the recorded Chronos-2 forecasts on the three standard cutoffs and the MASE scale of the other models: themedian is the point forecast, the 0.1 and 0.9 quantiles the 80% range, and the 0.025 and 0.975 quantiles the 95%range. The point about the lead rests on the mean MASE lines, and the code reads the recording only, never themodel."""import pandas as pdfrom lib.compare import (global_forecasts, recorded_forecasts, report_pinball, report_range, report_subset, score,                         seasonal_naive)from lib.electricity import loadfrom lib.panel import ALL30from lib.quirks import mask_clock_changesfrom lib.scoring import report_scoresfrom lib.settings import EVAL_CUTOFFS, showraw = load(ALL30)# clock-change hours set to missing, as chapter 3 sets themdf = mask_clock_changes(raw)[["unique_id", "ds", "y"]]cutoffs = pd.DatetimeIndex(EVAL_CUTOFFS)# the recording, read only after each output matches its manifest hash; its median is the Chronos columnforecasts = recorded_forecasts().merge(    seasonal_naive(df, cutoffs), on=["unique_id", "ds"], validate="one_to_one").merge(    global_forecasts(df, cutoffs), on=["unique_id", "ds"], validate="one_to_one")models = ["Chronos", "SeasonalNaive", "Global"]# per_client: one MASE per client and model; scored: forecasts beside readingsper_client, scored = score(raw, forecasts, models, ALL30)show("clients scored", scored["unique_id"].nunique())show("scored hours", len(scored))show("clients where Chronos beats seasonal naive", int((per_client["Chronos"] < per_client["SeasonalNaive"]).sum()))report_scores(scored, per_client, models)report_subset(per_client, models)report_pinball(scored, "Chronos", "q10", 0.1)                       # quantile loss at tau 0.1: the 0.1 quantile columnreport_pinball(scored, "Chronos", "q50", 0.5)                       # at tau 0.5 the loss is half the absolute errorreport_pinball(scored, "Chronos", "q90", 0.9)report_range(scored, "Chronos", "q10", "q90", 80)report_range(scored, "Chronos", "q025", "q975", 95)
Salidapython recorded-chronos2-score.py
clients scored: 30
scored hours: 2160
clients where Chronos beats seasonal naive: 27
MAE Chronos: 100.13
MAE SeasonalNaive: 148.26
MAE Global: 122.81
RMSE Chronos: 655.39
RMSE SeasonalNaive: 780.48
RMSE Global: 678.81
mean MASE Chronos: 0.81
mean MASE SeasonalNaive: 1.18
mean MASE Global: 0.91
mean MASE Chronos SUBSET8: 0.87
mean MASE SeasonalNaive SUBSET8: 1.16
mean MASE Global SUBSET8: 1.03
quantile loss 0.1 Chronos: 39.44
quantile loss 0.5 Chronos: 50.07
quantile loss 0.9 Chronos: 17.85
coverage 80 Chronos: 0.749
interval score 80 Chronos: 572.89
coverage 95 Chronos: 0.944
interval score 95 Chronos: 958.19
recorded-nhits-score.py Python 3.13.16 · holidays 0.110.dev20261118063242 · lightgbm 4.7.0 · mlforecast 1.1.0 · numpy 2.5.5 · pandas 2.3.3 · prophet 1.5.0 · statsforecast 2.1.1 · utilsforecast 0.2.17 Superado
"""Scores the recorded NHITS forecasts on the same three cutoffs and MASE scale as Chronos-2 and the other models: themedian is the point forecast, and each range comes from levels trained directly, so none is interpolated. The pointabout its ranges rests on the coverage and interval score lines, and the code reads the recording only, never themodel."""import pandas as pdfrom lib.compare import (global_forecasts, recorded_forecasts, report_pinball, report_range, report_subset, score,                         seasonal_naive)from lib.electricity import loadfrom lib.panel import ALL30from lib.quirks import mask_clock_changesfrom lib.scoring import report_scoresfrom lib.settings import EVAL_CUTOFFS, showraw = load(ALL30)# clock-change hours set to missing, as chapter 3 sets themdf = mask_clock_changes(raw)[["unique_id", "ds", "y"]]cutoffs = pd.DatetimeIndex(EVAL_CUTOFFS)# the NHITS recording, read only after each output matches its manifest hash; its median is the NHITS columnforecasts = recorded_forecasts("nhits", "NHITS").merge(    seasonal_naive(df, cutoffs), on=["unique_id", "ds"], validate="one_to_one").merge(    global_forecasts(df, cutoffs), on=["unique_id", "ds"], validate="one_to_one")models = ["NHITS", "SeasonalNaive", "Global"]# per_client: one MASE per client and model; scored: forecasts beside readingsper_client, scored = score(raw, forecasts, models, ALL30)show("clients scored", scored["unique_id"].nunique())show("scored hours", len(scored))show("clients where NHITS beats seasonal naive", int((per_client["NHITS"] < per_client["SeasonalNaive"]).sum()))report_scores(scored, per_client, models)report_subset(per_client, models)report_pinball(scored, "NHITS", "q10", 0.1)                         # quantile loss at tau 0.1: the 0.1 quantile columnreport_pinball(scored, "NHITS", "q50", 0.5)                         # at tau 0.5 the loss is half the absolute errorreport_pinball(scored, "NHITS", "q90", 0.9)report_range(scored, "NHITS", "q10", "q90", 80)report_range(scored, "NHITS", "q025", "q975", 95)
Salidapython recorded-nhits-score.py
clients scored: 30
scored hours: 2160
clients where NHITS beats seasonal naive: 24
MAE NHITS: 136.61
MAE SeasonalNaive: 148.26
MAE Global: 122.81
RMSE NHITS: 649.41
RMSE SeasonalNaive: 780.48
RMSE Global: 678.81
mean MASE NHITS: 1.03
mean MASE SeasonalNaive: 1.18
mean MASE Global: 0.91
mean MASE NHITS SUBSET8: 1.17
mean MASE SeasonalNaive SUBSET8: 1.16
mean MASE Global SUBSET8: 1.03
quantile loss 0.1 NHITS: 27.66
quantile loss 0.5 NHITS: 68.31
quantile loss 0.9 NHITS: 40.58
coverage 80 NHITS: 0.700
interval score 80 NHITS: 682.37
coverage 95 NHITS: 0.877
interval score 95 NHITS: 1250.94
Model Mean MASE, ALL30 Mean MASE, SUBSET8 MAE, ALL30 (kWh)
Seasonal naive 1.18 1.16 148.26
Global model (Chapter 6) 0.91 1.03 122.81
Chronos-2, median 0.81 0.87 100.13
NHITS, median 1.03 1.17 136.61
Prophet, point forecast 1.35 1.39 182.44

Chronos-2 has the lowest mean per-client MASE of the five models on both sets of clients: 0.81 over the thirty clients and 0.87 on SUBSET8. It beats seasonal naive on 27 of the thirty clients. The global model from Chapter 6 is second on both sets. Chapter 4's fixed seasonal ARIMA, on the same eight clients, cutoffs and scale, scores 1.09 on SUBSET8, between the global model and seasonal naive.

NHITS is third on the thirty clients and fourth on SUBSET8, where it is narrowly behind seasonal naive, 1.17 against 1.16. Prophet has the highest mean on both sets.

The lead needs its caveat. Chronos-2's pretraining corpus includes this dataset (the recording section), so these readings may have been part of its training, and its score here is an upper bound on what to expect on readings it has not seen. NHITS has no such advantage, because each of its cutoffs was trained only on that cutoff's own context. NHITS trails Chronos-2 on the same hours, but the two are not on equal footing, and this chapter does not rank them on this comparison.

The median is also a check on the quantile arithmetic. The pinball loss at 0.5 is half the absolute error, so the median's loss should be half its MAE. For Chronos-2 the loss at 0.5 is 50.07, against the MAE of 100.13 kWh divided by two. For NHITS the loss at 0.5 is 68.31, against half its MAE of 136.61 kWh.

Chronos-2's 95% range covers 0.944 of the readings, close to its level, and its 80% range covers 0.749, a little under its level of 0.80. The Chronos-2 95% range is partly interpolated, so its score is for the model and the package together.

Prophet's 95% range has an interval score a little lower than Chronos-2's, 932.66 against 958.19, but its coverage of 0.918 sits further from its level.

NHITS's ranges are too narrow for their levels on these hours: its 80% range covers 0.700 of the readings, and its 95% range covers 0.877.

Its 95% interval score is the highest of the three 95% ranges, and its quantiles are trained directly, so no range is interpolated. The recording does not put its quantiles in order, though. The check counts 22 rows with an adjacent pair out of order, by at most 21.11 kWh, and 0 inverted 80% or 95% ranges. The scores use the values as recorded.

Three limits apply. These are three cutoffs in one week of one dataset, so the results are not a general claim about any model. Neither range was calibrated on the calibration windows that Chapter 8 used, so the coverage here is a check, not a calibration. The comparison makes no choice, because decisions in this book are made on decision windows, and the recordings have no forecasts there, so nothing is picked with these scores.

Working rule: Score a recorded forecast on the same windows, the same scale and the same readings as every other model, and name any interpolated quantile wherever its range is reported. Before you read a lead, check what the model was trained on, because a model that saw the scored readings cannot show skill on them. Do not choose a model or a setting on these windows. A recorded forecast with no forecasts on the decision windows cannot be chosen for production until it has them.

Por qué el modelo neuronal se registra

Último cambio en la edición 2026.11.5

Esta edición no ejecuta ningún modelo neuronal. Sus dos pronósticos neuronales están registrados, y la razón es el costo del entorno de ejecución, no un juicio sobre los modelos neuronales.

NHITS es un pronosticador neuronal de la biblioteca neuralforecast, que empaqueta pronosticadores neuronales para paneles de series. La biblioteca tiene licencia Apache-2.0 (metadatos de neuralforecast). Registramos NHITS de la misma forma que Chronos-2, con su propio script, versión fijada y manifiesto, y lo puntuamos junto a Chronos-2 sobre los mismos contextos, puntos de corte, escala y métricas. Su registro entrena una vez por cada punto de corte, con los mismos treinta contextos de 1,344 horas que recibió Chronos-2. La ventana de entrada es de 168 horas, una semana, y el horizonte es de 24 horas. Los cinco niveles se entrenan directamente bajo la pérdida MQLoss, con una salida por cada nivel. El escalador es el escalador estándar de neuralforecast, la semilla está fijada y la ejecución se hace en CPU. El modelo de cada punto de corte solo ve el contexto de ese punto de corte, así que ninguna lectura posterior al corte llega a él. Los pesos no se distribuyen.

NHITS recibe la misma comprobación, contra los hashes de su propio manifiesto.

recorded-nhits-check.py Python 3.13.16 · holidays 0.110.dev20261118063242 · lightgbm 4.7.0 · mlforecast 1.1.0 · numpy 2.5.5 · pandas 2.3.3 · prophet 1.5.0 · statsforecast 2.1.1 · utilsforecast 0.2.17 Superado
"""Checks the recorded NHITS outputs before any score reads them: each file's SHA-256 against manifest.json, the shapeof the forecast file, and any quantiles out of order, without running the model. The point that the recording isintact rests on the hash mismatches line; the interpolated levels line shows that every level is trained directly, andthe rows with quantiles out of order line is the one the chapter reports."""from lib.compare import check_recordingfrom lib.settings import showfor label, value in check_recording("nhits").items():   # "nhits" is the NHITS folder in examples/data/recorded/    show(label, value)
Salidapython recorded-nhits-check.py
outputs checked: 1
hash mismatches: 0
rows: 2160
clients: 30
windows: 3
interpolated levels: 0
rows with quantiles out of order: 22
largest quantile crossing, kWh: 21.11
inverted 80 or 95 ranges: 0

Los archivos de NHITS coinciden con su manifiesto, y ninguno de sus niveles se interpola, porque cada uno se entrena directamente.

Dos costos mantienen a NHITS fuera de los ejemplos. Su biblioteca trae torch, ray y pytorch-lightning, que llevan el entorno de los ejemplos de 687 MB a 1.7 GB, y un entrenamiento en CPU tarda más de lo que debería tardar un ejemplo en el servicio de publicación de 2 vCPU y 4 GiB de memoria. No interviene ningún conflicto de versiones. El archivo de requisitos de NHITS, recordings/requirements-nhits.txt, instala neuralforecast con la misma versión de pandas que fijan los ejemplos (2.3.3).

Chronos-2 tiene su propia pila. Su archivo de requisitos, recordings/requirements.txt, instala chronos-forecasting con transformers y torch, y fija pandas 3 para el registro. El paquete no necesita pandas 3. Sus metadatos aceptan pandas desde la 2.0 hasta, sin incluir, la 4 (código fuente de chronos-forecasting, pyproject.toml). Los ejemplos nunca importan ninguno de los dos paquetes. Leen las salidas guardadas.

NHITS puede ejecutarse en vivo en una edición posterior, siempre que se disponga de un entorno de ejecución que quepa en los límites del servicio de publicación, o de un servicio propio. En cualquier caso, las ventanas, la escala y la puntuación de este capítulo se mantienen, de modo que un modelo posterior se puntúa junto a los demás sobre las mismas horas.

Regla práctica: Mantén un modelo fuera del entorno de ejecución de los ejemplos cuando sus dependencias romperían el presupuesto de ejecución, y regístralo en su lugar, con sus versiones fijadas en un archivo de requisitos aparte. Antes de añadir una dependencia, instálala en un entorno limpio y mide lo que añade en tamaño y en paquetes. Una resolución que cambie un paquete fijado es una razón para registrar el modelo, no para cambiar la versión fijada.

Qué falla en producción

Último cambio en la edición 2026.11.6

La tabla de festivos de Prophet cubre solo 2013 y 2014. Los experimentos del libro la dejan desactivada, porque las ventanas de decisión sobre festivos no daban motivo para activarla, pero los cortes estándar muestran que sí altera el resultado. Un pronóstico hecho en 2015 con la tabla sin cambios no recibe efecto de festivo en los festivos posteriores, y la ejecución no informa de ningún problema. Un equipo que active la tabla debe extenderla a los años del pronóstico. Los festivos se conocen de antemano, así que la solución es extender la tabla. Cuenta además las horas festivas en cada ventana. Una ventana que contenga un festivo debería mostrar un recuento mayor que cero, y un recuento de cero donde cae un festivo es la señal que hay que buscar.

El ancho del intervalo falla de otra manera: un intervalo de Prophet que se deja en su valor por defecto del 80 % lo lee como un rango del 95 % quien solo ve la palabra «intervalo» en un informe. La línea de cobertura es la comprobación. Impresa junto a su nivel, una cobertura cercana a 0.8 es correcta para un intervalo del 80 % y un fallo para uno del 95 %. Imprime siempre el nivel junto con la cobertura.

Un pronóstico grabado caduca de otra forma. La grabación cubre tres cortes. Un pronóstico del día siguiente en producción necesita un corte cada día, y el servicio de publicación no ejecuta el modelo. Cubrir otro corte exige ejecutar el modelo, guardar su salida y calcular su hash, trabajo que queda fuera de los ejemplos. La señal de que la grabación está desactualizada es una ventana sin filas en el archivo, que muestra el recuento de ventanas de la comprobación.

Una revisión posterior del modelo cambia todos los números que dependen de la grabación, aunque el código no cambie. Trata la revisión como un disparador de actualización. Regístrala, puntúala sobre las mismas ventanas que la anterior y compara ambas antes de usar la grabación de reemplazo. Un paquete que cambia su interpolación entre versiones modifica el rango del 95 % sin tocar la mediana, así que también conviene comparar la lista de niveles interpolados del manifiesto.

Un pronóstico grabado puede además romper el orden de sus propios cuantiles. La grabación de NHITS lo hace en 22 de sus 2160 filas, por un máximo de 21.11 kWh. Un rango construido con un par cruzado se leería como invertido. La comprobación cuenta los cruces, y un consumidor debe ordenar o rechazar esas filas antes de usarlas.

El supuesto de tendencia de Prophet falla cuando la demanda de un cliente pasa a otro nivel, como ocurre tras un cambio de contrato o la pérdida de una carga grande. Prophet espera que la tendencia cambie con la misma frecuencia que en el historial, y su intervalo por defecto no cubre el patrón estacional. La señal son residuos que mantienen un mismo signo en ventanas sucesivas, y una línea de cobertura que baja a lo largo de ellas. La solución es reajustar el modelo con un historial más corto para ese cliente, no ensanchar el intervalo.

Regla práctica: Antes de informar de cualquier pronóstico de Prophet o grabado, comprueba que la tabla de festivos cubre los años del pronóstico cuando la tabla está activada, que el nivel del intervalo se imprime junto a su cobertura, que la grabación tiene una ventana para cada corte en uso, que sus cuantiles están en orden y que la revisión del modelo coincide con la que se puntuó. Una comprobación fallida detiene el informe, y el informe indica qué comprobación falló.

Decisiones que tomó este capítulo

Último cambio en la edición 2026.11.5
  • Prophet se ejecuta en los treinta clientes, no solo en SUBSET8. Ejecutar los treinta sitúa a Prophet junto al modelo global en los mismos clientes. Las medias de SUBSET8 se imprimen a partir de la misma ejecución.
  • El intervalo de Prophet es del 95 %, definido en el código e impreso. Así, su cobertura y su puntuación de intervalo se pueden comparar con los rangos del 95 % de las grabaciones. No se usa el valor predeterminado del 80 %.
  • Los indicadores de estacionalidad de Prophet están fijados en el código. La estacionalidad diaria y la semanal están activadas, y la anual, desactivada. Son las opciones que Prophet adopta para ocho semanas de datos horarios, fijadas de forma explícita.
  • La tabla de festivos está desactivada. En las ventanas de decisión con festivos, la tabla elevó el MASE medio, así que se mantuvo el modelo más simple. En los cortes estándar, la tabla habría reducido el MASE medio de Prophet, y ese valor se informa sin usarlo para elegir.
  • Chronos-2 y NHITS están registrados, no ejecutados. Los ejemplos leen las salidas guardadas, comprueban sus hashes y nunca llaman a un modelo, porque los ejemplos deben ejecutarse sin conexión y dar siempre la misma salida.
  • NHITS se registra por su coste de tiempo de ejecución. Sus dependencias llevarían el entorno de ejecución de los ejemplos de 687 MB a 1.7 GB, por encima de los 2 vCPU y 4 GiB del servicio de publicación, y un entrenamiento en CPU tarda más de lo que debería tardar un ejemplo.
  • El rango del 95 % de Chronos-2 se evalúa con sus dos cuantiles interpolados, y el capítulo lo indica. Los cuantiles de NHITS se entrenan directamente, así que ninguno de sus rangos está interpolado.
  • La ventaja de Chronos-2 se interpreta como una cota superior. Su corpus de preentrenamiento incluye este conjunto de datos, por lo que su puntuación aquí no es fuera de muestra.
  • Las elecciones provienen solo de las ventanas de decisión. Las grabaciones no tienen previsiones en esas ventanas, así que este capítulo no toma ninguna decisión con ellas.

Este capítulo, en esta edición

Edición 2026.11.11Ejemplos verificados el 20 de nov de 2026Superan la prueba 6 de 6 ejemplos ejemplos de este capítulo

Traducido del libro en inglés, edición 2026.11.11.

Cita esta sección

Formato

Cada formato cita el libro en inglés, que esta traducción sigue.

Cada formato nombra esta edición. Su enlace abre la edición actual, donde el encabezado de cada sección indica cuándo cambió su texto por última vez. Las fechas de acceso y de recuperación son el día en que lo abres.

En esta página

Capítulo 9 de 10: Comparación de Prophet y un modelo fundacional

  1. Where Prophet fits
  2. Run Prophet on the same cutoffs
  3. Load a recorded foundation model's forecasts
  4. Score the recordings on the same cutoffs
  5. Por qué el modelo neuronal se registra
  6. Qué falla en producción
  7. Decisiones que tomó este capítulo

Volver arribaTodos los capítulos