Pronosticar muchos clientes con un solo modelo

Unos 24 minutos de lectura, con 4 ejemplos en 7 secciones.

Edición 2026.11.11 Ejemplos verificados el 20 de nov de 2026 41 de 41 ejemplos pasan Qué ha cambiado

Traducido con IA. Verificado por personas en inglés. Cómo hacemos los libros

Traducido del libro en inglés, edición 2026.11.11. Lee el original

Terminas este capítulo con un modelo global para los treinta clientes, una regla de inicio que mantiene fuera de él los ceros iniciales de los clientes que empiezan tarde, y una puntuación que lo compara con el naive estacional para cada cliente en los mismos cortes temporales. La compra del proveedor para el día siguiente es un total que el capítulo 7 construye a partir de treinta pronósticos de clientes. Un modelo por cliente necesitaría treinta ajustes en cada ventana, uno para cada cliente. Los clientes comparten un ritmo diario, así que un solo modelo ajustado con todos ellos puede aprender ese ritmo una sola vez.

El capítulo 5 ajustó su modelo con los ocho clientes de SUBSET8. Ajustar un solo modelo con los treinta plantea tres decisiones que determinan si se puede confiar en su puntuación: si conviene darle un identificador de cliente, cómo tratar a los tres clientes que empiezan tarde (clientes cuyos medidores comienzan a mitad de los datos), y cómo puntuar treinta pronósticos en una misma escala. Cuatro ejemplos sostienen el capítulo. Cada uno imprime primero sus recuentos, para que puedas comprobar que el modelo cubrió los clientes que debía antes de leer cualquier error.

Un modelo para muchos clientes

Último cambio en la edición 2026.11.5

Montero-Manso and Hyndman (2021), en Principios y algoritmos para pronosticar grupos de series temporales: localidad y globalidad, muestran que un modelo global no es más restrictivo que uno local, porque ambos pueden producir los mismos pronósticos. Comparan los métodos globales, que ajustan un solo modelo a un grupo de series, con los métodos locales, que ajustan cada serie por separado. Un modelo global ajusta un mismo conjunto de parámetros a muchas series en una sola pasada, de modo que cada cliente contribuye a lo que el modelo aprende. Su resumen parte de la idea de que, para grupos de series similares, los métodos globales superan a los métodos locales más establecidos, y señala que resultados posteriores de esa literatura muestran un buen desempeño de los modelos globales incluso en conjuntos de datos heterogéneos. También informan de que los modelos lineales globales pueden ofrecer una precisión competitiva con dos órdenes de magnitud menos de parámetros que los métodos locales. Las afirmaciones de este párrafo provienen de ese resumen. Bojer y Meldgaard (2020) encontraron que los modelos globales de conjunto y los árboles potenciados por gradiente funcionaron bien en seis competiciones de pronóstico de Kaggle (arXiv:2009.07701). Esos datos no son los del proveedor, así que estas fuentes son una razón para probar el enfoque. No son evidencia sobre este caso. Los clientes del proveedor tienen la misma forma diaria y tamaños distintos, por lo que la prueba debe hacerse con ellos.

Los datos se guardan en formato largo: una fila por cliente y hora, con tres columnas. unique_id identifica al cliente, ds es la hora y y son los kilovatios hora consumidos en esa hora. Cada fila lleva sus propios retardos, las lecturas de un día y de una semana antes. El Capítulo 5 comprobó que las variables de calendario y de festivos no reducían el error en estas ocho semanas de historial, así que este capítulo usa solo retardos, de 24 y 168 horas. Ambos retardos son al menos tan largos como el horizonte de 24 horas, de modo que ninguno lee el día que se está pronosticando.

Los retardos cuestan filas. Un retardo de 168 horas mira una semana atrás, así que la primera semana del historial de cada cliente no tiene ninguna fila con todos los retardos disponibles. El Capítulo 5 mostró que MLForecast elimina de forma predeterminada esas filas del entrenamiento. Por tanto, las filas de entrenamiento de un cliente son menos que las horas de su historial. El pronóstico en sí no se ve afectado, porque cada hora pronosticada solo lee lecturas anteriores a su fecha de corte.

Los clientes difieren en tamaño, y una pérdida de error cuadrático sobre kilovatios hora sin escalar deja que los clientes más grandes determinen las divisiones, mientras que los pequeños pesan poco. Por eso, el objetivo de cada cliente se escala antes del ajuste. La escala es LocalStandardScaler de mlforecast 1.1.0. Asigna a cada cliente su propia media y desviación estándar, y el modelo aprende los valores escalados:

z = (y - mean) / standard deviation

La media y la desviación provienen solo del historial de ese cliente anterior a la fecha de corte, y los pronósticos se convierten de nuevo con esos mismos dos números. El código de la biblioteca muestra dónde ocurre esto. MLForecast.fit llega a TimeSeries._fit en mlforecast/core.py, que ajusta el escalador al historial de cada cliente anterior a la fecha de corte. Ese historial incluye la primera semana, cuyas filas los retardos luego eliminan del entrenamiento. TimeSeries.predict en el mismo archivo invierte el escalado. La clase es _BaseLocalScaler en mlforecast/target_transforms.py. El Capítulo 5 comparó los objetivos escalados y sin escalar sobre SUBSET8, y el modelo sin escalar funcionó peor que el naive estacional. Este capítulo mantiene el objetivo escalado y no repite esa comparación.

make_forecaster en el módulo compartido examples/lib/features.py construye un MLForecast con un LGBMRegressor, configurado mediante LGBM_PARAMS. El modelo es determinista, usa un solo hilo y una semilla fija, y se ejecuta bajo LightGBM 4.7.0. El módulo examples/lib/panel.py ajusta ese pronosticador una vez por ventana, con el historial del panel anterior a la fecha de corte de la ventana. Rellena cualquier hora de cambio de horario a partir de ese mismo historial, como hace el Capítulo 3, y devuelve un pronóstico para cada cliente.

Regla práctica: Ajusta un modelo global cuando tus series compartan una forma y debas pronosticar muchas de ellas en cada ventana. Escala el objetivo de cada serie con estadísticos calculados solo a partir del historial de esa ventana, usa solo retardos al menos tan largos como el horizonte, y ajusta una vez por ventana, no una vez por serie. Después, evalúa el modelo serie por serie frente a una línea base en las mismas ventanas, como hacen las secciones siguientes, antes de confiar en su error agregado.

Esta parte aún no está traducida, así que está en inglés.

Identify each client

Último cambio en la edición 2026.11.11

Once the rows share one table, the model no longer knows which client a row belongs to, unless you tell it. Scaling removes each client's mean and spread, so what remains is mostly the shape of the day. A client identifier puts the client in the table as a static categorical feature. The trees can then split on the client and learn that client's own habits. The cost is one more split to learn for each client, and a client with no history has nothing to learn from. A new client therefore needs a refit before the model can forecast it.

The rule for keeping an identifier is the one Chapter 5 sets for every feature: keep it only if it lowers the mean per-client MASE on decision windows that the standard cutoffs do not use. The held-out week runs from Dec 25 to Dec 31, 2014, and it has seven daily cutoffs. The three standard cutoffs, Dec 29 to Dec 31, 2014, are the reported scores, so the decision does not use them, and it does not use Dec 25 to Dec 28, 2014 either. The decision windows are four ordinary days before the held-out week: cutoffs on Nov 25, Nov 26, Dec 2 and Dec 3, all in 2014, each forecasting 24 hours. None of their scored hours falls on a Portuguese holiday. The one holiday of 2014 between these windows and the held-out week, Dec 8, 2014, is not among them. The calendar lists no holiday on Dec 1, 2014.

Before the identifier goes into the model, test it on windows the reported scores never touch. The check fits the model twice on each decision cutoff, with and without the identifier, and refuses to start if a scored hour is a holiday or in the held-out week.

client-identifier-check.py Python 3.13.16 · holidays 0.110.dev20261118063242 · lightgbm 4.7.0 · mlforecast 1.1.0 · numpy 2.5.5 · pandas 2.3.3 · statsforecast 2.1.1 · utilsforecast 0.2.17 Superado
"""Tests whether a client identifier lowers the global model's error on four ordinary decision windows, before theheld-out week, on the book's MASE scale. The point rests on the mean per-client MASE lines and on `identifier kept`."""from lib.electricity import loadfrom lib.panel import ALL30, DECISION_CUTOFFS, check_decision_hours, mae_change, panel_windowsfrom lib.quirks import mask_clock_changesfrom lib.scoring import actual_hours, book_scale, report_scores, score_windowsfrom lib.settings import showraw = load(ALL30)                                             # all thirty clients, not the eight of SUBSET8df = mask_clock_changes(raw)[["unique_id", "ds", "y"]]        # clock-change hours are set to missing, so the scores leave them outcutoffs = DECISION_CUTOFFSactual = actual_hours(df, cutoffs)holiday_hours = check_decision_hours(actual)  # raises before any fit if a scored hour is a holiday or in the held-out weekplain = panel_windows(df, cutoffs, client_id=False, name="plain")               # the model without the identifieridentified = panel_windows(df, cutoffs, client_id=True, name="identifier")      # the same model, with the client as a featuremodels = ["plain", "identifier"]# the MASE scale reads only the readings before the first decision cutoff, so the scored hours cannot leak into itscored, per_client = score_windows(actual, plain.merge(identified, on=["unique_id", "ds"], validate="one_to_one"),                                   models, book_scale(raw, cutoffs[0]))lowers = per_client["identifier"] < per_client["plain"]size = raw.groupby("unique_id")["y"].mean().sort_values(ascending=False)   # mean kWh per client, largest firstcontrib = mae_change(scored, "plain", "identifier")           # each client's share of the pooled MAE change, kWh; the shares sum to the totalshow("first decision cutoff", f"{cutoffs[0]:%Y-%m-%d}")show("last decision cutoff", f"{cutoffs[-1]:%Y-%m-%d}")show("clients", scored["unique_id"].nunique())show("windows", len(cutoffs))show("scored hours", len(scored))show("decision hours on a holiday", holiday_hours)report_scores(scored, per_client, models)show("clients where the identifier lowers MASE", int(lowers.sum()))show("of the 15 largest clients, where the identifier lowers MASE", int(lowers[size.index[:15]].sum()))show("of the 15 smallest clients, where the identifier lowers MASE", int(lowers[size.index[-15:]].sum()))show("largest client", size.index[0])show("pooled MAE change from the largest client, kWh", contrib[size.index[0]])show("pooled MAE change, all clients, kWh", contrib.sum())# the decision rule keeps the identifier only if the mean per-client MASE is lower, not the pooled MAEshow("identifier kept", "yes" if per_client["identifier"].mean() < per_client["plain"].mean() else "no")
Salidapython client-identifier-check.py
first decision cutoff: 2014-11-25
last decision cutoff: 2014-12-03
clients: 30
windows: 4
scored hours: 2880
decision hours on a holiday: 0
MAE plain: 70.72
MAE identifier: 66.81
RMSE plain: 316.50
RMSE identifier: 303.68
mean MASE plain: 0.77
mean MASE identifier: 0.75
clients where the identifier lowers MASE: 23
of the 15 largest clients, where the identifier lowers MASE: 10
of the 15 smallest clients, where the identifier lowers MASE: 13
largest client: MT_370
pooled MAE change from the largest client, kWh: -3.80
pooled MAE change, all clients, kWh: -3.92
identifier kept: yes

The identifier lowers the mean per-client MASE, from 0.77 to 0.75, and it lowers the MASE of 23 of the thirty clients.

On these windows the pooled MAE falls too, from 70.72 kWh to 66.81 kWh. The two errors weigh clients differently, because the pooled MAE is weighted toward the largest.

The pooled change of -3.92 kWh is almost all from one client, MT_370, which accounts for -3.80 kWh of it.

The per-client MASE gives every client the same weight, and the identifier helps across sizes. It lowers the MASE of 10 of the fifteen largest clients and of 13 of the fifteen smallest. It does not help only the large clients.

The printed verdict is yes, so the identifier is kept and the final model uses it. The margin is small. Other windows could give the opposite sign by a similar small amount, so the verdict is close and depends on the windows. A reader with other windows should measure it again before trusting it.

The identifier has a cost at the standard cutoffs. There, the final model's mean per-client MASE is 0.91 with the identifier, against 0.98 without it, but its pooled MAE is 122.81 kWh, against 119.03 kWh without it. Chapter 7 has to check its total against that. The rest of the chapter uses the model with the identifier.

Working rule: Treat a client identifier as one more feature under Chapter 5's rule: keep it only if it lowers the mean per-client MASE on the decision windows, with the pooled MAE beside it. If the decision is close, say so, and measure it again on other windows. If the identifier wins, write down its cost: a new client has no learned split until it has history, and the model must be refitted before it can forecast that client.

Esta parte aún no está traducida, así que está en inglés.

Apply the start rule to every client

Último cambio en la edición 2026.11.11

A series begins at its first reading. Before that, the meter reports zeros, and those zeros are not low use. Chapter 4 met this for one client. With thirty clients, three are late starters. MT_039 first reads on 2013-10-03, MT_179 on 2013-12-20, and MT_115 on 2014-01-14. A model fitted on their leading zeros learns a level near zero for them. The start rule drops every reading before a client's first reading. It finds that reading in the rows before the cutoff only.

The rule refuses a cutoff in two cases only: a client has no non-zero reading before the cutoff, or it has fewer than 48 hours of history since its first reading. It does not skip the client, because a skipped client leaves the total short. The rule does not catch a gap or a run of zeros in the middle of a series, and a meter replaced under the same id passes too, because its zeros are not leading. The scale catches a gap, because it refuses a series that is not one unbroken hourly series. Nothing in the forecast checks for one, so a gap there shows only as fewer training rows.

The leak check asks whether the rule reads only the past and refuses the right cutoffs. It runs nine cutoffs, refused or not, and poisons each from the cutoff on, once with zeros and once with a value no meter could record. The history must not change, and every accepted cutoff must hold all thirty clients.

start-rule-leak-check.py Python 3.13.16 · holidays 0.110.dev20261118063242 · lightgbm 4.7.0 · mlforecast 1.1.0 · numpy 2.5.5 · pandas 2.3.3 · statsforecast 2.1.1 · utilsforecast 0.2.17 Superado
"""Checks that the start rule reads only rows before each cutoff, refuses the cutoffs the data's flag predicts, andremoves only leading zeros, on nine cutoffs. The same checks run on a deliberately broken history function, and eachmust fail there; the point rests on the refused, changed and missing counts. The checks are start_rule_checks in lib/panel.py."""import pandas as pdfrom lib.electricity import loadfrom lib.panel import ALL30, FEBRUARY_CUTOFFS, leaky_history, panel_history, start_rule_checksfrom lib.settings import EVAL_CUTOFFS, showraw = load(ALL30)# nine cutoffs: three around MT_179's and MT_115's first readings (refused), the February cutoffs and the standard cutoffsCHECKED = pd.DatetimeIndex(["2013-12-15", "2014-01-01", "2014-01-15", *FEBRUARY_CUTOFFS, *EVAL_CUTOFFS])def report(prefix, r):    show(f"{prefix}cutoffs checked", len(CHECKED))    show(f"{prefix}cutoffs refused by the start rule", r["refused"])    show(f"{prefix}cutoffs expected to be refused, from the flag", r["expected"])    show(f"{prefix}cutoffs where the refusal differs from the flag", r["mismatch"])    show(f"{prefix}cutoffs accepted", r["accepted"])    show(f"{prefix}accepted cutoffs with a client missing from the history", r["missing"])    show(f"{prefix}cutoffs whose history changes when the future is poisoned", r["changed"])# the book's history function, with the count of rows the rule removes switched onreal = start_rule_checks(raw, CHECKED, panel_history, removed_rows=True)report("", real)show("rows the start rule removes, accepted cutoffs", real["removed"])show("removed rows that are not zero", real["removed_nonzero"])# the deliberately broken history, which reads each first reading from the whole series, must fail every checkreport("broken version, ", start_rule_checks(raw, CHECKED, leaky_history))
Salidapython start-rule-leak-check.py
cutoffs checked: 9
cutoffs refused by the start rule: 3
cutoffs expected to be refused, from the flag: 3
cutoffs where the refusal differs from the flag: 0
cutoffs accepted: 6
accepted cutoffs with a client missing from the history: 0
cutoffs whose history changes when the future is poisoned: 0
rows the start rule removes, accepted cutoffs: 2064
removed rows that are not zero: 0
broken version, cutoffs checked: 9
broken version, cutoffs refused by the start rule: 1
broken version, cutoffs expected to be refused, from the flag: 3
broken version, cutoffs where the refusal differs from the flag: 2
broken version, cutoffs accepted: 8
broken version, accepted cutoffs with a client missing from the history: 2
broken version, cutoffs whose history changes when the future is poisoned: 2

The real rule passes all three checks. Its refusals match the data's own flag, which uses each first reading and the 48-hour rule: the rule refuses 3 of the nine cutoffs, the flag predicts 3, and 0 cutoffs differ from it.

Poisoning the future changes no history. Its history changes at 0 cutoffs, and 0 accepted cutoffs are missing a client.

The rule removes leading zeros and nothing else. Across the accepted cutoffs it drops 2064 rows, and 0 of them are not zero.

The same checks run on a deliberately broken version, which reads each first reading from the whole series, the leak the checks are written for. It fails each of them. Its refusals differ from the flag at 2 cutoffs, its history changes at 2, and 2 accepted cutoffs drop a client without a message. So a pass on the real rule means something.

The model runs with the start rule and without it, on the standard cutoffs and then the February cutoffs. The counts that differ, and each run's error, show what the rule does.

global-model-raw-starts.py Python 3.13.16 · holidays 0.110.dev20261118063242 · lightgbm 4.7.0 · mlforecast 1.1.0 · numpy 2.5.5 · pandas 2.3.3 · statsforecast 2.1.1 · utilsforecast 0.2.17 Superado
"""Runs the global model with the start rule and without it, on the February cutoffs, where a late starter's zeros enterthe history, and on the standard cutoffs, where the rule changes no row. The point rests on the February mean MASE linesand the count of forecast hours that differ; the scale lines compare the book's scale with a labelled counterfactual."""import numpy as npimport pandas as pdfrom lib.electricity import loadfrom lib.panel import (ALL30, FEBRUARY_CUTOFFS, IDENTIFIER_KEPT, LATE_STARTERS, late_starter_facts, mae_by_group,                       mase_by_run, scale_facts, with_and_without_start_rule)from lib.quirks import mask_clock_changesfrom lib.settings import EVAL_CUTOFFS, showraw = load(ALL30)df = mask_clock_changes(raw)[["unique_id", "ds", "y"]]        # clock-change hours are set to missing, so the scores leave them outfor label, value in late_starter_facts(raw, df).items():    show(label, value)standard = pd.to_datetime(EVAL_CUTOFFS)                       # the three standard cutoffs# two runs of the same model on the same cutoffs, differing only in their historiesscored = with_and_without_start_rule(df, standard, client_id=IDENTIFIER_KEPT)# compared with isclose, so a rounding difference between two runs does not count as a changed forecastshow("forecast hours that differ, standard", int((~np.isclose(scored["start"], scored["raw"])).sum()))# the February cutoffs, where the zeros of MT_115 and MT_179 enter the historyfeb = with_and_without_start_rule(df, FEBRUARY_CUTOFFS, client_id=IDENTIFIER_KEPT)show("scored hours, february", len(feb))show("forecast hours that differ, february", int((~np.isclose(feb["start"], feb["raw"])).sum()))for model in ["start", "raw"]:    for group, value in mae_by_group(feb, model).items():        show(f"MAE {group}, {model}, february", value)# both runs on one scale, read from the first February cutofffeb_mase = mase_by_run(raw, feb, ["start", "raw"])show("clients in MASE, february", int(feb_mase["start"].notna().sum()))for uid in LATE_STARTERS:    show(f"MASE {uid}, start, february", feb_mase.loc[uid, "start"])    show(f"MASE {uid}, raw, february", feb_mase.loc[uid, "raw"])for run in ["start", "raw"]:    show(f"mean MASE, {run}, february", feb_mase[run].mean())# 'counterfactual raw' keeps the zeros before each first reading, a scale the book does not use; 'start rule' is the book'sfor label, value in scale_facts(raw, standard, IDENTIFIER_KEPT).items():    show(label, value)
Salidapython global-model-raw-starts.py
first reading MT_039: 2013-10-03
first reading MT_115: 2014-01-14
first reading MT_179: 2013-12-20
zero share of history MT_115, 2014-02-01: 68
zero share of history MT_179, 2014-02-01: 23
zero share of history MT_115, 2014-02-15: 43
zero share of history MT_179, 2014-02-15: 0
zero share of history MT_115, 2014-02-28: 20
zero share of history MT_179, 2014-02-28: 0
forecast hours that differ, standard: 0
scored hours, february: 2160
forecast hours that differ, february: 2160
MAE late starters, start, february: 4.60
MAE other clients, start, february: 77.53
MAE MT_039, start, february: 4.64
MAE MT_115, start, february: 4.62
MAE MT_179, start, february: 4.54
MAE late starters, raw, february: 4.60
MAE other clients, raw, february: 77.95
MAE MT_039, raw, february: 4.57
MAE MT_115, raw, february: 4.68
MAE MT_179, raw, february: 4.53
clients in MASE, february: 30
MASE MT_039, start, february: 0.78
MASE MT_039, raw, february: 0.77
MASE MT_115, start, february: 0.90
MASE MT_115, raw, february: 0.91
MASE MT_179, start, february: 0.54
MASE MT_179, raw, february: 0.54
mean MASE, start, february: 0.70
mean MASE, raw, february: 0.71
MASE scale MT_039, start rule: 5.21
MASE scale MT_039, counterfactual raw: 3.28
MASE scale MT_115, start rule: 4.58
MASE scale MT_115, counterfactual raw: 2.22
MASE scale MT_179, start rule: 5.91
MASE scale MT_179, counterfactual raw: 3.04
mean MASE global, counterfactual raw scale, standard cutoffs: 1.01
mean MASE global, book scale, standard cutoffs: 0.91

At the standard cutoffs the rule matters less than the late starters might suggest. The three standard cutoffs have 8 weeks of history, reaching back to early November, and no client's history in that window holds a zero from before its first reading. So the rule changes no training row there, and 0 of the 2160 forecast hours change. The rule still changes the score through the MASE scale.

The February cutoffs, which Chapter 4 used for MT_115, are where the zeros enter the history. MT_115's history is 68 percent zeros on Feb 1, 2014, 43 percent on Feb 15 and 20 percent on Feb 28. MT_179's history is 23 percent zeros on Feb 1 and none after.

Here the forecasts change, because one model learns from all thirty clients at once: 2160 of the 2160 forecast hours in these three windows differ. Both runs are scored on the one book scale, read from Feb 1, 2014, so only the histories differ. The mean MASE is 0.70 with the rule and 0.71 without it.

Each late starter's MASE moves very little, and the direction is mixed. MT_039 scores 0.78 with the rule and 0.77 without it, and MT_115 scores 0.90 with the rule and 0.91 without.

MT_179 scores 0.54 with the rule and 0.54 without it, the same to two decimals.

The other 27 clients' MAE moves from 77.95 kWh without the rule to 77.53 kWh with it. The effect is small and mixed, and three windows are too few to call it. We keep the rule because the zeros are not demand, and because the scale depends on it.

You could instead mark the zeros as missing, as the clock rule does for the two clock-change hours in Chapter 3. That would be the wrong fix for this data. A zero before a first reading is the meter being silent, not a reading that is missing, and a missing value would still be a row the model learns from. The start rule removes the rows instead, and its check is simple to state: it removes only rows before the first reading, and every one of them is zero.

Working rule: Apply the start rule to every series that has leading zeros, and find each first reading from the rows before the cutoff. When the rule refuses a cutoff, stop the run and name the client and the cutoff. Do not fill the zeros, and do not publish a panel with a client missing. Judge the rule by the scale and by the windows where zeros enter the history, and expect the MAE effect to be small and mixed.

Esta parte aún no está traducida, así que está en inglés.

Score thirty clients on the same cutoffs

Último cambio en la edición 2026.11.11

The three standard cutoffs are Dec 29 to Dec 31, 2014, each at 00:00 on that day, with 24 hours forecast from each. For thirty clients that is 72 scored hours per client. Seasonal naive is the baseline Chapter 2 set. It takes each cutoff's own history, the 8 weeks before it, and copies the last day's readings forward. At the standard cutoffs no client's history holds a leading zero, so the global model and seasonal naive read the same readings. The comparison is fair because the inputs match, not because the methods are alike.

The headline is the mean per-client MASE, the measure Chapter 2 defines. The book uses one scale per client per backtest, the rule Chapter 2 sets, and the function book_scale in examples/lib/scoring.py builds it. The scale is the mean absolute change between each reading and the reading one day earlier, taken over every reading before the first scored cutoff. A late starter is counted from its first reading, and the clock-change readings are masked out. A client with no non-zero reading before the cutoff has no scale, so it is left out of MASE and reported, not treated as an error. Because the scale reads only readings before the scored windows, it is known at every cutoff and cannot leak. Each client counts once in the mean, so one large client cannot set the headline. The pooled MAE and RMSE follow as second lines. They show the error in kilowatt hours that the supplier pays, and they are weighted toward the large clients.

The scale is part of the score. MT_115's scale is 2.22 kWh if the zeros before its first reading are kept, and 4.58 kWh with the book's rule, which counts it from its first reading.

MT_039 and MT_179 move the same way. MT_039's scale rises from 3.28 to 5.21 kWh, and MT_179's from 3.04 to 5.91 kWh.

On the standard cutoffs the same forecasts score a mean MASE of 1.01 with the scale that keeps the zeros, and 0.91 with the book's scale. That difference is the scale, not the model. A score computed with a different scale rule is not comparable with one computed with the book's rule, so every score should record the rule it used.

The score below sets the global model beside seasonal naive on the three standard cutoffs.

global-model-thirty-clients.py Python 3.13.16 · holidays 0.110.dev20261118063242 · lightgbm 4.7.0 · mlforecast 1.1.0 · numpy 2.5.5 · pandas 2.3.3 · statsforecast 2.1.1 · utilsforecast 0.2.17 Superado
"""Scores one global LightGBM model, with the identifier that client-identifier-check kept, against seasonal naive on thethree standard cutoffs, per client on the book's MASE scale. The point rests on the mean per-client MASE lines; the modelwithout the identifier is printed for reference only."""import pandas as pdfrom statsforecast import StatsForecastfrom statsforecast.models import SeasonalNaivefrom lib.backtest import forecast_windowsfrom lib.baselines import SEASONfrom lib.electricity import loadfrom lib.panel import ALL30, IDENTIFIER_KEPT, panel_windowsfrom lib.quirks import mask_clock_changesfrom lib.scoring import actual_hours, book_scale, left_out_facts, report_scores, score_windowsfrom lib.settings import EVAL_CUTOFFS, showraw = load(ALL30)df = mask_clock_changes(raw)[["unique_id", "ds", "y"]]        # clock-change hours are set to missing, so the scores leave them outcutoffs = pd.to_datetime(EVAL_CUTOFFS)                        # the three standard cutoffs, Dec 29 to Dec 31, 2014actual = actual_hours(df, cutoffs)# seasonal naive from each cutoff's own history: the baseline of Chapter 2naive = forecast_windows(StatsForecast(models=[SeasonalNaive(season_length=SEASON)], freq="h", n_jobs=1), df, cutoffs)global_fc = panel_windows(df, cutoffs, client_id=IDENTIFIER_KEPT)   # the identifier setting that client-identifier-check keptplain_fc = panel_windows(df, cutoffs, client_id=False, name="plain")  # the same model without the identifier, for referenceforecasts = naive.merge(global_fc, on=["unique_id", "ds"], validate="one_to_one").merge(    plain_fc, on=["unique_id", "ds"], validate="one_to_one")models = ["SeasonalNaive", "global", "plain"]# the book's scale, read from the readings before the first cutoff onlyscored, per_client = score_windows(actual, forecasts, models, book_scale(raw, cutoffs[0]))beats = per_client["global"] < per_client["SeasonalNaive"]    # per client: True where the global model's MASE is lowershow("clients scored", scored["unique_id"].nunique())show("windows", len(cutoffs))show("scored hours", len(scored))# a client with no non-zero reading before the first cutoff is left out of MASE, and is not an errorfor label, value in left_out_facts(raw, df, cutoffs[0]).items():    show(label, value)show("clients in MASE", per_client["global"].notna().sum())show("clients where global beats seasonal naive", int(beats.sum()))show("clients where seasonal naive wins", ", ".join(sorted(per_client.index[~beats])))report_scores(scored, per_client, models)
Salidapython global-model-thirty-clients.py
clients scored: 30
windows: 3
scored hours: 2160
clients left out of MASE: 0
clients with no reading before 2013-12-15, check: 2
clients with any row before 2013-12-15, broken count: 30
clients in MASE: 30
clients where global beats seasonal naive: 23
clients where seasonal naive wins: MT_001, MT_128, MT_141, MT_179, MT_217, MT_268, MT_294
MAE SeasonalNaive: 148.26
MAE global: 122.81
MAE plain: 119.03
RMSE SeasonalNaive: 780.48
RMSE global: 678.81
RMSE plain: 657.63
mean MASE SeasonalNaive: 1.18
mean MASE global: 0.91
mean MASE plain: 0.98

The example scores 30 clients on 3 windows, 2160 scored hours, and it counts 0 clients left out of MASE, which is zero: every client has a non-zero reading before the first cutoff, so every client is in the MASE table.

The count is only as good as its rule. At a cutoff of Dec 15, 2013, 2 clients have no reading yet, and counting any row before the cutoff would give 30, which reports no gap.

One global model beats seasonal naive on these cutoffs, for most clients and in total: its mean per-client MASE is 0.91 against 1.18, and it wins for 23 of the thirty clients. It loses for MT_001, MT_128, MT_141, MT_179, MT_217, MT_268, MT_294, and those clients are the first place to look when the model is refreshed.

The pooled figures point the same way. The MAE is 122.81 kWh against 148.26 kWh, and the RMSE is 678.81 kWh against 780.48.

The model without the identifier, printed for reference, has a mean MASE of 0.98 and an MAE of 119.03 kWh, which is the trade-off the identifier section describes.

The printed figures are means and counts, and a mean can hide one client that is far worse than the rest. When a figure changes, open the per-client table first.

The chapter does not explain why seasonal naive wins for those clients, and that is a reason not to report only the pooled figure. The result covers three windows in one week of one season, so it describes these cutoffs, not every day of the year. The longer test is the seven-window held-out week, which the rolling example in Chapter 10 uses.

Working rule: Score a global model per client on the same cutoffs as the baseline, with the MASE scale fixed before the first scored cutoff. Report the mean per-client MASE beside seasonal naive, and print the counts of clients and scored hours so a reader can check them. State any client left out of MASE and why. Read the pooled MAE and RMSE as the cost in kilowatt hours, not as the measure of model quality.

Esta parte aún no está traducida, así que está en inglés.

Budget the run time

Último cambio en la edición 2026.11.11

A backtest costs fits, and the count is what to check before you run one. A per-client model is fitted once for each client, each window and each method, so the count is clients times windows times methods. Thirty clients over three windows is ninety fits for one per-client method. A global model is fitted once for each window and method, however many clients it covers, so the same three windows cost three fits. More clients make each global fit larger, but they do not add fits.

The book sets a subset rule for the expensive per-client methods. SUBSET8 is the eight-client subset that Chapter 4 runs its per-client statistical models on. A global model covers all thirty in one fit, so it runs on all thirty clients, as ALL30. The subsets are fixed for the whole book, so a number in one chapter can be set beside the next. The window count follows the same logic. Expensive methods use three windows, and cheap methods can use more. Each window of the identifier check costs two global fits, one with the identifier and one without, so its four windows cost eight fits.

The two kinds of cost move differently when the number of clients grows. A per-client method gets more fits, one for each new client. A global model gets bigger fits, because each one reads more rows, and it keeps the same number of fits.

The book does not quote timings. The examples run with one thread, so their output repeats on the machine that recorded the book's outputs, and that choice costs run time. A time depends on the machine, and a reader's machine may differ. Count the fits, not the seconds.

Working rule: Before you run a backtest, count its fits: clients times windows times methods for a per-client method, and windows times methods for a global one. If the count is over budget, cut the subset or the window count, never the scored windows, and write the cut down beside the score. Then a reader can compare like with like.

Qué sale mal en producción

Último cambio en la edición 2026.11.5

Una ejecución puede detenerse. Un cliente sin ninguna lectura distinta de cero antes de un punto de corte, o con menos de 48 horas de historial desde su primera lectura, hace que la regla de inicio rechace ese punto de corte. La regla lanza un error en lugar de omitir al cliente, porque un cliente omitido deja el total incompleto, y el Capítulo 7 suma los clientes para calcular el total. En producción, la ejecución debe detenerse, indicar el cliente y el punto de corte, y esperar a una persona. La comprobación de fugas muestra los mismos rechazos en los datos del libro.

Un solo modelo afecta a todos los clientes. En los puntos de corte de febrero, los ceros de los clientes que empezaron tarde cambiaron 2160 horas de pronóstico, y el MAE de los otros 27 clientes también cambió, como muestra la sección sobre la regla de inicio. Cualquier cambio en el historial de un cliente, en una regla o en una variable altera los pronósticos de todo el panel. Después de cualquier cambio, revisa a todos los clientes, no solo a aquel que modificaste.

La escala forma parte de la puntuación. Los mismos pronósticos obtienen un MASE medio global de 1.01 o de 0.91, según si la escala conserva los ceros anteriores a la primera lectura. La sección sobre puntuación explica las escalas. Registra la regla junto con cada puntuación, y trata un cambio en la regla como un cambio en la puntuación.

Las decisiones pueden desviarse con el tiempo. La decisión sobre los identificadores se apoya en cuatro ventanas ordinarias que quedan fuera de la semana reservada, y su veredicto es ajustado. Cuando los clientes se incorporan, se van o cambian de tamaño, la decisión debe volver a tomarse.

Un cliente que falta no aparece como error en la tabla. Un panel de veintinueve clientes sigue siendo una tabla válida, y un pronóstico sobre ella sigue pareciendo un pronóstico. Una ejecución que puntúa menos clientes u horas de los esperados, o un recuento de clientes en los que el modelo global supera al estacional ingenuo, y que cambia bruscamente entre ejecuciones, es motivo para detenerse. También lo es un MASE por cliente que salta para un solo cliente mientras el MAE agrupado permanece estable.

Regla práctica: Imprime y revisa los recuentos (clientes, ventanas, horas puntuadas, puntos de corte rechazados, clientes excluidos del MASE) en cada actualización, antes de leer cualquier puntuación. Después, informa el MASE medio por cliente junto al estacional ingenuo. Detén la ejecución ante cualquier rechazo, y no publiques un total calculado a partir de un panel incompleto.

Decisiones de este capítulo

Último cambio en la edición 2026.11.5

Capítulo 7 parte de lo que este capítulo entrega: treinta pronósticos por cliente para cada punto de corte estándar, hechos por un único modelo con el identificador de cliente y la regla de inicio aplicadas, y evaluados en la escala MASE única del libro. El identificador se conserva por la decisión en client-identifier-check, y examples/lib/panel.py guarda ese veredicto en IDENTIFIER_KEPT. El error agregado en estos puntos de corte es peor con el identificador, así que el Capítulo 7 debe revisar el total antes de confiar en los pronósticos por cliente.

  • Un modelo global LightGBM cubre los treinta clientes, con rezagos 24 y 168, sin variables de calendario ni de festivos, y con el objetivo de cada cliente escalado solo a partir del historial de esa ventana, el escalado que el Capítulo 5 establece.
  • Se conserva un identificador de cliente. La decisión usó cuatro ventanas ordinarias, del 25 de noviembre al 3 de diciembre de 2014, fuera de la semana reservada y sin festivo portugués, como exige la regla del libro para elegir una configuración. El MASE medio en esas ventanas pasa de 0.77 a 0.75. El veredicto es ajustado, como muestra la sección del identificador.
  • La regla de inicio se aplica a todos los clientes. Rechaza un punto de corte que no puede iniciar, en lugar de omitir al cliente.
  • Las horas del cambio de hora en el historial de cada panel se completan solo a partir de ese mismo historial, de modo que una ventana móvil que cruza un cambio de hora se ajusta, no se rechaza. Las ventanas móviles del Capítulo 10 dependen de esto.
  • El titular es el MASE medio por cliente de los treinta clientes, en la escala única del libro (establecida en el Capítulo 2 y calculada por book_scale), junto al modelo ingenuo estacional en los mismos puntos de corte. Un cliente sin ninguna lectura distinta de cero antes del punto de corte se excluye del MASE y se informa aparte. El MAE y el RMSE agregados se muestran como líneas secundarias. No se excluye a ningún cliente en los puntos de corte estándar, porque cada uno tiene una lectura antes del primer punto de corte.
  • El modelo se ejecuta en un solo hilo con una semilla fija, de modo que su salida se repite en la máquina que registró las salidas del libro. El libro no ha probado otras máquinas.

Este capítulo, en esta edición

Edición 2026.11.11Ejemplos verificados el 20 de nov de 2026Superan la prueba 4 de 4 ejemplos ejemplos de este capítulo

Traducido del libro en inglés, edición 2026.11.11.

Cita esta sección

Formato

Cada formato cita el libro en inglés, que esta traducción sigue.

Cada formato nombra esta edición. Su enlace abre la edición actual, donde el encabezado de cada sección indica cuándo cambió su texto por última vez. Las fechas de acceso y de recuperación son el día en que lo abres.

En esta página

Capítulo 6 de 10: Pronosticar muchos clientes con un solo modelo

  1. Un modelo para muchos clientes
  2. Identify each client
  3. Apply the start rule to every client
  4. Score thirty clients on the same cutoffs
  5. Budget the run time
  6. Qué sale mal en producción
  7. Decisiones de este capítulo

Volver arribaTodos los capítulos