Quantile traffic forecasting and N+2 capacity planning

Why forecast when you have graphs

Backbone capacity planning is a "expand this direction or not" decision made months before a link hits its ceiling. Last year's graphs answer "how it was". You need an answer to "how it will be, and how confident are we".

Hence the first requirement: the forecast must be quantile-based. The model returns not one number but three - P10, P50 and P90, 168 hours ahead. The planner does not look at the median but at the upper quantile: capacity is sized against the bad case, not the average one.

The trap: normalization

Backbone sectors differ by an order of magnitude - from hundreds of gigabits to several terabits per second. The first version of the pipeline normalized the target globally, and the result was predictably poor: the global model tuned itself to the largest directions and produced a nearly flat line on the small ones - at the scale of the overall variance, their swings were noise.

The fix is a z-score computed per sector:

# normalize within a sector, not across the whole dataset
stats = df.groupby("sector_id")["traffic_gbps"].agg(["mean", "std"])
df["traffic_normalized"] = (
    df["traffic_gbps"] - df["sector_id"].map(stats["mean"])
) / df["sector_id"].map(stats["std"])

The statistics are stored next to the model checkpoint: without them the forecast cannot be converted back into gigabits. That, incidentally, is the most common way to break a pipeline like this - train with one set of mean/std and run inference with another.

From forecast to decision

A forecast on its own is not yet useful. It has two consumers.

The capacity planner applies the N+2 method: a direction must survive the loss of two elements, utilization must stay under a threshold, and expansion happens in fixed increments - you cannot buy an arbitrary number of gigabits. The output is not "traffic will grow" but "this direction needs this much more in that week; order now".

The anomaly detector compares actuals against the P10-P90 band. Leaving the band does not mean "traffic is high" but "traffic is not what the model expected", which is a fundamentally different signal: it also fires on drops, where the cause is an outage rather than growth.

Features

25 features: calendar (hour, weekday, holidays), lags and rolling statistics. A separate layer with a separate smoke test that runs the whole pipeline from data loading to normalization and checks shapes, absence of NaN and completeness of the feature lists. With time series it is silent data corruption, not a failing training run, that eats the most time.