Quantile traffic forecasting and N+2 capacity planning

Why forecast when you have charts

Backbone capacity planning comes down to deciding whether to expand a route, and that decision has to be made months before the link hits its limit. Last year's charts show what happened. Planning requires knowing what will happen and how reliable that forecast is.

So the first requirement is that the forecast must be quantile-based. The model outputs three values, P10, P50 and P90, for 168 hours ahead. The planner relies on the upper quantile so that capacity is sufficient even in an unfavourable scenario.

The normalization problem

Traffic across backbone sectors differs by an order of magnitude, from hundreds of gigabits to several terabits per second. The first version of the pipeline normalized the target variable globally, and the result was poor. The model adapted to the largest routes and produced an almost flat line for the small ones, because against the overall variance their fluctuations looked like noise.

The fix is a z-score computed separately for each sector:

# normalize within a sector, not across the whole dataset
stats = df.groupby("sector_id")["traffic_gbps"].agg(["mean", "std"])
df["traffic_normalized"] = (
    df["traffic_gbps"] - df["sector_id"].map(stats["mean"])
) / df["sector_id"].map(stats["std"])

The statistics are stored next to the model checkpoint, since without them the forecast cannot be converted back to gigabits. This is the most common way such a pipeline breaks: the model is trained with one set of mean/std values and inference runs with another.

From forecast to decision

A forecast on its own does not deliver value. Two components use it.

The capacity planner follows the N+2 method. A route must withstand the failure of two elements, utilization must stay below a threshold, and expansion happens only in fixed increments, so an arbitrary number of gigabits cannot be added. The planner produces a concrete recommendation: which route, by which week and how much capacity needs to be ordered now.

The anomaly detector compares actual values with the P10-P90 band. Leaving the band means that traffic behaves differently from what the model expected. This signal differs from simply high traffic and also fires on drops, which may point to an outage.

Features

There are 25 features in total: calendar features (hour, day of week, holidays), lags and rolling statistics. They are built in a separate layer with its own smoke test, which runs the whole pipeline from data loading to normalization and checks the data shape, the absence of NaNs and the completeness of the feature lists. With time series, silent data corruption costs the most time, while training crashes are rarer and show up immediately.