Forecasting with five observations and the parameter budget that leaves you

The request usually arrives with the reforecast. Somebody wants a proper model of attendance, with a trend in it, and maybe something that picks up the pattern where every other edition runs hot. The show has run five times. You have five numbers.

Forecasting with five observations is a budgeting problem before it is a modelling problem. Each method you might reach for wants to estimate a certain number of quantities, the five numbers are what you have to pay for them, and the arithmetic of that trade is short enough to do in the meeting.

Count the parameters before you pick the method

Hyndman and Kostenko put the constraint plainly in Foresight in 2007, in a piece specifically about minimum sample sizes for seasonal forecasting models. Their closing sentence is the one to keep: "One certainty is that it is always necessary to have more observations than parameters." They go on immediately to say that because real data carry a lot of random variation, "it is usually necessary to have many more observations than parameters."

That is a floor, and the floor is lower than people expect. For a straight line, an intercept and a slope, they point out that three observations is theoretically enough. Below three, as they put it, "the estimated parameters have infinite standard errors, and therefore prediction intervals will be infinitely wide." Three points buys you a fitted line and nothing to say about how wrong it might be.

With five annual editions and a straight line you have spent two parameters and kept three residual degrees of freedom. Those three are not spare. They are the entire basis on which the error variance gets estimated, which is the entire basis on which any interval around the forecast gets drawn.

How many parameters does an exponential smoothing model actually want?

This is where the count usually goes wrong, because the answer people carry in their heads is the number of smoothing weights.

Holt's linear trend method, as set out in Hyndman and Athanasopoulos's Forecasting: Principles and Practice, third edition (2021), has a level equation with a smoothing weight alpha and a trend equation with a smoothing weight beta star. Two weights. That sounds affordable.

It is not the whole bill. The method also needs a starting level and a starting trend, and both of those are estimated from the same five numbers. The book is explicit about this when it defines the information criterion for these models: the correction term uses k, "the total number of parameters and initial states that have been estimated (including the residual variance)".

So the count for Holt on an annual series is alpha, beta star, the initial level, the initial trend, and the residual variance. Five quantities. Five observations. You have exactly enough data to determine the parameters and nothing left over.

Hyndman and Kostenko make the same point for the seasonal case. For data with m seasons per year they count m plus 1 initial values and 3 smoothing parameters, so m plus 4 parameters in all, and they give the theoretical minimum as m plus 5 observations: 9 for quarterly data, 17 for monthly. Their qualification matters as much as the numbers: "these minima are not necessarily adequate to deal with randomness in the data."

The budget for five editions, worked

Take a show with attendance of 8,900, 9,400, 9,150, 10,100 and 10,600 across five editions. That series has one obvious feature, which is that it goes up, and one obvious complication, which is that the third edition went down.

Fit a mean. One parameter, four residual degrees of freedom. The mean is 9,630, and the forecast for the sixth edition is 9,630, which is lower than the last two editions and which nobody will accept.

Fit a straight line. Two parameters, three residual degrees of freedom. The least squares slope through those five points is 410 per edition, the intercept is 8,400, and the forecast for edition six is 8,400 plus six times 410, or 10,860. That is a defensible number and you can put a width on it, because three residual degrees of freedom is enough to estimate a variance, badly.

Fit Holt with an estimated initial state. Five parameters, zero residual degrees of freedom. The fit will be very close to perfect and the residual variance will be zero or undefined, which means the model has no way to tell you how uncertain it is. The information criterion cannot rescue you either. The corrected AIC for these models divides by T minus k minus 1, and with T of 5 and k of 5 that denominator is minus 1, so the correction term is negative and the criterion has stopped being a criterion.

That last line is the practical value of counting. You do not have to run the model to know the answer will be uninterpretable. The count told you.

What can five points honestly carry?

A level. A trend, if you accept that the trend estimate is the thing most sensitive to which five editions you happen to have.

Nothing else, on that series alone. No damping parameter, because a damping parameter is a third structural quantity competing for the same three residual degrees of freedom. No transformation parameter estimated from the data. No autoregressive term, since one lag costs a parameter and also costs an observation, taking an already short series to four usable points.

The uncomfortable implication is that most of what a forecasting package will happily fit to five annual observations is arithmetic performed on your data with no statistical content. The software returns a number. It returns a number for any input. What it cannot return is evidence that the number is better than the last edition plus a bit, which is the comparison that matters and which belongs with holding an edition out and testing, the subject of O12.

Where the extra observations actually come from

If the parameter budget on one show is five, the way out is to stop estimating everything from one show.

Hyndman and Kostenko say this themselves in the same 2007 piece, under the heading of using supplemental information: when data are scarce, consider using other information in addition to the available data, and they point at analogous time series and Bayesian pooling as the route. The idea has a direct reading for a portfolio. Eight shows with five editions each is forty observations, and the shape of a growth path is plausibly common across them even though the levels are not.

That is a different post, because the mechanics of partial pooling and the weight each show gets are worth their own treatment, and borrowing strength across shows is O14's subject. The point here is narrower. The parameter budget is a property of the data you choose to use, and choosing to use one show's history is what makes it five.

There is a second source that costs nothing. Within-year registration snapshots are observations too. A show with five editions and a weekly registration history has hundreds of data points describing the shape of the campaign, even though it has five describing the final total. Any method that forecasts the final number from the pacing curve is spending its parameters on a much longer series.

The number that gets quoted anyway

Somebody will produce a growth rate from these five points and it will go on a slide. It is worth knowing what that rate costs.

Five observations give four year-on-year changes: 500, minus 250, 950 and 500. The mean change is 425 and the standard deviation of those four changes is about 500. The standard error of the mean change is 500 divided by the square root of 4, which is 250. So the average annual increase is 425 plus or minus roughly 250 at one standard error, and a conventional interval on four degrees of freedom is wider still.

That is the honest state of the trend estimate on this series. It is positive, it is probably real, and it is not precise enough to distinguish a 3 per cent growth story from a 7 per cent one. Whether the interval you publish around it actually covers what happens is a separate and worse problem, and the coverage of intervals built on five editions is O20's.

Where this stops

Counting parameters tells you which methods are inadmissible. It does not tell you which of the admissible ones is right, and it can make a bad model look acceptable.

A straight line through five editions passes the count and can still be the wrong shape entirely. If the show moved venue in the third edition, the two segments either side of the move have different levels and the fitted line is a compromise between them that describes neither. The parameter budget has no opinion about that, because the budget only knows how many quantities you asked it to estimate, and it is satisfied by two.

The count is also silent about the thing Hyndman and Kostenko put at the centre of their argument, which is variability. Two shows with five editions each can support very different models if one of them is steady and the other swings 15 per cent a year. Their advice on published rules of thumb applies here too: such tables are misleading because they ignore the underlying variability of the data. The count sets a ceiling. The noise in your own series sets a lower one, and only your series knows where it is.

Take the five final numbers for one show, write down the method someone has asked you to fit, and list every quantity it will estimate, initial states and residual variance included. If that list is as long as the series, you have your answer before you open the software, and the honest deliverable that week is the trend with its standard error attached rather than a model. The rest of the argument for what a system like this needs to hold sits with the forecasting methods it runs on.