Skip to content

Forecast value added tells you whether the model beat last year

Forecasting methodsUpdated 2026-08-238 min read

In short

Forecast value added compares the accuracy of each step in a forecasting process against the step before it and against a naive baseline. A step that increases average error has negative value added and should be removed or corrected. Gilliland (2010) sets out the method in The Business Forecasting Deal.

The final attendance number in the board pack has been through four hands. A statistical baseline comes out of the pacing model. The audience acquisition lead adjusts it. The show director adjusts it again after the exhibitor call. Somebody rounds it before it goes in the deck.

Nobody has ever measured whether any of those steps helped. Forecast value added is the ledger that answers it, and the answer on most processes is that at least one step is subtracting accuracy.

Gilliland (2010) set the method out in "The Business Forecasting Deal", published by Wiley, as a way of evaluating the performance of each step in a forecasting process rather than only the performance of the finished number. The idea is deliberately unglamorous: score every stage against the stage before it, and score the whole chain against a forecast so simple that nobody could claim credit for it.

What does forecast value added actually measure?

The baseline is the naive forecast. For annual show editions that is last year's final figure, written down unchanged.

Every subsequent step gets two comparisons. Its accuracy against the naive baseline tells you whether the process as a whole is worth running. Its accuracy against the immediately preceding step tells you whether that particular step earns its place.

Both comparisons use the same error measure across the same set of editions. Absolute error in registrations works fine for a single show. For a portfolio you need something that pools, and scaled error is the usual choice, covered in O29.

The reason this framing has teeth is that it makes each participant's contribution separately visible. An accuracy figure for the published number tells you the chain works or does not. A value added ledger tells you which link is the problem, and that is the only version of the finding anyone can act on.

Morlidge (2014), writing in Foresight, gave the empirical background that makes the exercise worth the trouble. Using a relative absolute error metric, he presented evidence that many companies fail to achieve levels of relative error better than a simple same as last period naive forecast, and that around 50 per cent of individual forecasts fail to meet that benchmark. That is supply chain data rather than event data, and the structure of the problem is the same: a process with several human adjustment steps, scored against a rule that takes no effort at all.

A five edition ledger, worked

Take one show with five editions of history, and suppose the organisation kept what each step produced at the time, which most do somewhere in an email chain or an old board pack.

EditionActualNaiveModelAfter override
20229,4008,9009,1509,800
202310,9009,40010,30011,400
202411,60010,90011,85012,600
202512,05011,60011,90012,900
202612,40012,05012,60013,500

Work the three columns of absolute errors.

The naive errors are 500, 1,500, 700, 450 and 350. They sum to 3,500, so the mean absolute error is 700.

The model errors are 250, 600, 250, 150 and 200. They sum to 1,450, so the mean absolute error is 290.

The errors after the override are 400, 500, 1,000, 850 and 1,100. They sum to 3,850, so the mean absolute error is 770.

Now the ledger. The model against the naive baseline is 700 minus 290, which is 410 registrations of average error removed, a reduction of 59 per cent. That step is earning its keep comfortably.

The override against the model is 290 minus 770, which is minus 480. The override adds 480 registrations of average error to a forecast that was already good.

The published number against the naive baseline is 700 minus 770, which is minus 70. After all four hands, the number in the board pack is less accurate than writing down last year's total and going home.

The override had a bias, and bias is fixable

Before anyone deletes the override step, look at the direction of its errors.

All five overrides are above the actual: by 400, 500, 1,000, 850 and 1,100. Not one is low. The mean of those five is 3,850 over 5, which is 770 registrations of systematic upward bias.

A step that is wrong in a consistent direction is a different animal from a step that is wrong at random, and the difference is worth 500 registrations here. Subtract 770 from each override and recompute. The adjusted forecasts become 9,030, 10,630, 11,830, 12,130 and 12,730.

The new absolute errors are 370, 270, 230, 80 and 330. They sum to 1,280, so the mean absolute error is 256.

That beats the model's 290. The override was carrying real information about the show that the model did not have, and the entire apparent damage came from a constant upward push. Remove the push and the human step becomes the best forecast in the ledger.

This is the finding forecast value added exists to produce, and it is the reason the analysis is worth running before anyone starts arguing about whether sales teams should be allowed near a forecast. The step was not worthless. It was miscalibrated, and calibration is fixable in a spreadsheet.

Where the bias comes from is usually not mysterious. A number that gets quoted to exhibitors, or that anchors a revenue conversation, has a reason to lean high, and the people producing it are responding rationally to what the number is used for. The bias correction works as long as the incentive stays constant. It stops working the moment the show changes how the forecast is used, which is one reason to recompute the correction every year rather than baking a constant into a template.

What does a negative value added tell you to do?

Three responses are available and they are not equally good.

Correcting the bias, as above, is the first thing to try, because it preserves whatever information the step contains. It needs the errors to be consistently signed, which you can see by eye in five rows.

Constraining the step is second. If the override is allowed to move the model by at most a fixed percentage, the damage is bounded while the local knowledge still gets in. On the ledger above, capping the override at 3 per cent of the model figure would have cut the 2026 adjustment from 900 registrations to 378 and removed most of the loss.

Removing the step is third and it is the one people reach for first. It is correct when the errors are unsigned and large, meaning the step is adding noise with no recoverable pattern, and it is worth saying plainly that removing a step from a forecasting process is a legitimate improvement rather than an admission of failure.

There is a fourth outcome that the ledger sometimes produces, which is that the model itself has negative value added against the naive baseline. When that happens the honest response is to stop modelling this series and publish last year's number with a stated band around it. Recognising a series that should not be modelled at all is its own decision in O39.

Running it before you build anything

The order most teams use is backwards. They build a model, deploy it, and then look for a way to prove it helped.

Run the ledger first, on the last five editions, using whatever numbers already exist in old decks. It takes an afternoon and it costs nothing, and it tells you the size of the prize before you commit a quarter to a modelling project. A show whose naive mean absolute error is 700 registrations on a base of 12,000 has about 5.8 per cent of error available to attack, and a model that removes half of it is worth roughly 350 registrations of precision. Whether that is worth a quarter of engineering time is a business question, and it is answerable only once the 700 is on the table.

The same run also tells you which series are worth attacking at all, which overlaps with measuring how predictable a series is in O40.

Where this stops

Five editions means every average in the ledger is computed from five numbers, and five numbers do not support fine distinctions.

The difference between the model's 290 and the bias-corrected override's 256 is 34 registrations of average error, which on this sample is well inside the noise. Announcing that the corrected human forecast beats the model would be overreading the table. The difference between 290 and 770, on the other hand, is large enough that no amount of statistical caution makes it disappear, and that asymmetry is typical: value added analysis is reliable about disasters and unreliable about close calls. Where the line sits between those two cases is a question about power rather than about the ledger.

The deeper limit is that value added measures error reduction, and error reduction is a proxy for the thing you care about. A forecast that is 300 registrations better on average but wrong in the direction that matters on the one edition where a hall commitment was due has added measured value and destroyed real value. The ledger cannot see that, because it weights every edition equally and the decisions attached to those editions were not equal.

Pull the last five board packs and write the four columns out: actual, last year's actual, what the model said, what got published. Compute three mean absolute errors. If the published column loses to the second column, you have found something more useful than any model you could build this quarter, and the rest of your forecasting methods work should wait until it is fixed.

Questions people ask about forecast value added

What is the naive baseline in a forecast value added analysis?
For annual show editions it is repeating the previous edition's final figure. Every other step in the process, including the statistical model and any human override, is scored against the average absolute error that simple rule would have produced across the same set of editions.
What does negative forecast value added mean?
It means the step made the forecast worse. If the average absolute error after a sales override is larger than the average absolute error before it, the override is subtracting accuracy. That is a measurable finding about a process step, and the usual remedies are removing the step or correcting its bias.
How many editions do you need to run a forecast value added ledger?
Five is enough to be useful and not enough to be conclusive. Each average is computed from five absolute errors, so differences smaller than a few hundred registrations should be treated as noise. The ledger is most valuable when a step is dramatically wrong, which is more common than people expect.

Related reading

All forecasting methods articles