Skip to content

Why MAPE misleads on small counts such as a forty person session

Forecasting methodsUpdated 2026-08-238 min read

In short

Mean absolute percentage error divides by the actual value, so a small session produces a huge percentage from a tiny miss and a zero attendance figure makes the measure undefined. It also penalises over-forecasts harder than under-forecasts, which biases room allocation downward across a whole programme.

The conference programme dashboard has one row per session and a column headed accuracy. The keynote room shows 3 per cent. Breakout 14C shows 100 per cent. The average across 84 sessions is 41 per cent, and the programme director has drawn the obvious conclusion, which is that session-level forecasting does not work here.

The forecasts are probably fine. MAPE misleads on small counts in a way that has nothing to do with model quality, and a programme of breakout sessions is close to the worst possible place to use it.

Hyndman and Athanasopoulos (2021) put the structural problem plainly in the third edition of "Forecasting: Principles and Practice": measures based on percentage errors "have the disadvantage of being infinite or undefined if y_t=0 for any t in the period of interest, and having extreme values if any y_t is close to zero." A breakout programme contains sessions with six people in them and sessions that nobody attended.

What does a 100 per cent error on a session actually mean?

Breakout 14C was forecast at 12 and 6 people turned up. The absolute error is 6 people. Divide by the actual of 6 and you get 100 per cent.

The keynote was forecast at 1,200 and 1,140 turned up. The absolute error is 60 people. Divide by 1,140 and you get 5.26 per cent.

The keynote forecast was wrong by ten times as many people. MAPE scores it as nineteen times better. Nothing about that ranking corresponds to anything anyone has to do about it, because the operational consequences of the two errors are not remotely proportional to their percentages.

Average the two and you get 52.6 per cent, and the programme's headline accuracy figure is now being driven by a room that seats twenty. Scale that up: if a third of your 84 sessions are small breakouts, a third of your programme accuracy number is arithmetic noise from denominators near zero.

The zero case removes any doubt. One session on the Thursday afternoon had nobody in it. The absolute percentage error is a division by zero, and whatever your reporting tool did with that, whether it dropped the row, printed a blank, or silently substituted something, is now part of your accuracy figure and nobody knows which.

The asymmetry that pushes every room a size too small

This is the part that costs money, and it is less well known than the small-denominator problem.

Because the denominator is the actual, and the actual is whatever happened, the same absolute miss scores differently depending on direction. Forecast 12 for a session that draws 6, and the score is 6 over 6, which is 100 per cent. Forecast 6 for a session that draws 12, and the score is 6 over 12, which is 50 per cent.

Same miss, same six people, twice the penalty for the over-forecast.

Now consider what happens when anyone optimises against that measure, whether by tuning a model or by learning from being told off in a review. The cheapest way to reduce MAPE is to forecast low. A systematically low forecast collects half-price penalties, and over a programme of 84 sessions that adds up to a visibly better accuracy number and a set of room allocations that are consistently one size too small.

The operational signature is recognisable. People standing at the back, sessions closed at the door, and a dashboard reporting that forecasting accuracy improved year on year. Both things are true at once and the measure caused them.

The size of the pull is easy to estimate on your own programme. Take every session, shade the forecast down by ten per cent, and recompute the MAPE. On a set of sessions where the current forecasts are roughly unbiased, that deliberate downward shift will usually improve the reported figure by a point or two, because the sessions it now under-forecasts are charged at half rate while the sessions it stops over-forecasting were being charged at full rate. If shading your numbers down improves your score, the score is not measuring accuracy.

Why is sMAPE not the fix?

The usual proposed repair is symmetric MAPE, which divides by the average of the actual and the forecast rather than the actual alone.

It does remove the direction asymmetry in the simple case. Forecast 12 with an actual of 6 gives 200 times 6 divided by 18, which is 66.7 per cent. Forecast 6 with an actual of 12 gives the same 66.7 per cent. So far so good.

The trouble is elsewhere. Hyndman and Athanasopoulos note that "the value of sMAPE can be negative, so it is not really a measure of 'absolute percentage errors' at all", and they state directly that Hyndman and Koehler (2006) recommend that the sMAPE not be used. A measure whose sign is not guaranteed cannot be averaged and interpreted as an error rate.

It also inherits the small-count problem in a slightly disguised form. When a session draws 2 people and the forecast was 8, sMAPE gives 200 times 6 divided by 10, which is 120 per cent. The number is bounded rather than infinite, which feels like progress, but a bounded nonsense number is still a nonsense number, and it now sits in an average alongside the keynote's 5 per cent as though the two were commensurable.

The MAD over mean ratio, worked on five sessions

Kolassa and Schutz (2007), in Foresight, set out the alternative in "Advantages of the MAD/Mean Ratio over the MAPE". Their argument is that the ratio of mean absolute deviation to the mean can be read as a weighted mean of absolute percentage errors, weighted by the size of each item, and that it stays well defined for series with intermittent and near-zero values.

Compute it on five sessions. Actuals of 6, 18, 40, 120 and 1,140. Forecasts of 12, 14, 46, 100 and 1,200.

The absolute errors are 6, 4, 6, 20 and 60, summing to 96. The actuals sum to 1,324. The MAD over mean ratio is 96 divided by 1,324, which is 7.25 per cent.

Now the same five rows under MAPE. The individual percentage errors are 100, 22.2, 15, 16.7 and 5.3. Their mean is 159.2 over 5, which is 31.8 per cent.

Same forecasts, same outcomes, and two headline numbers that differ by a factor of more than four. The 7.25 per cent says that across the whole set you got 96 people wrong out of 1,324, which is a checkable statement about the programme. The 31.8 per cent says that the average of five ratios is 31.8, which is true and describes nothing anybody schedules against.

What to report for a room decision

Neither number answers the question a programme team is actually asking, which is whether the room assigned to session 14C is the right size.

For that, report the absolute error in people next to the capacity of the assigned room. A miss of 12 people in a room seating 40 is a 30 per cent capacity swing and will be visible on the day. A miss of 12 people in a room seating 400 is invisible. The comparison that matters is error against capacity, and both quantities are in people, so no percentage of the actual ever enters the calculation.

Then use the MAD over mean ratio as the single programme-level summary, because that one is genuinely a portfolio question and the weighting by size is the correct weighting for it.

Two numbers, each answering the question it can answer. The single accuracy percentage per session, averaged across the programme, answers neither, and it is the thing currently on the dashboard.

There is a third view worth adding once the first two are in place, which is the count of rooms that were the wrong size in a way that mattered. Define a threshold, say a forecast that missed by more than a fifth of the assigned capacity, and report how many of the 84 sessions crossed it in each direction. That converts the whole question into two integers a programme team can act on, and it survives the small-count problem entirely because no division by an attendance figure ever happens.

Small counts are one problem. Counts that are frequently zero, because a category only runs in some editions, are a different problem with a different treatment, and that one belongs to intermittent demand in O37 and count models for sparse categories in O38.

Where this stops

The MAD over mean ratio hides exactly what MAPE exaggerates, and being clear about that is the price of recommending it.

Because it weights by size, a programme-level ratio of 7.25 per cent can coexist with every small session being forecast atrociously. The keynote dominates the denominator, and 30 breakouts each wrong by five people contribute 150 to a numerator that the keynote's own error already puts at 60. If small sessions are where your operational pain is, the aggregate ratio will not show it, and you need the per-session error-against-capacity view as well.

There is also no scaling in it. A 7.25 per cent ratio on one show cannot be compared with 7.25 per cent on another show whose sessions are twice the size, because the underlying difficulty differs. Making accuracy comparable across shows requires dividing by a benchmark rather than by a level, and that is what scaled error does in O29.

Open your last programme's session-level accuracy report and count two things: how many rows had an actual below 25, and how many had an actual of zero. Then recompute the headline figure as total absolute error over total attendance. If the two headline numbers differ by more than a few points, the one currently in circulation has been describing your denominators rather than your forecasts, and everything downstream in forecasting methods has been judged against it.

Questions people ask about mape misleading small counts

Why does MAPE penalise over-forecasting more than under-forecasting?
The denominator is the actual value, which is fixed, so the percentage depends only on the size of the miss relative to what happened. Forecasting 12 when 6 turn up scores 100 per cent. Forecasting 6 when 12 turn up scores 50 per cent. The same absolute miss of six people costs twice as much in one direction.
Is sMAPE a fix for MAPE on small counts?
No. Hyndman and Koehler (2006) recommend that sMAPE not be used, and the reason is structural: because the denominator mixes the actual and the forecast, the measure can take negative values, so it is not a percentage of anything stable. It also still breaks when both the actual and the forecast are zero.
What should you report instead of MAPE for session attendance?
For a room allocation, report the absolute error in people alongside the capacity of the room assigned. A miss of twelve people matters in a room seating forty and does not matter in one seating four hundred. For a programme-level figure, the ratio of total absolute error to total attendance is a defensible summary.

Related reading

All forecasting methods articles