When AI generated event reports invent a number nobody measured

A colleague sent me a post-show summary last year that read beautifully. Three paragraphs, confident, specific. Registration up 6.7 per cent, international mix improved, exhibitor satisfaction holding. It went to a board pack.

The 6.7 per cent was not in any system. It came from a demo state that somebody had left switched on, and the language model had faithfully described what was on the screen.

Nobody lied. This is the characteristic failure of AI generated event reports, and the model did exactly what it was asked. The number was on the page, so it went in the summary, and the summary was correct about the page and wrong about the world.

Is this a hallucination problem?

Most conversation about hallucination treats it as a property of the model. Bigger models hallucinate less, better prompts hallucinate less, retrieval fixes it. Some of that is true and none of it addresses the case above, because the model did not hallucinate anything. It summarised its input accurately.

The defect was that the input was a rendered dashboard rather than a query result, and the dashboard had a value in it that no measurement produced.

This distinction matters operationally. If your AI summary is grounded in whatever the page happens to be displaying, then every fabricated placeholder, every hardcoded fallback, every demo toggle and every seeded random number in your reporting layer is now laundered into prose that reads like a finding. Prose is more portable than a chart. It gets pasted into emails and repeated in meetings, stripped of the context that would have let someone catch it.

I would go further. An organisation that puts a language model over a reporting layer it has not audited has made its reporting layer's defects considerably more dangerous, and has done so in a way that produces no error message.

What should grounding an event summary actually mean?

Grounding usually gets described as retrieval: fetch relevant context, put it in the prompt, tell the model to use only that. Fine as far as it goes.

For a post-show summary the useful version is stricter. The model should receive a small, explicit set of aggregates, each with a name, a value, a period, and a source, and the instruction that it may only reference values in that set. Not the page. Not a screenshot. Not a large dump of rows it might average incorrectly.

So instead of handing it a dashboard, you hand it something closer to:

registrations_total, 14382, 2026 edition, source fact_registration registrations_total_prior, 13511, 2025 edition, source fact_registration verified_attendance, 11204, 2026 edition, source fact_badge_scan exhibitor_count, 612, 2026 edition, source dim_exhibitor

This is what grounding a summary in aggregates means in practice. Now the model can write that registrations rose by 871 and that verified attendance was 78 per cent of registrations, and every figure in the output traces to a row somebody can open. If a value is missing, it is missing from the list, and the model has nothing to say about it, which is the correct behaviour.

The rule this enforces is worth stating plainly: never display a number nobody measured. Applied to AI output it becomes: never let the model see a number nobody measured, because it cannot tell the difference and you have given it no way to.

Make the model quote its arithmetic

The second control is cheap and catches a large share of the remaining errors.

Require every derived figure in the output to be accompanied by the operands. Registration is up 6.4 per cent, from 13,511 to 14,382. Verified attendance is 78 per cent of registrations, 11,204 of 14,382.

Two things happen. A reader can check the division in their head, which people do far more often than they check an unadorned percentage. And the model's own error rate on arithmetic becomes visible instead of hidden, because a wrong percentage sitting next to its own operands is obviously wrong, while a wrong percentage on its own is just a number.

There is a cost. The prose is slightly heavier. In a document that will be read by people who are going to make a decision about next year's floor plan, that is a good trade.

Ranges, when the underlying figure is a range

Some of what an organiser wants summarised is genuinely uncertain. A registration forecast four weeks out. An emissions estimate built from assumed travel modes. A footfall figure from sensors with a known error band.

The temptation is to give the model the point estimate, because point estimates make cleaner sentences. This is where confident prose does real damage, because the interval was the honest part and it has been thrown away before the model ever saw it.

Pass the interval and require it in the output. A forecast of 14,000 to 15,600 registrations at show open reads less impressively than 14,800 and is worth considerably more to the person deciding whether to open the overflow hall. If your model keeps collapsing the interval into a midpoint, that is a prompt problem with a straightforward fix, and it is worth testing for explicitly rather than assuming.

Tables are where summaries go wrong quietly

Most of what an organiser wants summarised arrives as a table. Registration by channel. Scans by exhibitor. Attendance by day and hall.

Language models handle small tables well and large ones unreliably, and the unreliability has a shape worth knowing. They are good at reading a specific cell you point at. They are worse at ranking, worse again at summing a column, and worst at any operation that requires holding the whole table in view, such as identifying which of two hundred exhibitors improved most against last year.

The practical consequence is that the aggregation should happen in SQL and the model should receive the answer, not the table. If you want the top five exhibitors by scan growth, compute the top five, hand over five rows, and ask for prose. Handing over eight thousand rows and asking which improved most is asking a model to do a database's job, and it will produce something plausible that is sometimes right.

A useful test: ask for the same summary twice from the same table and compare the numbers. If they differ, the model is computing rather than reading, and that computation belongs upstream.

Where a human has to sit

The pattern that seems to hold up is a review gate at the point where output leaves the building, not at the point where it is produced.

Draft freely. Let the model write the exhibitor narrative for six hundred booths if that is useful. But the transition from draft to something an exhibitor or a board sees should require a named person to approve it, and the interface should make the numbers easy to verify, which means showing the source aggregates next to the prose rather than in a separate system.

This is less exciting than an autonomous reporting pipeline and it is what actually ships. The review is also where you find out whether the system is any good, because the edits people make are your evaluation data. If reviewers are changing the same sentence every time, the prompt is wrong, and those edits are the raw material for an evaluation set. If they are changing numbers, something upstream is wrong and you should stop and find it.

Three questions that will tell you where you stand

If you are already running AI over your event reporting, these are worth asking this week.

Can you name the source of every figure in the last AI summary you published? Not the dashboard it appeared on. The table and the column. If the answer takes more than a few minutes for any one number, you have a provenance gap that the model is currently papering over.

Does your reporting layer contain any value that is not a measurement? Placeholder figures, demo modes, hardcoded fallbacks that display when a query returns nothing. Anything in that category is one prompt away from being asserted as fact in a document with your logo on it.

What happens when the data is missing? A good system says the figure is unavailable. A bad one shows last year's number, or a default, or the value from a different event, and does it silently. The audit trail for one platform I looked at recently found empty records rendering another event's content as confirmed, which is exactly this failure with no AI involved at all. Adding a model on top would have turned a visible oddity into a fluent paragraph.

The limit

None of this makes the summary true. It makes the summary faithful to a set of measurements, which is a smaller claim and the only one you can actually enforce.

If your badge scan coverage is 60 per cent and you treat scans as attendance, a perfectly grounded, fully sourced, arithmetic-quoting AI summary will confidently report an attendance figure that is 40 per cent low. Grounding protects you from invention. It does nothing about a measurement that is wrong at source, and it can make a wrong measurement more persuasive by wrapping it in careful prose.

That is an argument for spending the effort on the measurement layer first and the language layer second, which is the opposite of the order most teams are currently attempting. The same discipline applies to asking questions of event data in natural language, and to anything else on an AI platform reading from the warehouse.

Start by taking your most recent AI-written summary and marking up every number in it with the table and column it came from. Whatever you cannot trace in ten minutes is the thing to fix before you generate the next one.