Building provenance for AI generated numbers into every report line

Two documents left the building in the same week. The post-show deck said verified attendance was 13,908. The exhibitor report said 14,061. The difference is 153 people, 1.1 per cent, and it took six days and four people to establish which one was right.

Both were right. Provenance for AI generated numbers is what turns that six days into about a minute, and it is cheaper to build than most teams assume because the system already knows every fact it needs to record.

What a provenance stamp has to carry

Seven fields, attached to the figure rather than to the report.

The metric name and its value, which is the part everybody already has. A fingerprint of the exact query text, usually a short hash, so two figures produced by the same query are provably the same measurement and two produced by different queries are provably not. The source tables read. The filter set in plain words, one line per predicate. The warehouse load watermark at the moment of the run, meaning the timestamp of the most recent successfully loaded batch in each source table. The run timestamp itself. And the requesting agent, whether a person, a scheduled job or a model.

The watermark is the field people leave out and the one that resolves most arguments. A run timestamp tells you when the query executed. A load watermark tells you what data existed when it executed, and those are different questions whenever a pipeline is running behind.

The query fingerprint deserves a note too, because there is a choice in how you compute it. Hashing the raw text means two logically identical queries with different whitespace get different fingerprints, which is noisy. Hashing a normalised form is better and harder. I would start with the raw text hash and accept the noise, because the fingerprint's job on day one is to answer whether two figures came from the same code, and a false difference prompts a look while a false match hides one. Get the cheap version shipped and normalise later.

Where AI is involved the seven fields extend by two. The prompt version that produced the sentence, and the model identifier including its version string. A summary written in June by one model and in July by its successor are different artefacts, and a portfolio that cannot tell them apart cannot investigate a change in tone or a new class of error.

Why is "according to" the right test for a report line?

Rashkin and colleagues published a framework in 2021 called Attributable to Identified Sources, built for exactly this problem in a different domain. Their formal test is a sentence you can say out loud: a statement is attributable to a set of sources when a generic hearer would affirm "According to P, E", where P is the source and E is the statement.

Apply it to a report line and the standard becomes concrete. "Verified attendance was 13,908" is a claim floating in space. "According to fact_badge_scan for the 2026 edition, with duplicate scans collapsed, as loaded at 23:10 on 19 March, verified attendance was 13,908" is a claim somebody can accept or refute. The second version is what a stamped figure lets your interface render on demand.

Their annotation pipeline has two stages, and the first one is the part worth stealing. Before annotators are shown the source at all, they judge whether the statement is interpretable on its own: whether pronouns resolve, whether the sentence is ambiguous, whether it can be understood without the surrounding context. Only statements that pass that stage go on to be judged for attribution.

That first stage maps directly onto report writing. A line reading "Attendance was up 4 per cent" fails it, because up against what, over what period, on which measure. Report lines get pasted into emails, forwarded, and quoted in meetings where the surrounding page is absent, so a line that only makes sense in place will be misread the moment it travels. Interpretability is a precondition for provenance rather than a nicety, because you cannot attribute a claim nobody can pin down.

Entity, activity and agent, applied to one figure

The W3C published PROV-DM as a Recommendation on 30 April 2013, and it defines provenance as "a record that describes the people, institutions, entities, and activities involved in producing, influencing, or delivering a piece of data or a thing". Its three core types give a report line a structure that survives contact with a real warehouse.

An entity is a thing with fixed aspects. The figure 13,908 is an entity. So is each source table it was derived from, and so is the generated paragraph that quotes it.

An activity is something occurring over a period that acts on entities. The query run at 09:15 on the Friday is an activity. So is the duplicate collapse job that ran at 23:10 the night before, and so is the summarisation step that turned the figure into a sentence.

An agent bears responsibility. The analyst who requested the report is an agent. The scheduled job is an agent. The model that wrote the sentence is an agent, and recording it as one is the small discipline that keeps "the system said so" from being an acceptable answer.

The value of using the published vocabulary rather than inventing your own is that the derivation chain composes. A sentence was generated by a model from a figure, which was produced by a query, which read tables loaded by a job. Four links, each with a responsible agent, and any of them can be walked in either direction.

Tracing a disputed attendance figure in under a minute

Back to the 153 people.

The exhibitor report's figure of 14,061 carries a stamp: query fingerprint a4f19c, source fact_badge_scan and dim_registration, filters edition 2026 and scan type entry, load watermark 18:05 on 19 March, run timestamp 18:40 on 19 March, requested by the exhibitor reporting job.

The deck's figure of 13,908 carries a different stamp: query fingerprint a4f19c, the same two source tables, the same filters, load watermark 23:10 on 19 March, run timestamp 09:15 on 20 March, requested by an analyst.

Same query. Same filters. Different watermark. Between 18:05 and 23:10 the duplicate scan collapse job ran and removed 153 double entries, which is 1.1 per cent of the final figure and entirely explicable: a scan at the hall entrance followed by a second scan when the same badge was re-presented after lunch. Neither report was wrong. One of them was early.

Reading those two stamps side by side takes about forty seconds. Without them the same question requires somebody to find both queries, work out when each ran, discover the collapse job exists, and confirm its schedule, which is the six days. The arithmetic on effort is not subtle: seven fields per figure against roughly twenty person-hours per dispute.

Where should the stamp live?

Beside the number, and travelling with it.

A separate data dictionary does not work, because the dispute happens in a meeting where nobody is going to open a second system. A footnote on the report does not work either, because the number gets copied out of the report. The two placements that survive are a hover or expander on the figure itself in the interface, and an embedded record in every export, so a downloaded spreadsheet carries a column of query fingerprints next to its values.

For generated prose the same rule applies per sentence. A paragraph containing four figures needs four stamps reachable from the four figures, which is a by-product of the sentence-level grounding trace Q4 describes and costs nothing extra once that exists.

The storage argument that gets raised is not real. A stamp with seven fields serialises to roughly 400 bytes. A show producing 220 stamped figures across 40 reports generates 8,800 stamps, which is about 3.5 MB per edition. A portfolio of eight shows over five years is under 150 MB, which is smaller than one exhibitor's floor plan file.

Where this stops

Provenance records where a number came from. It has no opinion on whether the number is any good.

A figure with a perfect stamp can still be produced by a query that joined the wrong way and quietly dropped 400 registrations, and the stamp will faithfully record the fingerprint of that wrong query. Catching the query itself needs the filter set and row count read before the number travels, which is Q3's subject and a different control entirely.

There is a worse case, and it is the reason to build this at all rather than in spite of it. A figure that never came from a query has no stamp, and once every real number carries one, the absence becomes the signal. That is how a placeholder or a demo value gets caught, and the several routes by which such a figure reaches a deck are what Q7 works through.

The honest limit is organisational. Stamps only stay accurate if every path that produces a number goes through the stamping layer, and in most event businesses at least one important figure is still calculated in a spreadsheet by a person who has done it that way for nine years. That figure will arrive unstamped, it will be correct, and it will make the absence-as-signal rule leak. Either bring it inside or mark it explicitly as manually produced, with a named owner.

Pick the last number that caused an argument in your organisation and write its seven fields out by hand. If you cannot fill in the load watermark, that is the field to build first, and it is the one that would have ended the argument in your own reporting layer on the first morning.