The event data quality dimensions worth measuring on a registration file

Somebody puts a data quality figure on a slide in the January planning review. Ninety-four per cent. The show director asks what the other six per cent is, and the room goes quiet, because the number came from a vendor dashboard and nobody in the room built it.

The six event data quality dimensions everyone quotes are real and they are useful. They arrive with a problem attached. Completeness, uniqueness, consistency, timeliness, validity and accuracy are abstractions written for banks and government registers, and on a registration export they float free of anything you can query until each one is tied to a specific column and a specific rule.

So tie them down. One show, one file, six numbers, published with their definitions. Then have the argument about which of them matters.

Where the six dimensions come from

A DAMA UK working group published a short paper in October 2013 setting out six primary dimensions for data quality assessment, and that set has since become the default vocabulary. The UK Government Data Quality Hub adopted it directly in the Government Data Quality Framework of 3 December 2020, which says it draws on the Data Management Body of Knowledge and on DAMA UK's data quality dimensions paper, and then gives one-line definitions worth quoting because they are unusually blunt.

Completeness is "the degree to which records are present". Uniqueness is "the degree to which there is no duplication in records". Consistency is "the degree to which values in a data set do not contradict other values representing the same entity". Timeliness is "the degree to which the data is an accurate reflection of the period that they represent, and that the data and its values are up to date". Validity is "the degree to which the data is in the range and format expected". Accuracy is "the degree to which data matches reality".

Read those against your registration file and the trouble is obvious. Every one of them is a degree of something, and none of them names a column. Two competent people will produce two different completeness figures from the same export and both will be defensible, which is how you end up with an unsourceable ninety-four per cent on a slide.

What does each dimension bind to on a registration file?

The binding is the work. Here is the set I use, and I would defend each choice over the obvious alternative.

Uniqueness binds to the badge identifier. One badge, one row, for the population you have declared. Not email, because two people at a small dealership legitimately share an inbox, and a uniqueness test on email will report a duplication problem that is really a shared-mailbox problem. Not the resolved person either, because that is an output of a matching pipeline and its errors would land inside a quality metric that is meant to be independent of it.

Validity binds to the country code. Country has a published value domain, ISO 3166-1 alpha-2, so membership is a mechanical test with no judgement in it. It also has a business reason to be right. UFI's auditing rules of June 2021 state in Annex 2 that a visitor's nationality is determined by place of residence and must be stated on visitor registration forms, and the international-event threshold is calculated from exactly that field.

Timeliness binds to the export timestamp. The dimension is usually explained as data being up to date, which invites hand-waving. On an event file it has a hard version: what proportion of the registrations that existed at show close are in the file the report was built from? Onsite registrations are the ones that go missing, and they are systematically different from the rest.

Completeness binds to the fields the published report consumes. Using the whole form as the denominator produces a figure that drives no work, for reasons worked through in measuring completeness against the report.

Consistency binds to a rule between two fields. Postcode format against country is the cheapest one that catches real errors. A row reading FR with a five-digit postcode passes. A row reading FR with SW1A 1AA does not.

Accuracy binds to nothing you own, which I come back to below.

Six numbers from one show

Take an edition with 41,800 registration rows in the export used to build the post-show report.

Uniqueness first, because it is the easiest. The file holds 41,362 distinct badge identifiers across 41,800 rows, so 438 rows are repeats of a badge already present. Uniqueness is 41,362 divided by 41,800, which is 98.9 per cent.

Validity next. 39,244 rows hold a country value that appears in the ISO 3166-1 alpha-2 list. That is 39,244 over 41,800, or 93.9 per cent. The 2,556 failures are mostly free text: United States, USA, U.S.A., and a few hundred blanks.

Consistency, on the postcode rule, applied only to the 39,244 rows that have a valid country to test against. 2,110 of them carry a postcode that does not match the pattern for the country in the same row, so consistency is 1 minus 2,110 over 39,244, or 94.6 per cent.

Timeliness. The export ran at 07:00 on the second morning of a three-day show, because that is when the analyst had time. 38,900 of the 41,800 registrations were created before that timestamp, so the file represents 93.1 per cent of the population it claims to describe, and the 2,900 it misses are almost entirely onsite.

Completeness, against the six fields the exhibitor scorecard and the demographic breakdown actually read. 34,700 rows hold all six, giving 83.0 per cent.

Accuracy is unmeasured, and the report says so.

Five numbers and one honest gap beats a single composite that nobody can take apart. If you do want the composite, the weights have to be published with it.

Which dimension should an event team fix first?

Not the lowest one. Completeness at 83.0 per cent looks like the emergency and usually is not, because the missing values cluster in optional profile fields that degrade a chart rather than break it.

Timeliness at 93.1 per cent is the one I would move on, because its failure mode is silent and directional. Every missing registration is an onsite registration, onsite registrants skew toward local visitors and toward the trades, and every downstream cut of the file inherits that skew without any indication that it has. A geographic breakdown built from that export understates local attendance and nobody looking at the chart can tell. Fixing it costs one line in a scheduler.

Uniqueness at 98.9 per cent is the one to fix second, and for the opposite reason. It is small, it is visible, and the 438 rows will otherwise reappear in every count anyone runs for the next year.

Validity at 93.9 per cent is a normalisation job with a known answer and no analytical judgement in it, so it belongs in a queue rather than in a discussion.

Can accuracy be measured on a registration file at all?

Not from inside the file. Accuracy compares a stored value against reality, and reality is not in your export. A job title of Director of Procurement is valid, complete, unique and consistent, and it can still be two promotions out of date.

The two honest options both cost something. An external reference, where you match a sample against a source that carries the same attribute, tells you about the fields that reference happens to hold and nothing about the rest. Or a hand-checked sample: draw 300 rows, have somebody verify company and job title against a public profile, and report the result as an estimate with its sample size attached.

Three hundred rows checked by one person over two days gives a proportion with a margin of error around plus or minus 3 percentage points at 95 per cent confidence, which is precise enough to distinguish 70 per cent accurate from 90 per cent accurate and not precise enough to track quarter-on-quarter movement. That is usually the right trade. Most teams do not need to know whether title accuracy moved by two points. They need to know whether it is 90 or 60, because those two numbers imply completely different things about whether a governed record across systems can be built on top of the file at all.

If you will not pay for either option, write accuracy as unmeasured and move on. An empty cell labelled unmeasured is information. A guess is not.

Where this stops

Six dimensions scored on one file tells you about one file. It says nothing about the registration process that produced it, and the process is usually where the defect lives.

A validity rate of 93.9 per cent on country has three quite different causes with three different fixes. A free text field on the form. A dropdown that a partner's landing page bypasses. An import from a media partner's list that arrives with country in a different format. The dimension score is identical in all three cases and gives you no way to tell them apart. You need the profile underneath it, which is what a one time pass over every column is for.

The second limit is that these six do not exhaust the ways a registration file can be wrong. The DAMA UK set is a starting vocabulary, and a Dutch DAMA working group counted sixty dimensions in circulation when Black and Van Nederpelt catalogued them in 2020. Traceability, for instance, is not in the six, and on an event file the question of which system a row came from is usually more urgent than its consistency score.

The third is the one that matters most in practice. A dimension score is only actionable if the same person can change the thing it measures. Timeliness is owned by whoever schedules the export. Validity on country is owned by whoever owns the form. If the six numbers land on a slide with no name against each one, they will be read out at the next review, unchanged.

Pick one show that has already closed. Run the uniqueness test on badge identifier and the validity test on country, which between them are about twenty lines of SQL, and put both numbers in the post-show report with their definitions written underneath. Two measured dimensions with visible arithmetic will do more for the next planning review than six estimated ones.