Registration record completeness measured against what the post show report needs

The data lead reported registration record completeness at 61 per cent and the room reacted the way rooms do, which is to say nobody knew what to do next. Sixty-one per cent of what? Somebody asked whether that was bad. It was the mean fill rate across all 34 fields on the registration form, and it was bad in the sense that it was unusable, because no decision anywhere in the business depends on the mean fill rate of 34 fields.

A completeness figure earns its place by naming the thing that breaks. So build it backwards, starting from the outputs you publish and ending at the fields those outputs read.

Two different things called completeness

Before the measurement, a distinction that keeps causing arguments. Black and Van Nederpelt, writing for the DAMA NL data quality working group in 2020, separate two dimensions that both get called completeness.

Completeness of records is the degree to which all required records are present in the data set. Their indicator is the percentage of required records that are present, and their examples are the kind that hurt: a product file missing some of the products, a city register missing some of the inhabitants.

Completeness of data values is the degree to which all required data values are present, indicated by the percentage of possible values that are present. Their example is an attribute left blank in some records.

On a registration file both are live and only one of them is usually measured. Missing values are visible. Missing records are invisible, and the standing case on an event file is an export that ran before onsite registration closed, which produces a file with a perfect value completeness rate and a missing tenth of the audience. Whenever you report a value completeness figure, say what population it is a share of and when the file was cut.

The rest of this post is about the value version, which is the one a queue can be built from.

Work backwards from the outputs

The method takes an afternoon.

Write down the outputs that leave the building. For most organisers there are three that matter to exhibitors and a board: the geographic breakdown, the job function mix, and the exhibitor scorecard. Everything else is either derived from those or internal.

For each output, open the query behind it and list the columns it reads. Not the columns available. The columns in the select, the where and the group by.

Take the union of those lists. That set is the completeness scope, and it will be far smaller than the form. Three outputs on a typical show need four fields: the badge identifier, company, country, and a job function derived from job title.

Then count records complete on that set. The set is the thing you report against, the individual field rates are the thing you queue work from, and the pass rate for a single field, including how to stop placeholders passing is the mechanic underneath both.

Two rules keep the scope honest. A field enters because a query reads it, so somebody has to open the query rather than ask the report owner what it uses, and those two answers differ more often than they agree. And a field that only appears in an optional appendix section stays out, because a completeness figure driven by an appendix will drive work that nobody outside the appendix benefits from.

There is a version of this exercise that goes wrong, which is to build the scope from what the form collects and then argue about which fields to drop. That runs the wrong way round and it never converges, because every field has somebody who wanted it. Starting from the output side gives you a list that arrives already agreed, since the outputs are already published and nobody disputes what is in them.

What does the post show report actually consume?

Less than the people who designed the form assumed, and more than the people who read the report realise.

On a file of 33,600 registrations, here is what the three outputs need and what the file holds.

The badge identifier is populated on all 33,600 rows, which is 100 per cent, as it should be on a system-generated key.

Company is populated on 31,900 rows, or 94.9 per cent. The exhibitor scorecard reads it, because exhibitors are shown which companies visited their stand.

Country is populated on 30,240 rows, exactly 90.0 per cent. The geographic breakdown reads it and nothing else can substitute.

A usable job function is present on 24,700 rows, or 73.5 per cent, and the shortfall here is not blank fields. It is free text job titles that no mapping resolves.

Records complete on all four fields: 22,800. Against 33,600 rows that is 67.9 per cent.

Set that against the 61 per cent that started the meeting, and against the figure you get if you demand all 34 form fields, which on this file is 90 records, or 0.3 per cent. The 0.3 per cent is arithmetically correct and completely inert. The 67.9 per cent tells three teams something they can act on, and the report design that decides which outputs count sits with the post show report itself.

Do the gaps overlap?

This is the check almost nobody runs and it changes the plan.

If the four fields failed independently, the share of records complete on all four would be the product of the four rates. That is 1.000 times 0.949 times 0.900 times 0.735, which comes to 0.628, or 62.8 per cent.

The file actually shows 67.9 per cent. Observed completeness is 5.1 percentage points above the independent expectation, which means the missing values are concentrated in the same records rather than spread across the file.

That is good news and it changes what you do. Concentrated gaps mean a source problem: one acquisition channel, one partner import, one form variant producing rows that are thin across several fields at once. Fixing the source recovers several fields per record. Spread gaps mean a field problem, and each field needs its own fix.

Run the same check the other way and it is equally informative. If observed completeness came in below the independent product, something is actively creating gaps in different records for different fields, which usually means several sources each dropping a different column.

The follow-up query is one group by. Compute the four field rates per acquisition channel and the pattern will name itself in about a minute.

What the audit rules put on the list regardless

Some fields are not yours to deprioritise.

UFI's auditing rules of June 2021 set out, in the calculation standards at Annex 2, what an approved event has to be able to certify. The organiser's space figure is total net exhibition space. The exhibitor count includes main exhibitors and co-exhibitors as direct exhibitors, with represented companies excluded. Total attendance is unique visitors plus exhibitor staff plus speakers plus media representatives, and the components have to be separable.

The clause that lands on the registration file is the one about nationality. A visitor's nationality is determined by place of residence, and Annex 2 states it must be stated on visitor registration forms. International event status depends on it: either five per cent international visitors or ten per cent international exhibitors.

Country at 90.0 per cent is a problem with an external deadline attached, and that is a different argument from the one about job function at 73.5 per cent. One has an auditor waiting. The other has an exhibitor who will be mildly disappointed by a chart. Ranking work by how many outputs a field carries, and by who is waiting on it, is the field ranking exercise that should run before any of these targets get set.

Where this stops

Measuring against outputs means the measurement moves when the outputs move. Add a section to the post-show report that cuts by company size and the completeness scope gains a field, the headline rate drops, and nothing about the data has changed. Anyone reading the series will see a fall that is really a scope change, so the scope has to be versioned alongside the number.

The method also cannot see fields that should be in a report and are not. If nobody ever built the industry sector breakdown because the field was too sparse to bother with, industry sector never enters the scope, its completeness is never measured, and the absence is self-reinforcing. Once a year it is worth listing the fields excluded from every output and asking whether any of them are excluded because they are empty rather than because they are useless.

And a complete record is not a correct one. All four fields populated tells you nothing about whether the company name is the current employer or whether the country is where the person actually lives. Completeness is the cheapest of the dimensions to measure and the weakest evidence of quality, which is why it should sit next to the others in a governed view of the record rather than standing in for them.

Open the query behind your geographic breakdown, list its columns, and run one count of records holding a usable value in every one of them. Divide by the row count and put that fraction in the post-show report with the column list printed underneath. It replaces an unactionable average with a number one person can own.