Cross event attendee matching, and why one buyer becomes four records

Someone in your commercial team asks a question that sounds simple. How many people came to more than one of our shows last year?

The answer takes a week, comes back with a caveat, and nobody quite believes it. The caveat is always the same. We matched on email, and a lot of people use different email addresses.

That is the whole problem. Cross event attendee matching is the discipline that answers it properly, and it is worth being precise about why the usual fixes make it worse.

How many records does one buyer actually generate?

Take a single buyer. She registers for your kitchen and bath show in 2023 using her work address at a regional dealer. She changes jobs. In 2024 she registers for your hospitality design show with a new employer and a new work address, and because the registration form asks for a mobile number she gives a personal one. In 2025 her assistant registers her for both, using a shared inbox, and spells her surname the way it sounds. In 2026 she registers herself again, from her phone, using a personal Gmail because she cannot remember the work password.

Four records. One person. Nothing in any of the four points at any of the others.

Now scale that. A mid-sized portfolio with eight shows and five years of history has forty event instances and, depending on the verticals, somewhere between two hundred thousand and two million registration rows. The proportion of those rows that belong to a person who appears more than once is exactly the number your commercial team wants, and it is unmeasurable by exact match on any single field.

Worse, the exact-match answer is not merely low. It is biased in a specific direction. It systematically undercounts the people you care about most, because senior buyers change employers more often than junior ones, and because the people who attend several of your shows are the ones most likely to have registered through several different routes.

Deterministic matching, and where it stops

The first thing every team tries is a rule. Match on lowercased email. Then match on surname plus company. Then match on surname plus postcode.

Rules are fine. They are fast, they are explainable, and for a large share of your file they are correct. The trouble is that each additional rule you bolt on either misses cases or starts merging strangers, and there is no dial to trade one against the other.

Consider surname plus company. In a registration file of two hundred thousand rows from a US show, you will have several hundred people surnamed Smith, and you will have twenty separate spellings of the same national retailer. Surname plus company matches the wrong Smiths and misses the right ones, and adding first initial fixes some of the first problem and none of the second.

Consider email. Two records with the same email are almost certainly the same person, which makes it a good rule. Two records with different emails tell you nothing at all, which makes it useless as a negative test. Any rule built by chaining tests together inherits the weakest test in the chain.

The reason the rule approach runs out is that it forces a binary decision on evidence that is not binary. Agreement on surname is weak evidence. Agreement on an unusual surname is strong evidence. A rule cannot express that difference. A model can.

The 1969 answer, which is still the answer

Fellegi and Sunter published the formal treatment in the Journal of the American Statistical Association in 1969, building on work by Newcombe and colleagues in Science ten years earlier on linking birth and marriage records in British Columbia. Their framing is the one to hold in your head.

You are not comparing records. You are classifying pairs. Every pair of records is either a true match or a true non-match, and you have to decide which without being told. For each pair you compute a comparison vector: does the surname agree, does the first name agree, does the email agree, does the company agree, and so on for whatever fields you have.

For each field you then need two probabilities. The m-probability is the chance that the field agrees given that the pair really is the same person. It is below one because people mistype, change names, and move. The u-probability is the chance that the field agrees given that the pair is two different people. For surname in a US file it is small. For country in a file where eighty per cent of registrations are domestic, it is very large.

The weight contributed by a field that agrees is the base-two logarithm of m divided by u. The weight contributed by a field that disagrees is the log of one minus m divided by one minus u. Sum the weights across fields and you have a score for the pair.

The arithmetic is worth doing once by hand, because it makes the behaviour obvious.

Suppose surname agrees between two records. You estimate that among true matches surnames agree ninety per cent of the time, so m is 0.90. You estimate that among random pairs from your file, surnames agree in one case in two thousand, so u is 0.0005. The agreement weight is log2 of 0.90 divided by 0.0005, which is log2 of 1800, about 10.8.

Now country agrees. Among true matches, say m is 0.98. Among random pairs, because most of your file is one country, u is 0.65. The agreement weight is log2 of 0.98 over 0.65, which is log2 of 1.51, about 0.59.

Surname agreement is worth roughly eighteen times as much as country agreement, and the model derived that from the data rather than from anyone's opinion. That is the entire appeal.

Disagreement carries negative weight by the same logic. If email disagrees, and among true matches email agrees only sixty per cent of the time while among random pairs it agrees essentially never, the disagreement weight is log2 of 0.40 over 0.9999, about minus 1.3. Small. Email disagreement is weak evidence against, which matches intuition and is the opposite of what the rule-based approach assumes.

How do you match Kathryn to Katherine?

The weights above assume a field either agrees or does not. Real registration data does not cooperate. Kathryn and Katherine. Bob and Robert. Muller and Müller and Mueller. A surname typed into a mobile form at a taxi rank.

The standard treatment is a string similarity function that returns a number between zero and one, which you then band into agreement levels rather than a single yes or no. Jaro published his comparator in 1989 while working on the 1985 Tampa census test, and Winkler added the prefix adjustment in 1990 that gives extra credit to strings agreeing on their first few characters, which is where human typing errors are least likely.

Jaro-Winkler on Kathryn against Katherine returns roughly 0.95. On Smith against Smyth, roughly 0.93. On Smith against Jones, about 0.44. Band that into three levels, exact, close, and different, estimate m and u for each level, and the model handles nicknames and typing errors without anyone writing a rule about Bob.

Company names need a different treatment, because the failure mode is different. Nobody mistypes a company name into something unrecognisable. They write a shorter version of it, or a longer one, or a divisional one. Acme Retail Group Inc, Acme Retail, and Acme Retail Group LLC are the same firm. A token-based measure such as Jaccard overlap after stripping legal suffixes handles that far better than a character-based one, because it is measuring how much of the name is shared rather than how many characters were transposed.

Two thresholds, not one

The output of the scoring stage is a number per pair. What you do with it decides whether the whole exercise is trusted.

The instinct is to pick one cut-off. Above it, merge. Below it, do not. Fellegi and Sunter argued for two, and the argument holds up. Above an upper threshold, the evidence is strong enough to merge automatically. Below a lower threshold, the evidence is weak enough to leave alone. Between them sits a band where a human should look.

The two numbers are chosen from the error you are prepared to accept in each direction, and those errors are not symmetric in an event business. A false merge collapses two buyers into one, which means one of them stops receiving your marketing and disappears from the audience count. That failure is invisible and permanent. A false non-match leaves you with a duplicate, which is visible, annoying, and recoverable.

So the upper threshold should be set conservatively and the review band should be wide enough to catch the interesting cases, sized against how many pairs a person can actually review. A steward working through a queue can sensibly handle a few hundred decisions a day for a short period. If your band contains forty thousand pairs, the band is wrong, or you need to sample it and use the sample to move the thresholds.

What this costs you in compute, and the fix

Comparing every pair in a file of two hundred thousand records means roughly twenty billion comparisons. That is not a scale problem you optimise your way out of.

The standard answer is blocking: only compare pairs that share some cheap key, such as the first three characters of the surname combined with the postcode prefix. It is the single most consequential design decision in a matching pipeline, because any true pair whose records land in different blocks can never be found, no matter how good the scoring is. Christen's 2012 book on data matching treats it at length and it deserves its own consideration, which is not this post's.

Where this stops

Probabilistic matching gives you calibrated evidence. It does not give you truth, and it will be confidently wrong in two situations that event data produces constantly.

The first is shared identity. Two people at the same small dealership, same surname because they are related, same address, same company, both attending. The model will merge them and no amount of threshold tuning fixes it, because the evidence genuinely does point one way.

The second is the missing-field trap. If half your registration rows have no phone number, the model cannot use phone disagreement as evidence, and pairs from sparse rows will systematically score lower than pairs from complete rows. Your match rate will then look worse for the channels that collect fewer fields, which is usually the cheap self-service ones, and someone will conclude those channels bring worse-quality registrants when the truth is you know less about them.

Neither problem is a reason to avoid the method, and neither excuses skipping the work of evaluating how well the matcher performs. Both are reasons to report a match rate with the field-completeness of the underlying rows attached, and to keep merge decisions reversible with an audit trail. Which value wins when two matched records disagree is a separate decision, governed by survivorship rules, and it sits downstream of everything here inside a unified data layer.

The first step is smaller than a project. Take one show's registration file for two consecutive editions, strip the emails to lowercase, and count how many rows in the later file have an email that appears in the earlier one. Then count how many have a surname and company that agree without the email agreeing. The ratio between those two numbers is the size of the problem you have, measured on your own data, and it takes an afternoon.