Email address normalisation and the duplicate registrations it removes for free

The deduplication report for a spring edition shows four registrations that look like four people. [email protected], [email protected], [email protected] and [email protected]. One person, four badges printed, four rows in the attendance count, and one very annoyed delegate who received four confirmation emails.

Email address normalisation removes this class of duplicate with string handling alone, before any similarity model runs. It is the cheapest work in the whole matching pipeline and the one with the most rules attached, because the rules are set by mailbox providers rather than by the standards.

What the standard actually says

RFC 5321, the specification for the Simple Mail Transfer Protocol published by the IETF in October 2008 with John Klensin as editor, is precise on this and its answer is split.

Section 2.4 states that mailbox domains follow normal DNS rules and are therefore not case sensitive. So Gmail.com, GMAIL.COM and gmail.com are one domain, and lowercasing the domain is correct under the standard rather than a convenience.

The same section states that the local part of a mailbox must be treated as case sensitive, and that implementations must preserve the case of local parts. Under the standard, jkowalski and JKowalski are different mailboxes and a server is entitled to deliver them to different people.

That is the letter of it, and I would still lowercase the local part when building a match key. Almost no mailbox provider in production distinguishes case in the local part, the standard itself discourages exploiting the distinction on interoperability grounds, and the practical risk of treating two case variants as one person is far smaller than the certainty of splitting one person into two rows. The condition attached to that judgement is absolute: the lowercased value is a derived key used for matching, and the address you send to is always the one the registrant typed.

Why is the rule provider specific?

Because the interesting normalisations are decisions made by individual mailbox providers, and they differ.

Gmail Help documents that a gmail.com address owns all its dotted variants, so [email protected] and [email protected] reach the same mailbox and nobody else can register the other spelling. Removing dots from the local part is therefore safe for that provider and recovers a real class of duplicate, since people type their own address inconsistently across editions.

The same Gmail Help page says the opposite for organisations using Gmail on their own domain, where dots do change the address. That exception is the one that matters most for a B2B trade show audience, because a large share of your corporate registrants are on exactly that setup. Strip dots globally and you will merge [email protected] with [email protected], and at a firm of any size those can be two colleagues.

So the rule lives in a table keyed by domain, not in a global function. Consumer providers you have verified go in the table with their documented behaviour. Everything else gets the domain lowercased and the local part left alone. The table is short, it is auditable, and adding a provider to it is a decision somebody signs off rather than a regex somebody widens.

Plus tags, and the standard behind them

The plus sign convention has a specification too. RFC 5233, published by the IETF in January 2008 with Kenneth Murchison as author, describes subaddressing: adding detail information to the local part of an address, separated by a character sequence such as a plus sign, so that the address splits into a user part and a detail part. Its worked example delivers mail addressed to [email protected] into a folder belonging to the user ken.

Registrants use this deliberately. Somebody who tags their sign-ups will register as jkowalski+expo2026 one year and jkowalski+badge the next, and both reach the same inbox. Cutting the local part at the separator gives you the base mailbox and collapses those rows.

Two cautions. The separator is not universally a plus sign, and some providers use a different character, so the separator belongs in the same per-provider table as the dot rule. And a tagged address is sometimes deliberate on the delegate's part in a way that matters to you: a registrant using a different tag per promo code is the pattern behind promo code abuse, so collapsing tags for identity is right while quietly discarding the tag is not. Keep the detail part in its own column.

How many duplicates does this actually remove?

Work it on a plausible file and the number is larger than most teams expect.

Take 60,000 registration rows across two editions of one show. Counting distinct raw email strings gives 47,900 values.

Lowercase the whole address and recount. Say the distinct count falls to 47,180, so 720 rows were duplicates separated only by capitalisation, which arrives mostly from mobile keyboards that capitalise the first letter of a field by default.

Now apply the provider table to consumer domains only, removing dots and cutting at the separator where documented. Say the distinct count falls to 46,640, so a further 540 rows collapse.

The total is 1,260 duplicate registrations, which is 2.1 per cent of the file, found without a single similarity computation. Those rows were inflating your unique attendee count, splitting one person's history across two records, and triggering duplicate confirmation emails that make your registration system look broken to the person most likely to complain about it.

The saving compounds across the portfolio, because the same person registers for several shows and the same inconsistency follows them. It also improves everything downstream: a probabilistic model that gets a clean email key has a far stronger positive test to work with, which is the point of the field in the model described in cross event attendee matching.

Keeping the original, and where the key lives

The normalised address is a derived column. The raw address is the record.

Two rules make this survivable. The system never sends to the normalised value, since a dotless variant of a corporate address may not resolve at all and a stripped tag defeats the delegate's own filing. And the normalisation rule version is stored beside the key, so that when you add a provider to the table next year you can recompute every key from the originals and know which rows changed.

That is the same discipline as company name normalisation in J17, and the same reason applies: normalisation is lossy, the loss is deliberate, and the only way to change your mind later is to still have the input.

While the key is being built, one more column is worth populating from the same string. The domain, lowercased and separated from the local part, is what tells you whether a registration came from a corporate mailbox or a consumer one, and that split feeds audience quality reporting as well as the choice of comparison per field in J16, where email domain is one of the few fields that deserves exact matching and nothing else.

The rule that matters most operationally is where the normalisation runs. Every inbound file needs it: the registration export, the lead capture download, the CRM sync, the badge collection log and whatever the conference platform sends back. Implement it in five places and you have five slightly different keys within a year, because somebody adds a provider to one copy and not the others, and the drift is invisible until two systems disagree about how many people attended. Put it in one library that every ingest calls, version it, and log which version stamped each row. That single decision is worth more over three years than any refinement to the rules themselves.

Where this stops

Shared mailboxes defeat all of it. [email protected], info@, and marketing@ are addresses belonging to a function, and a group of colleagues registering through the same shared inbox produce records with an identical email key and different people behind them. Normalisation makes that worse, because it hands the matcher a perfect agreement on the strongest field in the record. Maintain a list of role addresses, exclude them from being used as a match key, and let the other fields carry the decision for those rows.

The second limit is that a matching email is strong positive evidence and a differing email is almost no evidence at all. Two rows with the same normalised key are the same person nearly always. Two rows with different keys tell you nothing, because people change employers and use personal addresses on their phones. Any rule that requires email agreement to declare a match will lose most of your genuine repeat attendees.

The third is that provider behaviour changes and nobody sends you a notice. A rule you verified in 2024 against a provider's help page may not describe 2026, and the failure is silent because merged records do not raise errors. Re-read the documentation for each provider in your table once a year, and record the date you checked next to the rule.

Take one show's registration file this week and run three counts: distinct raw addresses, distinct addresses after lowercasing, and distinct addresses after applying dot and tag rules to consumer domains only. The gaps between those three numbers are duplicate registrations you already have, and closing them costs an afternoon rather than a unified data project.