Reactivation email measurement for people who attended two editions ago

There are 9,000 people in your database who came to the show once and have not been back. Somebody suggests mailing them, the mailing goes out, 421 of them register, and the campaign report claims 421 registrations recovered.

Reactivation email measurement has to answer a different question, which is how many of those 421 would have registered without the mail. Lapsed attendees are not strangers. They know the show, they may already have it in the diary, and a share of them were coming regardless.

Split the lapsed file by last attended edition

Start with the cohort definition, because every number downstream depends on it.

Split by the edition each person last attended, never by months since last activity. A person who last attended the 2025 edition and a person who last attended the 2023 edition are in completely different states, and a rolling twelve month window will mix them depending on where the show sits in the calendar. Editions are the unit the audience thinks in.

Three cohorts is usually enough for a show running annually: last attended one edition ago, two editions ago, three or more editions ago. Take 9,000 lapsed records and suppose they split 3,900, 3,100 and 2,000.

Attended is doing real work in that sentence. A person who registered for the 2024 edition and never scanned in is a different case with a different message, and mixing no-shows into a reactivation cohort will make the cohort look worse than it is. Build the cohorts from verified attendance where you have it and label them honestly where you do not.

Why hold a tenth of each cohort back?

Because the alternative is a number that cannot be interrogated. A gross count of registrations from a mailed file measures the mailing plus the show's own pull plus every other thing you were doing that month.

Randomise inside each cohort and hold back ten per cent. The held-back people receive nothing from this programme, and they stay held back for the whole measurement window, which means they must also be excluded from any other reactivation touch, or the comparison is meaningless.

Two practical rules. Randomise at the person level and store the assignment, so a person cannot drift between groups when the file is rebuilt next week. And hold back inside each cohort separately, so the mailed and held groups have the same mix of one, two and three edition lapsers. A single holdout drawn from the whole file will not, and the comparison then confounds cohort with treatment.

The hard part of running a holdout in an event business is defending it internally, since the randomisation itself takes one line of code. Somebody will point out that 900 people are being deliberately left alone during a campaign with a registration target, and they are right. The answer is that the same 900 people tell you whether the other 8,100 mailings did anything, and that a programme nobody has ever tested tends to grow every year on the strength of a gross number. Agree the holdout with whoever owns the target before the send, not after.

The cohort table, worked

Mail the 8,100, hold back the 900, and count registrations completed within twenty one days of the send.

In prose, because the arithmetic is the point. The one edition cohort registered at 7.81 per cent when mailed against 4.62 per cent when held back. The two edition cohort registered at 4.01 per cent against 2.26 per cent. The three edition cohort registered at 1.94 per cent against 1.00 per cent.

Incremental registrations are the difference applied to the mailed group. That is 3.19 per cent of 3,510, which is 112, plus 1.75 per cent of 2,790, which is 49, plus 0.94 per cent of 1,800, which is 17. The mailing produced about 178 incremental registrations against a gross count of 421, so 42 per cent of the reported figure is attributable to the mail and 58 per cent would have happened anyway.

Now the caution that keeps this honest. A holdout of 390 people is small, and only the first cohort's difference clears a conventional two proportion test at five per cent significance. The second and third are positive and unproven. Pooled across all three cohorts, 5.20 per cent against 3.00 per cent on 8,100 and 900, the difference does clear it comfortably. So the total is measurable and the per cohort splits are directional, which is a normal outcome and worth writing on the slide. Sizing a holdout properly is a calculation in its own right and it belongs with designing a test that can detect the lift you expect.

What the sender rules cost you on cold addresses

A reactivation mailing is the riskiest send in the calendar, because the addresses are old and the recipients have not heard from you in two years.

Google's email sender guidelines, in force since 1 February 2024 for anyone sending more than 5,000 messages a day to Gmail accounts, require authentication with SPF and DKIM, a DMARC record that may be set to none, one-click unsubscribe on marketing messages, and spam rates reported in Postmaster Tools kept below 0.30 per cent, with the guidance recommending below 0.10 per cent. Yahoo published matching requirements with enforcement beginning in February 2024, asking for both SPF and DKIM, a valid DMARC policy of at least p=none, a working one-click unsubscribe, spam rates below 0.3 per cent, and unsubscribes honoured within two days.

Put that against the cohort table. Suppose your healthy monthly programme delivers 40,000 messages at a complaint rate of 0.05 per cent, which is 20 complaints. Add a cold send of 9,000 that complains at 0.5 per cent, which is 45. The blended rate is 65 complaints over 49,000 delivered, or 0.133 per cent, which is comfortably under the 0.30 ceiling and over the 0.10 target. Send the same cold file in one drop during a quiet month with only 8,000 other messages and the blended rate is 65 over 17,000, or 0.38 per cent, which is over the line for both providers.

The volume of healthy mail around a cold send is what protects you. Split the reactivation file across three or four drops, put each one inside a week with heavy engaged sending, and warm the oldest cohort last.

Bounces work the same way. If the three edition cohort hard bounces at 14 per cent, 252 of those 1,800 addresses are dead, and a single drop containing them will do more damage to your sending reputation than the 17 incremental registrations are worth. Validate the oldest cohort before mailing it, or mail it in slices of a few hundred.

Which cohorts are worth mailing at all?

I would mail the first cohort every edition without testing it again, keep testing the second, and treat the third as a candidate for a different channel entirely.

The arithmetic supports that. At 112 incremental registrations from 3,510 mailed, the first cohort returns a registration for every 31 messages. The second returns one for every 57. The third returns one for every 106 messages, and those messages carry the highest complaint and bounce risk in the file.

The decision on the third cohort is not really an email decision. Those people are better reached as a suppressed-from-email audience on paid channels, or through the exhibitors who sold to them originally, and the reactivation mailing is competing with uses of the same address that carry less reputational cost.

Whatever you decide, the report has to name the window and the denominator. Registrations within twenty one days of the send, over people mailed, cohort by cohort. Any credit assigned to a specific mailing needs the send identifier written onto the registration row, which is the mechanic in attributing a registration to the send that produced it, and none of this reporting should carry an open rate, for reasons set out in why opens now measure infrastructure.

Where this stops

The holdout only measures what the mailing added on top of everything else running that month, so a reactivation programme tested during a heavy paid push will look weaker than the same programme tested in a quiet week. That behaviour follows from what incremental means, and it is why the test result travels badly between editions.

A more serious limit is that the people you can reach are the people whose addresses still work, and address decay is not random. Senior buyers change employers more often than junior ones, so the oldest cohort loses exactly the people whose return would be worth the most. The registration rate you measure on a three edition cohort is a rate among the survivors of that decay.

The last one is timing. A lapsed attendee often returns because their own budget cycle changed, and no mailing moves that. Measure the effect, expect it to be modest, and set the target accordingly rather than treating a low number as a creative failure. All of this sits inside the same acquisition and attribution picture as your paid programmes and should be reported next to them.

The first step this week is a count. Group your lapsed records by the edition each person last attended and put the three totals on one line. If the oldest cohort is the biggest, which it often is, you know before writing any copy that most of your reactivation file is the hardest part of it.