Skip to content

Weighting survey results so show feedback matches who actually attended

Attendee analyticsUpdated 2026-08-238 min read

In short

Weighting survey results means giving each respondent a factor equal to their group's share of attendance divided by its share of responses, then taking the weighted mean. Buyers at 62 per cent of attendance and 41 per cent of responses carry a weight of 1.51. The correction costs precision, so report the effective sample size beside the weighted figure.

Two thousand people answered the post show survey and the mean satisfaction came out at 7.79. Somewhere in the same report is a verified attendance table showing that buyers made up 62 per cent of the people who came through the door.

Buyers made up 41 per cent of the survey. Weighting survey results is the step between those two facts, and skipping it means the headline figure describes the mix of people who replied, which no stakeholder has ever asked about.

What does an unweighted mean actually describe?

It describes the respondent pool. That is a real population and it is the wrong one.

The mechanism is simple enough to see without any statistics. Exhibitor staff answer surveys at a higher rate than visitors, because the organiser is their supplier and the survey arrives with a relationship attached. Press and speakers answer at a higher rate still. Buyers, who are the people the exhibitors paid to meet, answer at the lowest rate of anyone. So the unweighted mean is dominated by the group with the least commercial relevance to the report.

Explori's published explanation of how it builds industry benchmarks makes the same correction at a different level. Its documentation states that the benchmarks "run off of the averages of the proportions of individual surveys that contribute to the benchmarks, rather than an average all respondents", so that a comparison reflects the average event rather than the average respondent. A large show with thousands of responses would otherwise dominate the benchmark for every small one. The principle inside your own show is identical, with badge types standing in for events.

Choosing the cells

Post-stratification needs cells where two things are true: you know the population share exactly, and the cell relates to what you are measuring.

Badge type qualifies at every show, because verified attendance is counted by badge type anyway and the categories are already defined for the attendee analytics tables in the same report. Region qualifies at most shows for the same reason. First visit against returning qualifies if your matching across editions is good enough to be quoted.

Company size usually does not qualify, and this is where teams overreach. If the registration record carries a size band for 64 per cent of attendees, weighting on it means the other 36 per cent either drop out or get an assumed band, and you have imported a coverage gap into a correction meant to remove one.

Keep the cell count low. Three or four cells on one variable, or a two by three grid on two, is enough for a satisfaction mean. Sixteen cells on a two thousand response survey produces cells of forty people and weights nobody should trust.

If you want to correct on two variables and you do not know the joint distribution, the standard route is raking, which adjusts the weights iteratively until they match each margin in turn. It works, and it hides a decision. Raking assumes the two variables are independent inside the population, which for badge type and region is often false at a show with a hosted buyer programme that recruits internationally. Where you do hold the joint counts, and verified attendance usually gives you them, weight on the cross-tabulation directly and skip the assumption.

The arithmetic on one show

Three cells, taken from the verified attendance table and the response file.

Buyers are 62 per cent of attendance and 41 per cent of responses, so the weight is 62 divided by 41, which is 1.51. Exhibitor staff are 26 per cent of attendance and 38 per cent of responses, giving 26 over 38, or 0.68. Everyone else, meaning press, speakers and students together, is 12 per cent of attendance and 21 per cent of responses, giving 0.57.

Mean satisfaction inside each cell is 7.2 for buyers, 8.3 for exhibitor staff and 8.0 for the rest.

The unweighted mean is 0.41 times 7.2, plus 0.38 times 8.3, plus 0.21 times 8.0. That is 2.952 plus 3.154 plus 1.68, or 7.79.

The weighted mean substitutes the population shares. It is 0.62 times 7.2, plus 0.26 times 8.3, plus 0.12 times 8.0. That is 4.464 plus 2.158 plus 0.96, or 7.58.

Two tenths of a point, which sounds trivial until you notice it is larger than the year-on-year movement most show reports lead with. And the direction is the one that matters commercially: the buyers, the people the whole event exists to assemble, rate it lowest and were the group most under-represented in the raw figure.

Little published the modelling case for this in 1993, in the Journal of the American Statistical Association, developing a Bayesian treatment in which the post-stratified mean emerges as the posterior mean of a normal post-stratification model. The practical value of that framing is that it makes the assumption explicit. The weighted estimate is right if respondents inside a cell resemble non respondents inside the same cell. Where that fails, weighting moves the number without fixing it.

What does weighting cost you in precision?

Unequal weights throw away information, and the amount is calculable.

The effective sample size is the square of the sum of the weights divided by the sum of the squared weights. With 2,000 responses split 820 buyers, 760 exhibitor staff and 420 others, the weights sum to 820 times 1.51 plus 760 times 0.68 plus 420 times 0.57, which comes to 2,000. The squared weights sum to about 2,368.

Effective sample size is 2,000 squared over 2,368, which is 1,689. The design effect is 2,000 over 1,689, about 1.18.

So a weighted survey with two thousand responses carries the precision of an unweighted one with roughly 1,689. You have given up 311 responses' worth of certainty to buy an estimate that describes the right population. That is a good trade and it should be stated, because an interval computed on the raw count will be too narrow. The same correction applies to a weighted net promoter score, where the interval was already wider than most organisers expect, and the arithmetic for that sits with putting an NPS confidence interval on your show score in D29.

Why weighting beats chasing a higher response rate

The alternative strategy is to keep the field open longer and send more reminders until the response rate looks respectable. It is expensive and the evidence for it is weak.

Hendra and Hill examined this in Evaluation Review in 2019, using detailed disposition data from 12,000 individuals fielded for 16 identical surveys in the Employment Retention and Advancement evaluation to simulate what nonresponse bias would have looked like at lower response rates. They found scant relationship between response rates and nonresponse bias, and noted that the pursuit of high response rates lengthens the fielding period, which introduces measurement problems of its own.

For a show survey the lengthened field period is a specific problem rather than an abstract one. Responses arriving in week four are answering about an event four weeks gone, so a longer field buys you a bigger sample with worse recall, and the composition of the late arrivals is different again.

Measure the composition gap, correct it, and close the survey. Sizing that gap before you decide whether it needs correcting is post show survey response bias in D23, and the two exercises use the same join.

Two rules that keep weights honest

Cap the weights. Any single respondent carrying a factor above about three is doing too much work, and the usual cause is a cell that is tiny in the response file. Collapse the cell into a neighbour, or trim the weight and say so.

Publish the weights. One small table in the methodology note with population share, respondent share and weight per cell, and both the weighted and unweighted means. A reader who can see that buyers were 41 per cent of the sample can judge the correction. A reader given 7.58 with no working has to take it on trust, and someone will eventually recompute it from the raw export and get 7.79.

Where this stops

Weighting corrects for the variables you weighted on. Nothing else.

If the buyers who reply are systematically the ones who had a good week, then multiplying their responses by 1.51 gives you more of a biased estimate rather than less of one. Post-stratification assumes respondents represent non respondents within a cell, and no amount of arithmetic tests that assumption from inside the survey.

There is a second limit that bites on smaller shows. Cells need enough responses to produce a stable mean before the weight is applied, and below about thirty responses a cell mean moves by a tenth of a point when two people change their minds. Multiplying an unstable mean by a large weight makes the whole estimate jumpy across editions, which will read as a trend.

One measure resists weighting more than most, because it is a difference of two proportions rather than a mean, and reweighting the promoter and detractor shares separately can move the headline in a way that surprises people. That behaviour belongs with net promoter score for events in D27.

Pull your last survey's respondent counts by badge type and put them next to the verified attendance counts by badge type. Compute one weight per row, apply them to the satisfaction mean, and see how far it moves. If the answer is under a tenth of a point, weighting is not your problem this year and you can say so in one line of the methodology note.

Questions people ask about weighting survey results

How do I calculate a post-stratification weight for survey responses?
Divide each group's share of the attendee population by its share of the respondents. A badge type making up 62 per cent of verified attendance and 41 per cent of responses gets a weight of 62 divided by 41, which is 1.51. Every respondent in that cell carries that factor, and the weighted mean uses population shares in place of respondent shares.
Which variables should I weight a show survey on?
Ones you know exactly for the whole audience and that relate to the answer you are reporting. Badge type and region qualify at almost every show, because verified attendance is counted by both. Company size usually does not, since the registration record carries it for only part of the file, and weighting on a partly known variable imports its coverage gaps.
Does weighting make survey results less precise?
Yes, and the loss is quantifiable. Unequal weights reduce the effective sample size to the square of the sum of the weights divided by the sum of their squares. Two thousand responses weighted at 1.51, 0.68 and 0.57 behave like about 1,689, so the interval around the weighted mean is wider than the raw count suggests.

Related reading

All post-show reporting articles