Building an audience quality score from fields you already collect
An audience quality score ranks registrants who have already passed the qualification rule, using five fields most registration files already hold: seniority, purchasing role, company size, sector fit and prior attendance. Publish the weights and the lookup table, show the sub scores next to the total, and calibrate against badge scans.
A sales director opens the pre-show lead file, sees a column headed audience quality score with numbers between 12 and 96 in it, sorts by company name instead, and never opens the column again. When you ask why, the answer is that nobody could tell him what 68 meant, and he was not going to stake a conversation with an exhibitor on a number he could not explain.
He was right. A score that cannot be explained in one sentence at the moment somebody questions it has no authority, and a number with no authority does not change any decision, which means building it was a waste of a fortnight.
What is an audience quality score for?
The score ranks registrants who have already passed your qualification rule. That is a narrow job and it is worth being strict about it, because most quality scores fail by trying to do the qualification rule's work as well.
Qualification is a yes or no about whether somebody matches the commercial promise you made to exhibitors, and it is a rule an auditor could apply. Everybody who passes is qualified, and they are qualified equally, because that is what a promise means. What differs between them is how much attention a limited resource should spend on them: which registrants get the hosted buyer invitation, which appear at the top of a matchmaking shortlist, which the concierge team calls before the show.
Those are ranking decisions against a fixed capacity, and a ranking is what a score is good for.
Five components you already collect
Every component has to be a field you hold on most of the file. A score built on something you have for a fifth of registrants will rank the fifth and rank nobody else, and the missing four fifths will silently drop to the bottom.
Seniority. Derived from job title or a selected job level. The most complete field on most registration files and the one exhibitors ask about first. It needs a mapping from free text titles to a fixed ladder, and the score inherits every weakness of that mapping.
Purchasing role. The registrant's stated relationship to a buying decision. Weaker completion than seniority on most shows, and more directly about the thing you care about.
Company size. An employee band, either self-selected at registration or matched from a company record. Stands in for how much the account can spend.
Sector fit. How close the registrant's stated industry sits to the sectors your exhibitors sell into. Core, adjacent, or peripheral.
Prior attendance. Whether the person has attended previous editions, and how many. This is the only behavioural component on the list and it is the only one the registrant cannot influence by choosing a flattering option.
CEIR's 2016 Attendee Retention Insights series is worth reading before you build the prior attendance component. It ran to five reports on the largest US trade shows, built on online surveys completed by more than 9,000 attendees and 75 planners with in-depth phone interviews with eleven of those planners, and the sample and the method sit on the front of it. That is the property to copy. A score that publishes what went into it can be argued with. A score that does not can only be believed or ignored.
Weights you can publish
Pick weights, write them down, and put them on the same page as the score. Illustrative weights for a B2B show whose exhibitors sell capital equipment into a defined sector.
Seniority 0.30. Purchasing role 0.25. Sector fit 0.20. Company size 0.15. Prior attendance 0.10. They sum to 1.00, which keeps the total on the same nought to one hundred scale as the components and makes the arithmetic checkable in your head.
Each component is scored nought to one hundred from a published lookup. Seniority might run C-suite and owner at 100, director and vice president at 80, manager at 60, individual contributor at 40, and unstated at 30. Purchasing role runs final decision at 100, approves at 80, recommends at 60, specifies at 40, none stated at 20. Sector fit runs core at 100, adjacent at 55, peripheral at 25. Company size runs by employee band. Prior attendance runs attended two or more of the last four editions at 100, attended one at 60, registered and did not attend at 20, no history at 0.
Whether these are the right weights for your show is a commercial question and the sales team should set them. What matters structurally is that they are numbers on a page rather than coefficients inside a model nobody has opened.
Three registrants, scored by hand
Take three qualified registrants from one file and push them through.
The first is a chief operating officer at a 400 person distributor in the core sector, stated purchasing role of final decision, who registered for the previous edition and did not attend.
Seniority 100 at weight 0.30 gives 30. Purchasing role 100 at 0.25 gives 25. Sector fit 80 at 0.20 gives 16. Company size 60 at 0.15 gives 9. Prior attendance 20 at 0.10 gives 2. The total is 82.
The second is a category manager at a 120 person firm in an adjacent sector, purchasing role of recommends, who has attended two of the last four editions.
Seniority 60 gives 18. Purchasing role 60 gives 15. Sector fit 25 gives 5. Company size 40 gives 6. Prior attendance 100 gives 10. The total is 54.
The third is an engineer at a 30 person firm in an adjacent sector, no purchasing role stated, no prior history.
Seniority 40 gives 12. Purchasing role 20 gives 5. Sector fit 55 gives 11. Company size 20 gives 3. Prior attendance 0 gives 0. The total is 31.
Eighty-two, 54 and 31, and every one of those numbers can be reconstructed at a desk with a calculator by anyone who has the lookup table. That property is worth more than any accuracy the model might gain by being cleverer.
Why show the breakdown as well as the total?
The reason the sales director ignored the column is that he got the total and nothing else.
Give him the five sub-scores next to it. Now the 54 reads as a registrant who is a genuine repeat attendee with a real purchasing role and a sector fit of 25, and the conversation with the exhibitor becomes specific: this person comes every year and buys, and they are not in your part of the floor. That is actionable. A 54 is not.
The breakdown also makes the score self-correcting in a way a black box never is. When somebody with local knowledge says a registrant scored too low, they can point at the component that is wrong, and that is a bug report against your lookup table rather than an argument about the model. Over one edition you will collect dozens of them, and they are the best evidence you will get about whether the weights match how the commercial team actually thinks.
I would go further and say an opaque score is worse than no score on this particular problem. A team with no score uses their own judgement, which is uncalibrated and at least honest about it. A team with a score they cannot interrogate either ignores it, which wastes the build, or defers to it, which quietly hands a commercial decision to whoever last edited the weights.
Calibrating against something observable
Weights set by argument are a starting point. Once the show has run, check them against behaviour.
The cheapest check uses badge scans. Take the previous edition, score every registrant with the current weights, split them into score bands of ten points, and compute the mean number of exhibitor scans per attendee in each band. If the score is ranking anything real, that curve rises. If it is flat between 40 and 80, the middle of your scale is not distinguishing anybody and you have five components where two would do.
Run the same check per component. A component whose sub-score has no relationship to scans is not carrying information, and it should either lose weight or leave the score. On several shows the component that fails this test is company size, because a large employer count says nothing about whether that particular person walks the floor.
Re-derive the weights no more than once a year and record the change with a date, exactly as you would with a qualification rule. A score whose weights move quietly is not a series, and the first time someone compares this edition's mean score against last year's you will need to say whether the scale moved.
Where a weighted score misleads
Three components out of five are self-reported by somebody who wants a badge, so the score inherits everything wrong with a registration form. If your form tells people that senior decision makers get access to a hosted programme, your purchasing role distribution will tell you something about your incentives.
The score is also mute about intent, and mute about whether the person will walk in at all. A director who buys once every seven years scores the same as one buying this quarter, because nothing in the five components knows about a live project, and a 90 scoring registrant who never arrives is worth nothing to the exhibitor waiting for them. Some shows can add a stated interest field, and the moment they do it becomes the most gameable field on the form. Turnout likelihood is a separate model on overlapping fields, and the two scores answer different questions about the same row.
The last limit is about how the score gets used at the edges. UFI and Explori's Global Visitor Insights study, presented in December 2018 from more than 13,000 visitor responses across 135 countries, reported that the preferences of younger visitors and of those from developing markets were beginning to change the balance of what a show has to offer. A score weighted 0.30 on seniority ranks the junior half of that audience at the bottom every year, and if the ranking drives who gets invited to anything, it becomes a forecast that arranges its own confirmation. Reserve some capacity outside the ranking, and check who is in the bottom quartile before you decide it does not matter. That check belongs in the same attendee analytics review as the score itself.
Start this week by writing the lookup table. Five components, the sub-score for every value each one can take, and the five weights, on one page. Score fifty registrants from last edition by hand, show the page and the fifty scores to whoever runs exhibitor relationships, and let them tell you which ones are wrong. Their objections are the specification.
Questions people ask about audience quality score
- What should go into an audience quality score?
- Components you hold for most of the file, because a score built on a field present for a fifth of registrants ranks that fifth and silently drops everyone else to the bottom. Seniority, purchasing role, company size, sector fit and prior attendance work on most B2B shows. Prior attendance is the only component a registrant cannot flatter.
- How do you weight an audience quality score?
- The weights are a commercial question and the sales team should set them. What matters structurally is that they are numbers on a page summing to one, with each component scored from nought to one hundred against a published lookup. That keeps the total on a familiar scale and lets anyone reconstruct a score with a calculator.
- Why do sales teams ignore a quality score?
- Because they get a total and nothing else. A number nobody can explain at the moment it is questioned has no authority, and a number with no authority changes no decision. Showing the five sub scores next to the total turns an argument about the model into a specific correction against one lookup value.
Related reading
- Setting attendee qualification criteria your exhibitors would accept
- Tracking buyer seniority mix across five editions of registration data
- Predicting event no shows from the registration data you already hold