Showing sub scores to exhibitors without handing them a cheat sheet

An exhibitor sales manager gets access to the matchmaking dashboard in early September and finds a column showing authority fit for every buyer proposed to them. By the second week of September her team has worked out that anything with director in the title lands in the top band. By the third week, her own staff registrations have all become directors, which does nothing to the buyer side and tells you exactly what would happen if the rule ran the other way.

Showing sub scores to exhibitors is a real decision with a real cost on both sides of it. Hide everything and the exhibitor experiences the meeting programme as a lottery run by people who will not explain themselves. Show everything and you have published the marking scheme to the party with the strongest incentive to score well on it.

The position worth holding is in between, and it has three parts: coarse bands, a fixed refresh cadence, and one term that never leaves the building.

What happens the day after you publish a sub-score?

People optimise the number. This is neither dishonest nor surprising, and planning for it is a design job.

Strathern, writing about audit in the British university system in European Review in 1997, gave the formulation that a measure ceases to be a good measure once it becomes a target. Match sub-scores become targets the moment an exhibitor can see them next to a leaderboard of the buyers they were proposed.

Tintarev and Masthoff, in their chapter in the Recommender Systems Handbook in 2015, separate the aims an explanation can serve, among them transparency, scrutability, trust and persuasiveness. That separation is the line to hold when deciding what to expose. Transparency means the exhibitor can see why a proposal was made. Scrutability means they can correct the input that was wrong. Persuasiveness means they accept more proposals, and building a view aimed at persuasiveness is how you end up with a dashboard nobody trusts by the third edition.

Expose what supports the first two. Withhold what only feeds the third.

The category ticking problem, with arithmetic

The main gaming route is not job titles at all. It is category selection, and it is available to every exhibitor with a profile form.

Suppose product fit is computed as the share of the buyer's declared categories the exhibitor covers. A buyer declares three categories. Exhibitor A lists six categories and overlaps on two, scoring two of three, which is 0.67. Exhibitor B lists nineteen categories and overlaps on all three, scoring 1.0. B wins, and B wins by ticking boxes.

Now score the same pair with the second term included: how much of the exhibitor's own list is relevant to this buyer. A covers two of six, which is 0.33. B covers three of nineteen, which is 0.16. Take the harmonic mean of the two terms for each exhibitor. For A that is 2 times 0.67 times 0.33, divided by 1.00, which is 0.442. For B it is 2 times 1.0 times 0.16, divided by 1.16, which is 0.276.

The ordering flips, and it flips for a reason an exhibitor can be told out loud. Covering everything makes you a worse match for everyone, because a buyer meeting you has to sit through eighteen categories they did not ask about. Publish that rule with the bands and the incentive points where you want it: exhibitors should tick the categories they actually sell, and prune the ones they added three years ago.

The effect on your own operation is worth sizing. On a show with 1,050 exhibitors where the median profile lists six categories, a publication that triggers half the floor to inflate to nineteen takes the average candidate pool per buyer from around 40 exhibitors to over 90, and every one of those extra candidates lands in a concierge review queue.

Bands instead of numbers

Publish three bands per exposed sub-score. High, medium, low. No decimals.

A decimal invites the wrong conversation. An exhibitor at 0.61 who sees a competitor at 0.68 will ask what the seven hundredths are made of, and the honest answer involves a taxonomy distance calculation that will not survive the retelling. Bands support the conversation you want, which is what would move this from medium to high.

Bands also absorb the noise. The sub-scores carry real uncertainty, especially intent and behaviour, and a number displayed to two decimal places asserts a precision the underlying data does not have. Three bands over a zero to one range means the boundaries sit at roughly 0.33 and 0.67, and an exhibitor moving from 0.61 to 0.68 crosses one, which is the level of change worth reporting.

Show the band with the reason attached, using the same controlled list your proposals already carry, which is the code list in I5. A band with no code is a grade with no feedback.

A fixed refresh cadence

Recompute the exhibitor view on a schedule and say what the schedule is. Every Monday at 06:00 works.

A live view is a probing interface. Change one category, reload, read the delta, repeat. Twenty minutes of that recovers most of your weighting scheme, and the exhibitor who does it gains an advantage over the one who does not, which is a fairness problem you created by accident.

The cadence has an operational benefit too. Exhibitor profile edits arriving continuously mean the candidate pool changes under your concierge team while they work, and a proposal built on Tuesday against a profile edited on Wednesday cannot be defended afterwards. Freeze, compute, publish, and keep every version.

Which term to hold back entirely

Behaviour fit. All of it, including whatever feeds it.

Two reasons. The first is that no exhibitor action can change a buyer's prior attendance or email engagement, so publishing it serves neither transparency in any useful sense nor scrutability at all. It generates complaints about buyers the exhibitor cannot have and cannot influence.

The second is that behaviour fit is built from things the buyer did without being told they were being scored on them. Pages viewed, sessions saved, how quickly they opened an email. Surfacing that to a commercial party is a privacy decision dressed as a reporting decision, and it is the kind that reads badly when a buyer asks how the exhibitor knew.

Intent fit is the borderline case. Declared interest, meaning categories the buyer selected at registration, is fine to expose because the buyer typed it knowing it would route them. Observed interest, meaning what they clicked, belongs with behaviour fit on the hidden side of the line.

What should exhibitors get instead?

A profile completeness score and a list of the specific things to fix, which is more useful than any sub-score and cannot be gamed in a harmful direction.

Your profile lists six categories and two of them have no buyers declaring interest this edition. Eleven buyers in your top band listed a category you do not cover. Your sector selection covers three industries and your last edition's meetings were 80 per cent in one of them. Each of those is actionable, each improves the data your matching runs on, and none of them tells the exhibitor how the weights work.

Give them a correction route as well, with a service level attached. An exhibitor who thinks a category is wrong on their profile should be able to change it themselves and see the effect at the next refresh. An exhibitor who thinks a buyer was mis-scored should be able to say so in one click, and that click should land in the same queue your concierge team already works, tagged so you can count how many corrections arrive and how many turn out to be right. A programme where 60 of 1,050 exhibitors raise a correction and 40 of those are upheld has a data problem worth naming. A programme where nobody ever raises one has an interface nobody believes.

Pair that with the reasons on individual proposals, generated the way the buyer-facing explanation is generated in I4, and the exhibitor has enough to work with. The thing they mostly want is not the score. It is a defensible answer to why they got 14 proposals when the stand opposite got 31, and that answer is usually about profile quality and category coverage rather than about the model.

Where transparency stops

Every rule above leaks eventually. Exhibitor sales teams talk to each other, some of them exhibit at forty shows a year, and the ones running the biggest programmes will work out the shape of your scoring within two editions whatever you publish. Designing as though the rule is secret is a mistake. Designing so that the best way to score well is to have an accurate profile is the only durable version.

The second limit is that bands hide movement. An exhibitor sitting at 0.34 all edition and one climbing from 0.34 to 0.66 both read as medium, and the second one has done exactly what you asked and sees no reward for it. If you band, add a direction marker against the last refresh, which restores the feedback without restoring the decimal.

Some of this also depends on what the sub-scores are made of in the first place, and a score with only two informative terms cannot support three bands on five of them, which is I1's territory.

Start by taking your current exhibitor view, if you have one, and listing every field on it that an exhibitor could change in an afternoon. For each, ask what the profile would look like if 200 exhibitors changed it in the direction that raises their score, then check whether your matchmaking engine would rank them higher afterwards. Any field where the answer is yes and the change makes the profile less accurate is a field to band, delay or remove before the next edition opens.