Treating meeting requests as intent and what that does to scores

An exhibitor on a sourcing show floor told me their recommended buyer list was useless, and the reason was specific. The top four names on it were the four buyers who had already sent them meeting requests. The system had ranked, with confidence, the people the exhibitor was already talking to.

Meeting requests as intent is the most defensible signal available and the easiest one to double count. A buyer who fills in a request form and sends it has spent effort naming a company, which no page view can match. Feed that straight into an intent term without thinking and the score stops ranking and starts echoing.

What does a sent request actually tell you?

It tells you one side of the pair wants the meeting. That is a smaller fact than it looks, because a meeting needs two people to turn up.

Palomares and colleagues reviewed reciprocal recommender systems in Information Fusion in 2021 and set out the property that separates this problem from ordinary recommendation: a recommendation succeeds only when both parties are satisfied, so preferences have to be computed on both sides and then aggregated into a single reciprocal score. Their review covers the aggregation choices and the evaluation problems that follow from having two sets of preferences and one outcome.

That framing has a direct consequence for requests. A request from a buyer raises the buyer's side of the pair and says nothing about the exhibitor's. If the exhibitor has a twelve slot day and 90 requests, those requests amount to a queue, and the queue says nothing about which pairs fit. The pairs worth proposing are the ones where the request is met by independent evidence on the other side, and a score that adds the request to a single pooled total loses the distinction.

Why does the request term need a cap?

Because without one, a single click moves the total score further than anything else a buyer can do, and the effect compounds across every exhibitor that buyer touches.

Split the intent sub-score into two halves that each run from 0 to 0.5. The first half comes from category evidence: bookmarks, saves and views, mapped onto the category the exhibitor sits in. The second half comes from requests, and it is capped at 0.5 no matter how many requests are sent to that exhibitor.

Take a buyer with reasonable category evidence, a category component of 0.6 before scaling, so 0.5 times 0.6, which is 0.30 for the intent sub-score. Their other sub-scores are product fit 0.80, industry fit 0.70, authority fit 0.60 and behaviour fit 0.40. Under the standard weights the total is 0.30 times 0.80, plus 0.20 times 0.70, plus 0.22 times 0.30, plus 0.15 times 0.60, plus 0.13 times 0.40. That is 0.240 plus 0.140 plus 0.066 plus 0.090 plus 0.052, which is 0.588.

Now the buyer sends a request to that exhibitor. The intent sub-score becomes 0.30 plus 0.50, which is 0.80, and the total becomes 0.240 plus 0.140 plus 0.176 plus 0.090 plus 0.052, which is 0.698. The pair moved 0.11 on a scale where the gap between the fifth and the fifteenth candidate is usually smaller than that.

Uncapped, with requests driving the intent term to 1.0, the same action would have moved the total to 0.742, which is 0.154 of movement from one form submission. The cap is not a nicety. It is the difference between a score that uses the request and a score that is the request wearing four other terms as decoration.

Two more rules travel with the cap. Count repeated requests to the same exhibitor once, because the second and third are impatience. And do not let requests leak into the behaviour term as well, which happens by accident when behaviour is defined as platform activity, and which double counts the same click at 0.22 and 0.13.

The loop, and how to see it in your own numbers

The score chooses what gets proposed. Buyers act on what they see. Their actions become the data that trains the next version of the score. Chaney, Stewart and Engelhardt demonstrated at RecSys in 2018 that data confounded in this way homogenises user behaviour without increasing utility, which is a formal statement of the exhibitor complaint above.

You can measure your own exposure to it with one query. Take last edition's confirmed meetings and split them by origin: meetings that began as a buyer request, meetings that began as an exhibitor request, and meetings that began as a system proposal neither party had asked for.

Suppose 1,240 confirmed meetings split as 610 buyer requests, 285 exhibitor requests and 345 system proposals. Any evaluation run across all 1,240 is dominated by pairs where somebody had already chosen the counterparty, and a score with a request term will look excellent on it, because 895 of the 1,240 contain a term that is a copy of the label. Evaluated on the 345 system proposals alone, the same score might hold up or might collapse, and that number is the one worth reporting.

The share also tells you what your matchmaking is for. At 345 out of 1,240, which is 27.8 per cent, the system is largely an admin layer over meetings that would have happened anyway, and the honest description of it in a board pack is scheduling. Getting that share up is a better objective than another point of precision on a contaminated test set.

Keeping the label clean

The fix is boring and it is the whole job: keep an origin column on every meeting and never train or evaluate without splitting on it.

Hold out the system-proposed meetings as the evaluation set. Train on whatever you like, including request-originated pairs, since they carry real information about which combinations work. Report the headline acceptance rate on the held out set, and report it next to the origin split so nobody reads one number without the other.

There is a second contamination worth blocking at the same time. When a buyer's request is accepted, that acceptance often raises the buyer's intent score globally in implementations that treat request volume as engagement, which then lifts every other proposal for that buyer. Keep the request term per pair, never per person, and the problem disappears.

Once the origin split exists, what the score means as a probability becomes answerable in I3, because a calibration curve fitted on request-originated pairs is fitted on the easiest cases in the file and will overstate acceptance everywhere else.

One case breaks the rule above, and it is worth naming because it comes up at every launch edition. When there is no browsing history and no prior edition to draw on, requests are the only intent evidence in the building, and holding them out leaves the intent term at its neutral default for everybody. In that situation the cap still applies and the evaluation still needs the origin split, but the honest description of the first edition's numbers is that the system ranked on declared attributes and scheduled what people asked for. Say that in the post-show review, because the following year somebody will compare acceptance rates across the two editions and conclude the model improved.

What to do with the exhibitor side of a request

Exhibitor requests behave differently from buyer requests and most systems treat them identically.

An exhibitor with a sales team and a target list will send requests in bulk, because the cost of sending is near zero and the upside of one acceptance is a booth's worth of pipeline. A buyer sends few. Pooling the two means the exhibitor's volume dominates the intent term across the whole file, and a buyer who receives 40 requests looks like a high intent buyer while having done nothing at all.

Score the two sides separately, cap them separately, and set the exhibitor side lower. If the reciprocal aggregation is a product or a minimum of the two sides, an exhibitor blasting requests gains nothing when the buyer side is empty, which is the behaviour you want. Bulk requesting is a rational exhibitor strategy under a pooled score and an unrewarded one under a reciprocal score, and choosing the second is a policy decision your commercial team should make knowingly.

Where this stops

A request is evidence about the moment it was sent, and buyers change their minds. Some share of requested meetings ends with somebody not turning up, that share is measurable on your own programme, and the request term cannot see it coming because the person who sent the request was sincere at the time.

The deeper limit is that all of this optimises acceptance, and acceptance is a proxy for a meeting that produced business. Requests are the signal most likely to look good on the proxy and least likely to add information about the outcome, because a buyer who already knew the exhibitor would have found them without you. A score dominated by requests will report improving numbers while the discovery value of the programme falls, and no amount of internal validation catches that, since every measurement is taken on the same contaminated file.

Run the origin split this week. One query over last edition's confirmed meetings, three counts, and the share of meetings the system actually originated. If that share is under a third, the next fortnight is better spent raising it, using the signals ranked by lift in I11 and the first party sources behind them in I12, than on tuning anything inside your matchmaking score.