Taxonomy granularity for matching and the cost of going too deep

The taxonomy rebuild lands, and it is a good one. Three levels, 400 leaves, scope notes written, exhibitors mapped. Two weeks later the concierge team reports that a third of hosted buyers have fewer than five proposals worth sending, on a programme where every buyer holds a twelve meeting quota.

Nothing is broken. Taxonomy granularity for matching is a trade, and the rebuild moved it in one direction without anybody checking what happened at the other end. Deeper categories describe the floor more precisely and leave fewer exhibitors in each bucket, and at some depth the bucket holds fewer exhibitors than the buyer needs meetings.

What granularity actually changes

Two things move in opposite directions as you go deeper, and both are measurable on your own file.

Precision rises, in the sense the information retrieval literature uses. Van Rijsbergen set out the trade in his 1979 book on information retrieval, and the shape is the familiar one: tightening the criterion raises the share of results that are relevant and lowers the share of relevant results you find. Matching a buyer against a fourth level leaf returns exhibitors selling nearly the same thing. Matching against a second level node returns exhibitors selling something in the same general area.

The candidate pool shrinks, roughly by the branching factor at each step. A tree with 12 top nodes, 80 mid nodes and 400 leaves has a branching factor of about 6.7 from top to mid and 5 from mid to leaf, so every level down cuts the expected pool by a factor of five or six.

The buyer's quota does not move at all. That is the constraint the depth choice has to respect, and it is the one nobody writes down.

Scoring one buyer at three depths

Take a show with 1,050 exhibitors and a buyer holding a twelve meeting quota. Score the same buyer three times, once against each level.

At level two, the buyer's category maps to a node containing 148 exhibitors, of which 96 clear the score threshold. At level three, the node contains 41 exhibitors and 34 clear. At level four, the leaf contains 11 exhibitors and 9 clear.

Now bring in acceptance. Proposals from a level four match get accepted more often, because they are better matches, so suppose acceptance runs at 22 per cent for level two matches, 38 per cent for level three and 44 per cent for level four. The number of proposals needed to fill the quota is the quota divided by the acceptance rate. At level four that is twelve divided by 0.44, which is 27.3, so 28 proposals. At level three, twelve divided by 0.38, which is 31.6, so 32. At level two, twelve divided by 0.22, which is 54.5, so 55.

Compare each requirement against the pool. Level four needs 28 candidates and has 9, so it cannot fill the quota even if every candidate accepts, and the maximum it can deliver is 9 times 0.44, which is 4 meetings. Level three needs 32 and has 34, which fits with almost nothing to spare. Level two needs 55 and has 96, which fits comfortably and delivers meetings a third of buyers will find generic.

Level three wins on this buyer, and the reason it wins has nothing to do with which level is more accurate.

How deep can you go before the pool collapses?

The rule is one line. The pool at the chosen level must be at least the quota divided by the acceptance rate at that level, with a margin for the exhibitors who are full.

The margin matters more than it looks. Exhibitor slots are finite, and the best exhibitors in a leaf fill first, so a pool of 34 against a requirement of 32 is not really a fit. By the time the concierge team works through the buyer list, the top eight exhibitors in that leaf are booked, and the buyer arriving late gets 26 candidates against a requirement of 32. Carry a margin of at least 50 per cent, which on this buyer means the level three pool of 34 should really be compared against 48 rather than 32, and it fails.

That is worth sitting with, because it inverts the intuition. On a floor of 1,050 exhibitors with twelve meeting quotas, level three is marginal and level two is the safe default for most buyers, and the deep tree earns its keep as a scoring input rather than as a filter.

The error propagation cost of depth

There is a second cost to depth that has nothing to do with pool size.

Silla and Freitas surveyed hierarchical classification across application domains in Data Mining and Knowledge Discovery in 2010, and one of the recurring findings is that top-down classification propagates errors downward: a record sent into the wrong branch at the top cannot be recovered at any level below it. Your exhibitor mapping has exactly this structure, whether or not anybody designed it that way.

Put numbers on it. If mapping to the correct top node is right 95 per cent of the time, the correct mid node 88 per cent, and the correct leaf 76 per cent, then matching at level four is running on data that is wrong for roughly one exhibitor in four. Matching at level two runs on data that is wrong for one in twenty. Deeper matching is more precise about categories that are less likely to be correct, and those two effects work against each other in a way that a precision figure computed on clean test data will never show.

The per-level accuracy figures come out of the review queue described in the mapping work in I8, which is one more reason to keep that queue's decisions rather than discarding them once the profiles are classified.

A fallback ladder instead of a fixed depth

Do not choose one depth for the whole programme. Choose a rule.

Start at the deepest level. Compute the pool of exhibitors above threshold. If the pool is smaller than the required proposals plus the margin, climb to the parent and recompute. Repeat until the pool is large enough or you reach the top, and record the level each proposal was generated at.

That last part is what makes the ladder pay off. After the show, group proposals by the level they came from and compute acceptance for each. If level three proposals accept at 38 per cent and level two at 22, you have measured your own version of the numbers above instead of assuming them, and next edition's ladder uses your figures. If the two rates come out the same, your tree's third level is not carrying information and the honest response is to stop maintaining it.

The ladder also fixes the uneven parts of the floor. A show has dense areas where a leaf holds 40 exhibitors and thin areas where a leaf holds three, and no single depth is right for both. A per-buyer ladder handles that without anybody hand-tuning a category list.

Why not just go deep and relax the threshold?

Because the threshold and the depth control different things, and swapping one for the other has a cost that shows up in the wrong place.

Relaxing the score threshold at level four adds exhibitors who are in the right leaf and weak on everything else, meaning low intent, low authority, or no behavioural history. Climbing to level three adds exhibitors who are strong on everything except that they sit in a neighbouring category. Those two groups do not perform the same way, and the second is usually the better meeting.

There is an operational reason too. The threshold is a number in a config file that somebody will move under pressure in week 40, and once it has moved, every acceptance figure computed before the change is incomparable. The ladder is a rule that adapts per buyer without anybody touching the score. Keeping the threshold fixed and the depth adaptive means the sub-scores keep meaning what they meant, which is the property the score design in I1 depends on.

Where depth stops

Everything above assumes the buyer's declared categories are correct and complete. A buyer who ticked one category at registration has one leaf, and no ladder rescues a buyer who told you almost nothing.

The second limit is that acceptance rate by level is confounded with what your engine has been doing. If your current system has always matched at level two, you have no level four proposals to measure, and the 44 per cent above would be an assumption wearing a data costume. Run a genuine split for one edition, sending a random tenth of buyers a level four first ladder, and compare. Without that, the numbers you use to choose a depth came from the depth you already chose.

The last thing is that granularity is a property of a tree that varies across it. A tree can be four levels deep in machinery and two levels deep in services because that is how the floor is shaped, and a global statement about the right depth will be wrong in both areas. Measure pool sizes by branch, and expect the tree itself from I7 to need rebalancing in the places where the answer keeps coming out at the top level.

Start by taking one buyer segment, twenty buyers with full profiles, and computing their candidate pool at each level of your current tree with your current threshold. Put the three counts next to the quota and the acceptance rate you observed last edition. If the deepest level leaves fewer candidates than the quota divided by acceptance, your matchmaking engine has been failing on arithmetic rather than on relevance, and the ladder is a week of work rather than a rebuild.