Product category mapping from free text exhibitor profiles

Six weeks before doors, the exhibitor profiles arrive in the state they always arrive in. Some are three words. Some are 400 words of company boilerplate about heritage and commitment. A handful are in German on an English form. About a fifth mention a product category by name, and the rest describe what they make without ever using the word your taxonomy uses for it.

Product category mapping is the job of turning that into taxonomy leaves, and it decides how much of your floor the matching engine can see. An exhibitor who never gets a category is invisible, whatever their stand cost.

The method that survives contact with this data is old, cheap and inspectable: term weighting, cosine similarity, an explicit threshold, and a review queue for everything the arithmetic is not sure about.

What arrives in the profile field

Take 1,200 exhibitor profiles from a mid-sized show. Sort them by length and read both ends.

At the short end, you have profiles like "Conveyor systems" and "Since 1974". The first maps cleanly. The second contains no information about product at all, and no method fixes that, so it belongs in a chase list for the operations team rather than in a model.

At the long end are profiles carrying six paragraphs, of which one sentence is about products. The rest is about sustainability commitments, the family who founded the business, and the awards shelf. Term weighting handles this better than it has any right to, because the product sentence contains the rare words and the boilerplate contains the common ones.

In the middle, which is most of the file, are two or three sentences that describe the product using the customer's vocabulary and never the taxonomy's. Filling machines for viscous liquids in the personal care sector needs to reach a leaf called liquid filling equipment, and that mapping is exactly the work.

Why is tf-idf still the right first tool?

Because the discriminating evidence in this text is rare technical words, and term weighting was designed for precisely that.

Salton and Buckley set out the weighting schemes in Information Processing and Management in 1988, and the logic has not changed. A term's weight rises with how often it appears in a document and falls with how many documents contain it. In a file of 1,200 exhibitor profiles, a term appearing in 24 of them has an inverse document frequency of the natural log of 1,200 divided by 24, which is the log of 50, about 3.91. A term appearing in 600 of them scores the log of 2, about 0.69. The rare term carries roughly 5.7 times the weight of the common one, and no human has to decide that.

That ratio is what makes the boilerplate harmless. Solutions, quality, leading and innovative appear in hundreds of profiles and are weighted down to nearly nothing. Vulcanising, palletiser and rheometer appear in a handful and dominate the vector.

Sebastiani's survey of machine learning for text categorisation in ACM Computing Surveys in 2002 sets out the wider family of methods, and the honest summary is that the harder methods win when you have labelled training data. On a show where nobody has ever labelled a profile, the first pass has to work without labels, which points at similarity against the taxonomy itself.

Building the document for each taxonomy leaf

The trick that makes the unsupervised version work is treating each category as a document.

For every leaf, concatenate the display label, the scope note, the non-preferred synonyms, and the profile text of any exhibitor already correctly classified there from a prior edition. A leaf called liquid filling equipment becomes a document containing filling, filler, volumetric, piston, viscous, bottling and the rest of its vocabulary.

Leaf documents built only from a two word label do not work. The whole method depends on the leaf having enough vocabulary to overlap with a profile that never uses the label, which is why the scope notes from the taxonomy build are worth writing properly, and why the tree design in I7 has to come first.

Working one profile through the arithmetic

Take a profile: "We design and build volumetric filling machines for viscous liquids, with servo driven piston fillers for personal care and food."

After stripping common words, the meaningful terms are volumetric, filling, machines, viscous, liquids, servo, piston, fillers, personal, care, food. Weight each by its inverse document frequency across the 1,200 profiles. Filling appears in 96 profiles, so it scores the log of 12.5, about 2.53. Volumetric appears in 18, scoring the log of 66.7, about 4.20. Food appears in 380, scoring the log of 3.16, about 1.15.

Now compute cosine similarity against each leaf document, which is the dot product of the two weighted vectors divided by the product of their lengths, giving a number between zero and one. Suppose liquid filling equipment returns 0.71, packaging machinery general returns 0.44, and personal care contract manufacturing returns 0.21.

Three things are readable from those numbers. The top candidate clears a 0.6 threshold, so it assigns automatically. The gap between first and second, 0.27, is wide, which is a second signal worth storing, because a profile scoring 0.62 and 0.61 against two leaves is a different situation from one scoring 0.71 and 0.44 even though both clear the bar. And the third candidate picked up the sector rather than the product, which is the most common error mode and the reason the sector terms are worth removing from leaf documents entirely.

Where should the auto assign threshold sit?

Wherever your measured error rate says, and the measurement is a morning's work.

Run the mapping at several thresholds and sample 100 auto assigned profiles at each. At 0.6, suppose 780 of 1,200 profiles assign automatically, which is 65 per cent, and 7 of the sampled 100 are wrong. At 0.5, 900 assign, which is 75 per cent, and 14 of 100 are wrong. The second option buys 120 profiles and doubles the error rate, and errors here are worse than gaps, because a wrongly classified exhibitor receives confidently wrong proposals all edition while an unclassified one merely receives none.

That leaves 420 profiles for review. At 90 seconds each, with the top three candidates pre-populated and a keyboard shortcut per candidate, that is 630 minutes, or a person and a half for one day. Sizing it in hours is what turns this from a research question into a scheduling one.

Two refinements pay for themselves. Auto-assign a second leaf when it also clears the threshold, because exhibitors genuinely sell more than one thing. And route anything where the top two candidates sit within 0.05 of each other to review regardless of how high they score, because a confident tie is the case where the automatic choice is arbitrary.

The review queue as a data asset

Every reviewed profile is a labelled example, and after one edition you have 420 of them.

That changes what is available next year. With labels you can train a supervised classifier, which is the family Sebastiani's survey covers, and you can measure it properly against a held out set instead of guessing at thresholds. You can also compute a per-leaf error rate, which tells you which parts of the taxonomy are ambiguous and which scope notes need rewriting.

Store the reviewer's decision, the candidates offered, and the position of the chosen one. If reviewers pick the second candidate 40 per cent of the time, the ranking is weak and the threshold conversation is premature. If they pick outside the top three regularly, the leaf documents are missing vocabulary and no threshold saves you.

Where the mapping stops

Some profiles carry no product information. A three word entry naming the company and nothing else cannot be mapped by any method, and the honest handling is a count on a slide: 68 of 1,200 profiles contain no product terms, here is the list, somebody needs to ring them.

Multi-line distributors break the method differently. A profile listing 30 product families will score moderately against 15 leaves and clear the threshold on none of them, so it falls into review, where the reviewer has to make a judgement about how many leaves is honest. Set a cap and apply it consistently.

Language is the third gap. A German profile scored against English leaf documents returns near zero everywhere, which at least fails visibly. Translate before scoring, or keep parallel leaf documents per language, and count how many profiles arrive in each language before deciding which.

The deeper limit is that mapping accuracy is bounded by the tree it maps into. If two leaves genuinely overlap, reviewers will disagree with each other, and their disagreement rate is a measurement of the taxonomy rather than of the reviewers. Ten per cent disagreement on a sample double-reviewed by two people is a taxonomy problem, and it should be fixed there before anyone tunes a threshold or argues about the right depth to match at in I9.

Start by taking 50 profiles at random and classifying them by hand against your current tree, timing yourself. That gives you a per-profile cost, a first read on how many are unclassifiable, and a small labelled set to check any automatic mapping against. Do it before buying anything, because the same 50 profiles will tell you whether your problem is the mapping method or the categories buyers are offered in I10, and those need different fixes inside the same matchmaking programme.