Job title normalisation when every attendee types their own title

The job function chart in the post-show report had a bar labelled Other that was taller than every other bar put together. The audience director asked what was in it. The honest answer was 8,940 distinct strings that people had typed into an open text box, and no one had ever tried to sort them.

Job title normalisation is the work of turning those strings into something a group by can survive. It is unglamorous, it is finite, and most teams overestimate it because they picture the hard part first.

The hard part is smaller than it looks, because two cheap mechanical passes remove most of the variety before anybody has to make a judgement about what a title means.

The three passes, in order

Run them in this sequence and each one shrinks the input to the next.

Pass one: case, punctuation and whitespace. Lowercase the string. Remove full stops, commas, slashes and quotation marks. Collapse runs of whitespace to a single space and trim the ends. Replace the ampersand with the word and.

On a file of 31,200 registrations carrying 8,940 distinct title strings, this pass takes the distinct count to 7,610. That is 1,330 strings gone, or 14.9 per cent, and not one of them required a decision. Director of Sales, director of sales, and Director Of Sales. were three rows and are now one.

Pass two: the synonym map. Expand the abbreviations and standardise the variants. vp and v p become vice president. sr becomes senior. mgr and mngr become manager. dir becomes director. asst becomes assistant. md becomes managing director, with a note that it means something else entirely at a medical show. Handle the ordering variants too, so sales director and director of sales converge.

This pass takes 7,610 distinct strings down to 5,180. That is 2,430 more gone, or 31.9 per cent of what pass one left.

Together the two mechanical passes cut the distinct count from 8,940 to 5,180, a reduction of 42.1 per cent, before anybody has thought about what a title means.

Pass three: the taxonomy join. Map what remains onto a target list. This is the pass that needs judgement, and it is the one to spend money on, because the first two have already removed the noise it would otherwise be paying for.

Doing them out of order is the common mistake. A team that starts by building a taxonomy and mapping strings onto it will map Director of Sales and director of sales separately, and will feel like the work is endless because it is doing the same decision three or four times.

What should the titles land on?

Two targets, and you want both.

The public one is the O*NET-SOC taxonomy, maintained by the National Center for O*NET Development for the US Department of Labor. Gregory, Lewis, Frugoli and Nallin documented the 2019 revision in August 2019: the O*NET-SOC 2019 taxonomy holds 1,016 occupational titles, of which 923 are data-level occupations with collected data behind them, and the whole thing is aligned to the 2018 Standard Occupational Classification. It is free, versioned, and detailed enough that mapping onto it is a defensible act rather than an internal opinion.

The internal one is a short list of functions your exhibitors recognise. Twenty to thirty entries, in the vocabulary of your market: specifier, buyer, operator, installer, whatever the sector calls its people. This is the list the post-show report groups by, because a chart with 1,016 categories is no better than a chart with 8,940.

Hold both and map between them. Titles go to an O*NET-SOC code, and codes roll up to your internal function. The reason for the two-step is durability. When somebody proposes changing the internal function list next year, the underlying codes do not move, so you can restate three years of history under the new grouping instead of losing the series.

Skipping the public taxonomy and mapping straight to your twenty functions works and it costs you exactly that: the day you change the twenty, every prior year becomes unrestateable.

Where the effort actually goes

Into a few hundred strings, and then into a long tail that is not worth the same treatment.

Of the 5,180 distinct strings surviving the two mechanical passes, the 300 most frequent cover 24,100 of the 31,200 registrations, which is 77.2 per cent of the file. Mapping 300 strings by hand, at a comfortable 60 an hour once somebody has the taxonomy open, is five hours of work.

The remaining 4,880 strings cover 7,100 rows, or 22.8 per cent. Mapping all of them at the same rate is roughly 81 hours, which is two weeks of one person's time to move coverage from 77.2 per cent to something approaching complete.

That arithmetic is the whole planning decision. Five hours buys three quarters of the file. Two weeks buys the last quarter. Almost nobody should buy the second one by hand, and knowing the shape in advance stops a normalisation project being scoped as though every string costs the same.

The sensible middle is to hand-map the head, run a classifier over the tail, and route low-confidence classifier output to a review queue that a steward works through in whatever time is genuinely available. That leaves an unmapped remainder, and the unmapped share belongs on the chart. A job function mix that reports 86.2 per cent coverage and shows the other 13.8 per cent as unmapped is more useful than one that quietly distributes the tail into the categories it half-matches.

Should a classifier do the mapping?

For the tail, yes, with the accuracy stated honestly on the slide.

The reference point is occupational epidemiology, which has been coding free text job descriptions to standard codes for decades and publishes what it achieves. Russ and colleagues built SOCcer, an algorithm that scores a free text job description against every code in the classification, and reported its evaluation in Occupational and Environmental Medicine in 2016. Agreement between the expert-assigned code and the algorithm's highest scoring code was 44.5 per cent at the six-digit level and 76.3 per cent at the two-digit level.

Read those two numbers together, because the pair is the finding. The same algorithm on the same data is right about three quarters of the time at coarse granularity and less than half the time at fine granularity. When the team evaluated an updated version in Annals of Work Exposures and Health in 2023, the pattern held: 73 per cent at two digits against 50 per cent at six.

That maps directly onto the two-target design above. Classify to the coarse level, which is where the accuracy is, and let the coarse level be what the report groups by. Anyone proposing to auto-assign titles to a fine-grained code and publish the result should be shown those figures first.

It also sets an expectation for evaluation. If you build a classifier for your own titles, hand-code a sample of several hundred and report agreement at the granularity you intend to publish. A classifier with no measured agreement is a mapping nobody checked, and it will be believed precisely because it is automatic.

Rules that keep a synonym map from rotting

Four things, learned the hard way.

Version the map and date every entry, because the map is the reason last year's numbers can be reproduced.

Never edit the source column. Write the normalised value to a new field and keep the raw string forever. Every mapping decision you regret is recoverable only if the input survives.

Treat additions as append-only within a reporting year. A synonym added mid-year changes historical counts silently, and somebody will notice the job function mix moved without any change in the audience.

Keep the map out of the reporting query. It belongs in the governed record layer where one definition serves every downstream use, since a synonym map maintained inside three separate dashboards will diverge within two editions.

Where this stops

Normalisation collapses variety and it cannot invent information. A title of Manager, after all three passes, is still a manager of something unknown, and on most registration files it is one of the twenty most frequent strings. No taxonomy join fixes that, and the answer is a form change rather than a data pipeline: a separate function field with a controlled list next to the free text title.

The method also does badly with titles that are correct and unhelpful. Founder at a two-person consultancy and Founder at a firm of four hundred are the same string with different meanings, and nothing in the string separates them. Company size, where you have it, does more work here than any amount of title processing.

And normalisation deliberately throws away the level information. Sales Director and Sales Executive both collapse toward a sales function, which is right for the function chart and wrong for the audience quality view, because the difference between those two people is exactly what an exhibitor is asking about. Seniority is a second axis extracted from the same string, and pulling a level out of the title text is a separate pass with its own precedence rules.

Take one closed edition, run the two mechanical passes in a single query, and count distinct strings before and after. Then count how many rows the top 300 survivors cover. Those three numbers give you a scoped project instead of an open-ended one, and they take about twenty minutes once you have profiled the column. The same shape of problem on exhibitor product descriptions is handled in product category mapping.