Deriving seniority level from job title text on a registration form

An exhibitor asked how many of our visitors were decision makers, and the answer we gave came out of a rule somebody wrote in 2019 that nobody had opened since. It searched the job title field for the word director and reported the hits. Managing Director counted. Chief Executive Officer did not.

Deriving seniority level from job title text is a small piece of engineering with a lot of edge cases, and it is worth doing properly because exhibitors renew on the answer. The method is an ordered keyword ladder, and almost everything that goes wrong comes from the order.

Two axes hiding in one string

A job title carries a function and a level, and the same twenty characters answer both questions. Head of Procurement is procurement and it is senior. Procurement Assistant is procurement and it is junior.

Occupational statisticians separate these deliberately. The UK Standard Occupational Classification 2020, published by the Office for National Statistics, classifies jobs by two concepts: skill level, which is the competence associated with the education or experience needed to do the job, and skill specialisation, which is the field of knowledge the tasks sit in. SOC 2020 defines four skill levels and arranges 412 unit groups under nine major groups on that basis.

The design principle is worth borrowing whole. Extract the level and the function in separate passes, store them in separate columns, and let the report combine them. Collapsing a title onto a function taxonomy throws the level away by design, so a pipeline that only does that will leave you unable to answer the question exhibitors actually ask.

Run the level extraction on the raw title, or on the case-and-punctuation normalised version, before any synonym mapping has flattened Director of Sales into a sales function.

The ladder, run highest first

Four rungs cover most B2B shows: owner and C-suite, director and vice president, manager, and individual contributor. That set is the one used for tracking the mix across editions, and holding it steady matters more than getting it perfect.

Each rung has a token list. Rung one takes chief, ceo, cfo, coo, cto, cmo, president, founder, co-founder, owner, proprietor, partner, principal and managing director. Rung two takes vp, vice president, svp, evp, director, head of and general manager. Rung three takes manager, supervisor, foreman, team lead and superintendent. Rung four takes the individual contributor tokens your sector uses: engineer, specialist, coordinator, analyst, buyer, designer, technician, operator.

Evaluate a title against rung one first. If it matches, stop. Only if it does not match do you test rung two, and so on down.

The stopping rule is the entire method, because real titles match several rungs. Chief Operating Officer and General Manager matches rung one and rung two. Founder and Head of Product matches rung one and rung two. VP and Sales Manager matches rung two and rung three. Evaluate lowest first, or evaluate all rungs and take the last match, and every one of those people gets filed a level or two below where they belong, which biases the whole chart downward.

Here is what the ladder produces on a file of 27,500 registrations with a non-blank title. Rung one takes 3,850 rows, or 14.0 per cent. Rung two takes 6,325, or 23.0 per cent. Rung three takes 7,150, or 26.0 per cent. Rung four takes 5,225, or 19.0 per cent. That leaves 4,950 rows, or 18.0 per cent, matching nothing.

Where the ladder gets it wrong

Two failure modes account for most of the damage, and both are fixable with a pass that runs before the ladder.

The first is the containment trap. Assistant to the President contains president. Executive Assistant to the CEO contains ceo. Deputy Head of Sales contains head of, and that one is arguably correct, which shows how carefully the exclusion list has to be built.

On this file, 610 rows carry a rung-one token inside a subordinate pattern such as assistant to, executive assistant to, or reports to. That is 2.2 per cent of the file. Left in place they inflate rung one from 3,240 to 3,850, which is a 15.8 per cent overstatement of the most valuable number in the report. An exhibitor sales team quoting a C-suite share would be quoting a figure that is a sixth too high because of a substring match.

The fix is an exclusion pass that runs first and short-circuits the ladder. Match the subordinate patterns, assign the row to rung four or to unknown depending on what else is in the string, and never let it reach rung one.

The second failure mode is the word executive, which means opposite things on either side of the Atlantic and inside different sectors. On this file, 1,940 titles contain executive, and 1,510 of them are in the pattern function-plus-executive: sales executive, account executive, marketing executive. That is 77.8 per cent of the executive titles, and in UK and European usage those are junior roles.

So executive cannot go on a token list on its own. It goes on a pattern list: chief executive and executive vice president to rung one, executive director to rung two, and function-plus-executive to rung four. A show with a mixed international audience needs both readings, which means the rule has to look at the words around it rather than at the word alone.

Every ladder has a handful of these. Write them down as patterns with a comment saying why, because in two years somebody will find the rule and delete the part they do not understand.

What should you do with the unknown 18 per cent?

Publish it. On the same chart, as its own bar, every time.

The temptation is to assign unmatched titles to the bottom rung, on the reasoning that senior people usually have recognisable titles. It is a defensible guess and it produces a chart nobody can correct, because the guess is invisible once it is applied. The Government Data Quality Framework published by the Government Data Quality Hub in December 2020 defines completeness as the degree to which records are present and validity as the degree to which data is in the range and format expected, and an unmatched title fails both without being empty. Recording it as unknown keeps that fact in the data.

There is a second reason, which is arithmetic. If unknown is 18.0 per cent, then any share you quote has two versions: the share of all titles and the share of evaluable titles. Rung one is 3,850 over 27,500, which is 14.0 per cent of the file, or 3,850 over 22,550, which is 17.1 per cent of the titles the ladder could read. Three points is enough to matter in a renewal conversation, so state which denominator you used.

The unknown share is also your maintenance signal. When it moves, either the audience changed or the form changed, and both are worth knowing about within a week rather than at the next post-show review.

How coarse should the ladder be?

Four rungs. Possibly five if your sector has a genuine distinction between board and executive, and never more than that.

The argument for coarseness is that precision you cannot achieve is precision you should not publish. Distinguishing senior manager from manager from assistant manager from team lead requires the title string to carry information it usually does not carry, and the resulting four buckets will be populated by tokenisation accidents rather than by facts about people.

The argument against coarseness is that exhibitors want detail, and it is a weak argument, because what exhibitors want is confidence in the number rather than more categories. A mix report with four rungs, an unknown bar, and a written rule set beats a report with nine rungs that nobody can reproduce.

Coarse rungs also survive contact with international audiences better. Director means a board member in some markets and a mid-level manager in others, and a four-rung ladder absorbs that variation into a bucket that is roughly right in both readings. A nine-rung ladder puts the same person two rungs apart depending on where they registered from.

Where this stops

The title string is self-reported, unverified, and typed by someone who wants to finish a form. Nothing in this method touches that. A person who types Director because it sounds better than Senior Analyst lands on rung two, and no precedence rule detects it. If seniority drives anything commercially significant, it needs corroboration from a source outside the form, and that belongs in a governed record built from more than one system.

The ladder is also blind to organisation size, which is the variable that makes a level mean something. A managing director of a three-person distributor and a managing director of a national wholesaler occupy the same rung and represent very different amounts of buying authority. Where you hold a company size field, the seniority cut is far more useful crossed with it than alone, which is one of the reasons company sits so high in any ranking of fields by report usage.

And the rules will drift unless somebody owns them. Every edition somebody adds a token to fix a title they noticed, the historical series moves, and no one recomputes the prior years. Treat the token lists as versioned reference data with a date on every change, and rerun the whole history whenever they change.

Take your most recent edition and run two counts before you build anything. How many titles match tokens on more than one rung, and how many contain a rung-one token inside a subordinate pattern like assistant to. Those two numbers tell you how much your current rule is getting wrong today, and both are single queries.