Skip to content

Using small models for narrow tasks inside an event data pipeline

AI for organisersUpdated 2026-08-238 min read

In short

Route by task. Classification, field tagging, extraction and routing decisions run well on compact models at a small fraction of the cost and latency of a frontier model, and grounded summarisation rankings regularly place compact models near the top. Keep the large model for narrative paragraphs a person will read.

An inbound question arrives in the exhibitor services inbox asking whether the stand allocation for hall 2 has been published. Somewhere in the pipeline, a model reads it and decides it belongs to the floorplan category rather than the invoicing one. That decision has four possible answers, takes about eleven output tokens, and needs no judgement about tone.

Sending it to the same model that writes your post-show exhibitor narratives costs roughly ten times what it needs to and takes several times as long. Using small models for narrow tasks is the routing decision that fixes it, and the reason it is worth writing down is that most pipelines default to one model everywhere because that is how the first prototype was built.

What makes a task narrow

Three properties, and a task needs all of them.

The output set is closed or nearly so. A category from a list of eight. A boolean. A date. A stand number. Free prose fails this test immediately.

The output is checkable without human judgement. If you can write an assertion that catches a wrong answer, you can monitor the small model in production and find out quickly when it degrades. "Is this category in the list" is checkable. "Is this paragraph appropriately worded for a first-time exhibitor" is not.

The task carries no requirement to be interesting. Classification either lands in the right bucket or it does not, and nobody will read the output and judge your organisation by its style.

Tasks in an event pipeline that meet all three, in rough order of how much volume they carry: routing inbound exhibitor and attendee questions to a category, tagging free text company or job title fields against a taxonomy, extracting fields from exhibitor documents, deciding whether a natural language question needs a warehouse query at all, detecting whether a free text field contains personal data, and normalising a country or industry string.

Tasks that do not meet the test: the exhibitor narrative, the board-pack summary, anything that has to explain why a figure moved, and anything a reader will forward.

Do small models actually hold up?

On the measure that matters most for grounded work, the evidence is better than most teams expect.

Vectara has published a hallucination leaderboard since November 2023, scoring how faithfully each model summarises a source document using its own evaluation model. In the edition updated in May 2026, running over 7,700 curated articles at temperature zero, a lite-class model from one major vendor records a 3.3 per cent hallucination rate with a 99.5 per cent answer rate, and a 14 billion parameter model records 3.7 per cent. A small model from another vendor sits at 5.5 per cent with a 100 per cent answer rate. Those are grounded summarisation figures competitive with far larger models on the same table.

The honest reading of that is narrower than "small models are as good". Grounded summarisation is a constrained task: the facts are in front of the model and the job is to restate them without adding. Constrained tasks are exactly where the gap between a compact model and a frontier one is smallest, which is the same reason classification transfers well. Open-ended reasoning is where the gap is large, and nothing on that leaderboard says otherwise.

That is the argument for routing rather than for downgrading. The whole point is to send the constrained work to the cheap model and keep the unconstrained work where it is.

A worked example: routing a season's traffic

Take one show's season of pipeline traffic and split it by task.

Inbound question classification: 38,000 calls, roughly 900 input tokens and 12 output tokens each. Free text field tagging on registration records: 14,000 calls at 400 input and 8 output. Exhibitor narrative summaries: 900 calls at 8,000 input and 600 output.

Send everything to one frontier-priced model at an assumed 3.00 US dollars per million input and 15.00 per million output.

Classification: 38,000 times 900 is 34.2 million input tokens, costing 102.60. Output is 456,000 tokens, costing 6.84. Tagging: 5.6 million input at 16.80, and 112,000 output at 1.68. Narratives: 7.2 million input at 21.60, and 540,000 output at 8.10. Total 157.62 dollars, of which the narratives are 29.70 and the two narrow tasks are 127.92.

Now route. Send classification and tagging to a compact model at an assumed 0.10 per million input and 0.40 per million output. Classification becomes 3.42 plus 0.18, which is 3.60. Tagging becomes 0.56 plus 0.04, which is 0.60. The narratives stay at 29.70. Total 33.90 dollars.

That is a 79 per cent reduction on the season, and the summaries a person reads are unchanged. The narrow tasks were 81 per cent of the spend and none of the visible value, which is the usual shape once anyone looks.

Latency moves as well, and for the classification path it moves in a way people feel. A routing decision that returns in 200 milliseconds instead of 1,400 is the difference between a form that responds and a form that hangs.

Two cautions on that arithmetic. Unit prices are assumptions here and yours will differ, sometimes by a lot, so redo the multiplication with your own rate card before quoting the 79 per cent anywhere. And the model spend is only one line of what the feature costs, which the full per-answer accounting in Q24 sets out; if the narrow tasks feed a queue somebody reviews, the review minutes will dwarf everything on this page.

Evaluate per task, because there is no best model

The instinct after a successful bake-off is to standardise. One model, one prompt library, one vendor relationship, done.

The evaluation research has been arguing against a single ranking for years. When Stanford's Center for Research on Foundation Models published Holistic Evaluation of Language Models in 2022, the design point was breadth: seven metrics measured for each of 16 core scenarios, reported per scenario rather than collapsed into one score. Different scenarios produce different orderings, which is why the report is a matrix.

For an event pipeline that means a separate, much smaller evaluation per narrow task, and each one is cheap because the outputs are checkable. For classification, take 300 real inbound questions, label them by hand once, and score candidates on exact category match. That is a morning of work and it gives you an accuracy figure with a real interval, which is more than you will ever get from a narrative eval of the same size. Building the larger graded set for the narrative model is a different and slower job, covered in Q27.

Set a floor rather than chasing a maximum. If your hand-labelled set says a compact model classifies at 94 per cent against the frontier model's 96, the question is what the two per cent costs. For routing an inbound question to a queue where a human reads it next, two per cent is a small amount of extra forwarding. For deciding whether a free text field contains personal data, two per cent is a compliance incident, and that task stays on the better model or gets a rule in front of it.

What should stay on the large model?

Anything a person outside your organisation will read as prose. The exhibitor narrative, the sponsor recap, the attendee follow-up.

Anything that has to hold several facts in tension and produce a judgement, such as explaining why rebooking fell in one hall and rose in another.

Anything where the input is a large table, because handling a wide table is one of the places compact models degrade most sharply, and the serialisation choices that make it survivable are a topic of their own in Q29.

Anything currently unmeasured. A task with no eval set goes on the model you trust most until somebody builds the eval, and then it gets reconsidered. That is a defensible default rather than a permanent answer.

Where this stops

Routing adds a component, and the router is now a thing that can be wrong. A pipeline with two models and a routing rule has more failure modes than a pipeline with one model, and the additional ones are subtle: a question that should have gone to the large model gets classified as narrow and comes back thin, and nobody notices because thin answers do not raise errors.

Keep the router dumb. Route on task type, decided by which code path called it, rather than on a model's judgement about difficulty. A router that itself uses a model to decide which model to use has moved the problem rather than solved it.

The second limit is operational rather than technical. Two models means two sets of prompts, two upgrade cycles, two sets of eval results, and two vendors if they come from different places. For a small data team that overhead is real, and if your total AI spend is 200 dollars a season, saving 79 per cent of it does not pay for the maintenance. The routing argument gets strong somewhere above a few thousand calls a week, and below that a single model is the right call for reasons that have nothing to do with capability.

Start by counting. Pull a week of calls to your model provider, group them by which code path made the call, and sum the tokens per group. If one narrow task is carrying more than half your input tokens, you have found the candidate, and moving that one path is an afternoon of work with a measurable result. The wider set of platform decisions around AI sits with the pillar.

Questions people ask about small models for narrow tasks

Which event data tasks suit a small model?
Anything with a closed output set or a short deterministic output. Classifying an inbound exhibitor question into a routing category, tagging a free text industry field against your taxonomy, extracting a date or a stand number from a document, and deciding whether a question needs the warehouse at all. All of them have checkable answers, which is what makes a small model safe to use.
Do compact models really score well on grounded summarisation?
On Vectara's hallucination leaderboard they consistently do. In the edition updated in May 2026, a lite-class model records 3.3 per cent with a 99.5 per cent answer rate and a 14 billion parameter model records 3.7 per cent. Size and grounded faithfulness are much less correlated than general capability rankings imply.
How much cheaper is routing by task in practice?
It depends on the mix, but the classification traffic is usually the high-volume half. A pipeline handling 40,000 classifications and 900 narrative summaries a season spends almost all its calls on the cheap task, so moving that half to a model priced ten times lower changes the invoice substantially while the summaries stay where they are.

Related reading

All ai for organisers articles