Natural language querying of event data and what it actually gets right

The VP of audience acquisition types a question into the box on Tuesday morning. How many of last year's attendees have registered again this year. She gets 3,144 back, and a bar chart, in about four seconds.

That number is defensible. So is 2,588, which is what the same question returns if the word attendee resolves to a badge scan instead of a registration row. Nobody told her a choice had been made. This is the part of natural language querying of event data that gets talked about least, and it decides whether the whole feature is worth having.

What that one question has to become

Take the English apart. Last year. Attendees. Registered again. This year. Four phrases, and each one is a decision about which rows to keep.

In a portfolio warehouse the query touches eight tables before it produces a single figure. A registration table for both editions. A person table, because the same buyer registers under two email addresses and the join has to run through a resolved identity rather than a raw string. A company table, if the question ever gets filtered by exhibitor or by sector. An event instance table, which is what turns "this year" into a specific edition id. An event table above it, because a portfolio has eight shows and the question is about one of them. A registration type table, to decide whether exhibitor staff and press count. A registration status table, to drop cancelled and duplicate rows. A badge scan table, if attendee means somebody who actually turned up.

Now the arithmetic. The 2025 edition finished with 18,640 registrations and 13,908 badge-scanned arrivals, so 74.6 per cent of the registered file walked through a door. The 2026 edition currently holds 9,210 registrations. Match this year's registrations back to last year's registration file and 3,144 people appear in both. Match them back to last year's scan file instead and 2,588 appear in both. The gap is 556 people, 17.7 per cent of the larger figure, and it exists entirely because one English word has two implementations.

Last year is the second trap. If the 2025 edition ran in March and the 2026 edition runs in September, then "last year" as a calendar filter and "last year" as the prior edition select different rows, and the prior-edition reading is almost always the one the asker meant. A model with no encoded knowledge of your edition calendar has no way to know that.

Why do enterprise schemas break text to SQL?

The published evidence on this is unusually clear. XLANG Lab released Spider 2.0 at ICLR 2025, a set of 632 real-world text-to-SQL workflow problems drawn from enterprise database use cases, with individual databases carrying over 1,000 columns. On the project's own comparison, GPT-4o scores 86.6 per cent on the original Spider benchmark and 10.1 per cent on Spider 2.0. The paper's abstract reports o1-preview at 21.3 per cent on Spider 2.0 against 91.2 per cent on Spider 1.0 and 73.0 per cent on BIRD.

Those are not small declines. They are the difference between a feature you can ship and a demo. What changed between the two benchmarks is exactly what changes when you move from a tutorial database to your own warehouse: many more columns, several SQL dialects, documentation and metadata that have to be read before the query can be written, and questions that need more than one query to answer. Reading those leaderboards properly is its own piece of work, which Q2 covers.

The useful conclusion for an event data team is about scope. The accuracy figures collapse as the schema widens, so the schema you expose to the box should be narrow by construction.

The semantic model is where the accuracy actually comes from

Microsoft's documentation for Power BI Q and A is the clearest published account of what that narrowing involves, and it is worth reading even if you never touch the product. It describes four things a designer configures: field synonyms, linguistic relationships called phrasings, a row label that identifies a single record in a table, and an option to hide a column from questions entirely. Microsoft's own framing of the first one is direct: "One of the most basic and effective ways to improve the Q&A visual experience is through adding synonyms for the names of tables and fields in your data."

Their worked example is a publishing company whose users ask about novel sales, in a model where the field is called product. Without novel mapped onto product, the question returns nothing useful. Substitute your own vocabulary and the point lands harder, because event data has more of this than most domains.

Microsoft has also announced that the Q and A experiences are going away in December 2026 in favour of Copilot, which tells you which way the industry is moving. I would still build the curated layer. The reason the curated layer worked was never the grammar engine sitting on top of it; it was that somebody wrote down what booth means in this model and which of the six date columns is the one people mean by "when they registered". A newer model on top of an uncurated warehouse inherits none of that.

Curating field synonyms from your own vocabulary

This is an afternoon of work and it is the highest-return afternoon in the project.

Start from what people actually type. Microsoft's tooling surfaces the questions users asked over the last 28 days along with the words the engine did not recognise, and that log is the shortest path to a synonym list you did not have to imagine. If you have no such log yet, the chat channel where your team asks the data person for numbers is the same corpus in a messier form.

Then map the terms that mean one thing in your warehouse and four things in conversation. Attendee, delegate, registrant, visitor. Booth and stand. Rebook, renew and resign. Net square feet, NSF and space sold. Comp, complimentary, guest pass and exhibitor allotment. A model with 40 exposed fields carrying an average of 2.3 approved terms each is 92 mappings, which one person can write and a second can review in a morning.

Two rules make the list hold up. Every synonym maps to exactly one field, because a term that points at two fields reintroduces the ambiguity you were removing. And every field that people should never query directly gets hidden, which for a registration model usually means the surrogate keys, the load timestamps and all but one of the date columns.

Which questions should the box be allowed to answer?

Not all of them, and saying so out loud is the difference between a feature people trust and one they quietly stop using.

Sort the logged questions by intent. In one quarter a team might log 214 questions, and when you collapse them into distinct intents the top 31 cover 167 of the 214, which is 78.0 per cent. Those 31 get a stored, reviewed query behind them, with a named owner, and the box routes to the stored query when it recognises the intent. The remaining 22 per cent go to a generated query, and a generated query gets shown with its SQL, its row count and its filter set so the reader can see what was decided on their behalf. That display is the verification layer Q3 works through in detail.

The tiering matters more than the model choice. A stored query for "how many registrations do we have for the current edition" is correct every time and costs nothing to run. Sending that question to a generator every morning trades a guaranteed right answer for a probabilistic one, and buys nothing.

Where this stops

A semantic model freezes a definition, and definitions move. The day somebody changes the rule for counting exhibitor staff, the box keeps answering with the old rule, fluently, with no error and no flag. That failure is quiet and it is the one I would watch for, because the whole appeal of the box is that people stop checking.

There is a second limit with no fix. Curating 31 intents leaves the other 22 per cent of questions unanswered by the reliable path, and those are disproportionately the interesting ones: the questions nobody has asked before, which is to say the questions worth asking. A curated model is good at what your team already knows how to ask and structurally worse at everything else. Anyone who tells you their system handles the tail as well as the head is describing an ambition.

The last one is easy to forget in the enthusiasm. The box cannot tell you that the question itself was wrong. Ask it for rebooking rate when what you needed was rebooked square footage, and you will get a fast, correct, useless answer, and the speed makes it less likely anyone stops to notice. Grading that kind of failure needs a question set with known right answers, which is Q10's subject and worth doing before you turn anything on for a wider audience.

Take the last 30 questions your team asked in your data channel, write the SQL for each of them by hand, and count how many need a table or a join your current model does not expose. That count is the size of the semantic model work, measured on your own vocabulary, and it will be larger than anyone guessed. Everything else in an AI reporting layer sits on top of it.