Designing a data stewardship queue that a human can finish each morning
A data stewardship queue should be ranked by expected value, the match score multiplied by the revenue attached to the record, and capped at the number of decisions one person completes in a day. Anything beyond the cap expires visibly, because an unbounded queue produces waiting times measured in weeks.
The data stewardship queue was set up in February with good intentions and a sensible interface. In August it holds 6,300 candidate pairs, the oldest is from March, and the person who was supposed to work it now opens it about once a fortnight, scrolls, closes it. Nobody decided to stop. The queue simply became a place where work goes to wait, and everyone learned that.
A review queue only works if it empties. That single property drives every other design decision, and it is a question of arithmetic before it is a question of tooling.
A queue that never empties is a backlog with a nicer name
Ten shows feeding a matching pipeline can easily produce 300 clerical review candidates a day in the weeks around their on-sale dates. Suppose one steward clears 80 in a day.
Arrivals of 300 against a throughput of 80 means the queue grows by 220 every working day. After a month of 20 working days, that is 4,400 waiting.
John Little's 1961 proof in Operations Research, of the formula that the mean number of items in a system equals the arrival rate multiplied by the mean time each spends there, gives you the consequence directly. Rearranged, the mean wait is the queue length divided by the throughput. With 4,400 waiting and 80 cleared a day, a candidate arriving that morning waits 4,400 divided by 80, which is 55 working days. Eleven weeks. The show it belongs to will have opened, run and closed.
Little's result describes a stable system, and the point here is that this one is not stable at all. When arrivals exceed service capacity there is no steady state and no equilibrium wait to compute: the queue and the delay both grow for as long as you leave them alone. Every queue in this condition ends the same way, with a person quietly abandoning it and nobody recording the decision.
So the cap comes first. Decide how many decisions get made in a day, admit that many candidates, and handle the remainder explicitly.
What should the queue be sorted by?
Not by score. Score is the model's confidence that two records are the same, and confidence is unrelated to whether the decision is worth making.
Rank by expected value: the probability the pair is a real match multiplied by the value attached to the record. Take three candidates.
Candidate A scores 0.62 on a contact at an exhibitor account with 18,400 in booth revenue attached. Its expected value is 0.62 times 18,400, which is 11,408.
Candidate B scores 0.94 on a day visitor record with 250 attached. Expected value 0.94 times 250, which is 235.
Candidate C scores 0.30 on a contact at an account with 42,000 attached. Expected value 12,600.
Sorted by score, the order is B, A, C, and a steward spends the morning on visitor records. Sorted by expected value it is C, A, B, and the same morning covers the two accounts where a wrong identity decision costs a renewal conversation. The gap between the top and bottom of that list is a factor of about 54, which is what makes the sort worth arguing about.
Candidate C also shows the tension. A 0.30 pair is probably not a match, and a queue ranked purely on expected value will fill with low probability pairs against large accounts, which wastes a steward's time in a different way. Put a floor under it: nothing enters the queue below a minimum score, and inside the eligible set, sort by expected value. Where the floor sits is a decision you revisit once you can see how often automatic merges are wrong, which is K25.
How long does a candidate wait once the cap is in place?
Compute it before you promise anybody anything.
With the cap, admissions equal throughput at 80 a day, so the queue reaches a length and stays there. If you allow a two day working buffer, the queue holds about 160 and the mean wait is 160 divided by 80, which is 2 days. That number can go into a service statement people can rely on: a candidate raised today is decided within two working days.
Contrast that with the uncapped version's 55 days, and note which of the two a sales team can plan around. Predictable and bounded beats thorough and unbounded, because the review only has value while the decision can still change what somebody does.
The buffer matters more than it looks. A queue held at exactly one day's capacity goes empty whenever arrivals dip, and an occasional empty afternoon costs nothing. A queue that regularly runs dry means your cap is above your arrival rate, and you should widen the eligibility floor rather than leave a steward idle.
Sizing the cap and letting the rest expire
The 220 candidates a day that do not fit have to go somewhere, and the only honest options are to expire them or to change the system that generates them.
Expire them visibly. Write an expired status with a reason and a date, count them daily, and put that count on the same report as the queue length. A number nobody looks at is still better than a silent drop, because the first time somebody asks why a duplicate was never caught, the answer exists.
Then check what the cap costs. Sort one day's 300 candidates by expected value, sum the top 80, and divide by the total. On a heavily skewed distribution, which is what account value normally produces, the top quarter of the list carries most of the total, so a cap that reviews 27 per cent of the candidates might capture something like three quarters of the expected value. Run that calculation on your own file, because if your top 80 carry only a third of the value, your ranking is doing nothing and the eligibility floor is set wrong.
The better move is upstream. Most large candidate volumes come from a small number of causes, and attributing the duplicates to the system that created them, which is K26, usually finds one form or one integration generating half of them. Fixing that removes candidates permanently, while reviewing them removes each one once.
What the queue has to show
A steward can only be quick if the screen holds the whole decision.
Both records side by side, in their original form before any survivorship rule ran, with the fields that drove the score highlighted. The source system and creation timestamp for each, because a ten minute gap between two registration records is a form resubmission and needs no thought. Any downstream links, so the steward can see that one of the two is already attached to an exhibitor contract. And a reason code list for the decision, short enough that nobody picks other.
The DAMA International DMBOK, second edition, published in 2017, defines data stewardship as accountability and responsibility for data and the processes that ensure effective control and use of data assets. Accountability is the part event teams skip. A queue owned by the data team collectively is a queue owned by nobody, and the practical version is one named steward per domain with a named deputy, both written down where the queue is configured.
Where expected value ranking breaks
Most event records have no revenue attached. A visitor who has never bought anything, a speaker, a journalist, a student on a free pass: all of them can be involved in a duplicate and none has a number to multiply by. Ranking them at zero pushes them permanently below the cap, which is defensible for a day visitor and wrong for the buyer whose booth spend sits under a different record entirely.
The workable fallback is a tiered proxy. Records linked to an exhibiting company first, then records for shows opening within 60 days, then everything else, with expected value used inside each tier where a number exists. It is cruder and it survives contact with a file where 80 per cent of records have no transaction on them.
Two other limits are worth stating. Expected value assumes the steward's decision is correct, and stewards disagree with each other often enough that the ranking is finer grained than the underlying judgement. And revenue attached to a record is itself an identity resolution output, so a queue ranked by it inherits whatever errors are already in the identity graph, which is circular in a way you cannot fully engineer around.
Start by measuring your own numbers for one week: candidates generated per day, candidates cleared per day, and the queue length at the end of each day. If the second is smaller than the first, you already know the queue's fate and you can compute the wait with one division. Then put the expected value sort in front of it and watch what moves to the top, because the answer usually reveals that the headline duplicate rate in K24 is being driven by records nobody would ever have chosen to spend a morning on. Deriving how many decisions one person actually completes in an hour is K31's arithmetic, and it is the input this whole design needs before the cap is anything more than a guess. All of it sits under the same unified data problem: a golden record is only as good as the decisions somebody had time to make.
Questions people ask about data stewardship queue
- How should a data stewardship queue be prioritised?
- By expected value rather than by score. Multiply the probability that a candidate pair is a real match by the revenue attached to the record, and sort on the product. A pair scoring 0.62 against an account worth 18,400 outranks a near certain pair against a record worth almost nothing.
- What happens if a stewardship queue is never emptied?
- The waiting time grows without limit. If 300 candidates arrive daily and a steward clears 80, the queue lengthens by 220 every working day, and after a month a newly arriving candidate sits behind more than four thousand others. By the time anyone looks at it, the show it belonged to has closed.
- Should low value duplicate candidates be reviewed at all?
- Not by a person. Below the cap they should expire from the queue and be recorded as expired, so the volume is visible and the decision to skip them is deliberate. Many of them are better addressed by fixing the form or the integration that generates them than by reviewing them one at a time.
Related reading
- False merge rate is the number your stewards should watch most closely
- Duplicate rate as a metric you can actually report to a board
- Which source creates duplicates and how to prove it with a simple query