Skip to content

The key influencers visual finds correlation and readers hear causation

BI and reportingUpdated 2026-08-238 min read

In short

The key influencers visual ranks categorical and continuous fields by how strongly they associate with an outcome, reporting a lift for each segment. It runs a regression on a sample and applies a significance test. It reports association only, and it degrades badly when a segment holds few records.

The slide said that registrations from the partner list were 3.2 times more likely to no-show, and the recommendation underneath said to stop buying partner list placements. It came from the key influencers visual, screenshotted straight out of the report, bubble and all.

The segment behind that 3.2 held 38 registrations. Nothing on the slide said so, because the visual does not show counts unless somebody turns them on.

The key influencers visual is a useful tool and this is its characteristic failure. It ranks factors by strength of association, presents the result as a multiplier in plain language, and leaves the two things a reader needs for judgement, which are the size of the group and the direction of cause, off the default view.

What the visual actually computes

Microsoft's description is accurate and worth reading literally: the visual "helps you understand the factors that drive a metric that interests you. It analyzes your data, ranks the factors that matter, and displays them as key influencers" (Microsoft Learn, 2026).

The mechanics behind that are documented. For a categorical outcome the visual runs a logistic regression using ML.NET, comparing the group that has the outcome against the group that does not. For a numeric outcome it runs a linear regression. The Top segments tab runs a decision tree to find combinations of values with a high concentration of the outcome. A Wald test decides whether a factor counts as an influencer, at a p-value threshold of 0.05.

Three operating limits matter more than the algorithm. The analysis runs on a sample of 10,000 data points. Microsoft recommends at least 100 observations for the state being analysed and at least 10 for the comparison states. And Direct Query is not supported, which quietly means the visual is looking at an imported model that may be a day behind the number on the tile beside it.

None of that stops the visual rendering when the recommendations are not met. It renders, ranks, and produces a sentence.

The sentence is the part that travels. Microsoft calls it a restatement, and it exists to help a reader interpret the panes. It reads like a finding, it can be copied out of a screenshot, and it carries no qualification about sample size or field selection. Anybody who has watched a plain-language model output get pasted into a board pack will recognise the pattern, and the fix is the same: make the supporting figures travel with the sentence rather than sit one click away.

Thirty-eight registrations behind a headline

Work the partner list claim properly.

The segment holds 38 registrations. Seventeen of them did not attend, which is a no-show rate of 44.7 per cent. The comparison group, everybody else, ran at 14.0 per cent. The lift is 44.7 divided by 14.0, which is 3.19, and the visual rounds it to 3.2 times more likely. The arithmetic is correct.

Now test how firm it is. Badge scan coverage is imperfect on most shows, and a handful of attendees are recorded as no-shows because their badge never scanned. Move three of those 17 into the attended column and the segment rate becomes 14 of 38, or 36.8 per cent, and the lift becomes 2.63. Three records, out of a file of tens of thousands, and the headline moves by more than half a multiple.

Then size the prize, which is the number nobody computes. At the base rate of 14.0 per cent, 38 registrations would have produced 5.3 no-shows. The segment produced 17. The excess is 11.7 people. The slide recommending an end to partner list placements is, in attendance terms, an argument about twelve attendees.

Twelve might still be worth acting on if partner list placements cost real money. The point is that the reader was given 3.2 and not 12, and 3.2 is the number that sounds like a strategy.

Why is the segment size hidden by default?

The visual can show counts. Microsoft documents a counts toggle on the Analysis card of the Format visual pane, which draws a ring around each influencer's bubble representing "the approximate percentage of data that influencer contains", and a sort option that orders bubbles by count instead of impact.

It ships off. That is the wrong default for anything an executive reads, and I would turn it on in every report before publishing, without exception.

I would go further and say the ring is not sufficient on its own. An approximate proportion drawn as an arc is a magnitude encoding at the bottom of the perceptual accuracy ordering, wrapped around a bubble, at a size of maybe 40 pixels. A reader trying to judge whether a ring is 0.4 per cent or 4 per cent of the data has been handed a task nobody performs well. Put the raw count in the text: "partner list, 38 registrations". It costs six characters.

The Top segments tab has the same shape of problem and handles it slightly better. Microsoft's worked example reports a segment where 74.3 per cent of customers gave a low rating against an average of 11.7 per cent, and states explicitly that the segment "contains approximately 2.2% of the data". That sentence is the model. Every influencer should arrive with its share attached.

Adding a field changes the answer

The deeper issue is that the ranking is conditional on which fields you put in, and the visual gives no signal about that.

Microsoft's own documentation makes the point with houses. Bedrooms drive house price until house size is added to the analysis, after which holding size constant means extra bedrooms explain very little. The documentation summarises it as a factor being an influencer by itself and not when considered with other factors.

Event data is full of these. Acquisition channel and badge type are correlated, because a channel that sells mostly conference passes will look like a channel effect until badge type is in the model. Region and show edition are correlated, because a region grew in the same year you moved the dates. Company size and seniority are correlated in every registration file ever collected.

The practical consequence is that a key influencers screenshot is only interpretable alongside the list of fields that were in Explain by. Put that list on the page, or in the tooltip, and a reader can at least ask what is missing. A screenshot with no field list is uninterpretable, and it is the artefact that travels.

The visual also stops short of telling you where a movement sits in the totals, which is what a decomposition tree does with a measure. V28 covers that, and the two answer genuinely different questions: one ranks factors, the other splits a number.

What would it take to call this a cause?

Austin Bradford Hill set the framing out in "The Environment and Disease: Association or Causation?" in the Proceedings of the Royal Society of Medicine in 1965, volume 58, pages 295 to 300. His subject was deciding when an observed association between an environmental factor and a disease should be treated as causal, and he offered a set of considerations to weigh rather than a test to pass.

A ranked list of lifts supplies one of those considerations, which is strength of association, and it supplies it on a sample. It says nothing about whether the factor preceded the outcome, whether the effect scales with exposure, or whether any plausible mechanism exists.

Two of those are cheap to check in event data. Ordering is usually available, because registration timestamps and channel tags are recorded at the same moment and you can confirm the tag was set before the no-show occurred. A dose response check is also available: split partner list registrations by how many partner touches preceded them and see whether the no-show rate climbs. If it does not, the association is more likely to be a property of who that partner reaches.

The third check needs an intervention. Withhold the placement in one comparable market and measure the difference, which is the holdout design B35 sets out. That is the only route from a ranked bubble to a defensible claim about cause, and it takes an edition.

Where this stops

The visual is honest about what it does and the interface undersells the caveats. A p-value threshold of 0.05, applied across every field you supply, means that with twenty uninformative fields you should expect roughly one to clear the bar by chance, and the visual will present that one in the same language as a real finding.

The 10,000 point sample is the second thing to hold in mind. On a portfolio file of 400,000 registrations the sample is 2.5 per cent, and a segment that is genuinely 0.4 per cent of the population arrives in the sample as around 40 records. Small segments are therefore doubly fragile: rare in the data and sampled thinly on top.

None of this argues against using it. It argues for using it as a search tool that generates hypotheses for someone to test, and for never letting a screenshot of it leave the building with counts turned off.

Open your most recent key influencers report, switch on counts in the Analysis card, and write the raw record count next to each of the top three influencers. Any influencer sitting under Microsoft's recommended 100 observations comes off the summary slide until somebody has checked it against the full file. That check takes ten minutes and it is the cheapest correction available anywhere in the BI and reporting set. The same instinct applies to markers on a time series, which V30 covers, where a mark with no expected value beside it is the same defect in a different visual.

Questions people ask about key influencers visual

How many records does the key influencers visual need?
Microsoft recommends at least 100 observations for the state being analysed and at least 10 for the comparison states. The analysis runs on a sample of 10,000 data points. A segment holding a few dozen records can still produce a confident looking multiplier, which is why the count belongs beside every influencer on the page.
Does the key influencers visual show causation?
No. It fits a regression that compares one group against another and reports which factors separate them. Microsoft's own documentation shows a factor losing its influence once a related field is added to the analysis, which is what a confounder does. Establishing a cause requires an intervention such as a withheld placement in one comparable market.
Why does an influencer disappear when I add another field?
Because the visual evaluates all explanatory factors together. Microsoft gives the example of bedrooms driving house price until house size is added, after which bedrooms matter far less. In event data the same thing happens with channel and badge type, since a channel that sells mostly conference passes looks like a channel effect until badge type is in the model.

Related reading

All bi and reporting articles