With fifty answers, a small lead can happen by chance. The confidence line tells you how likely it is that the leader would still lead if you asked fifty different people.
The three levels
- High: the lead is well outside what chance would produce. Act on it.
- Medium: the leader is probably ahead, but the gap could close. Fine for a low-stakes choice.
- Low: the options are close. Treat it as a tie.
We work it out from the gap between the top two options and the number of answers, so confidence firms up as answers come in.
When confidence is low
- Ask more heads. Doubling from 50 to 100 narrows the uncertainty noticeably.
- Drop the weakest option. Fewer options spread the answers less thinly.
- Ask why. A free-text follow-up to 20 Tier 2 heads often explains a split.
- Accept the tie. Sometimes the honest answer is that people don't mind.
What confidence isn't
It says nothing about whether your question was a good one. A leading question can produce a confident answer to the wrong thing. See writing good questions.