How to read statistics and polls: averages, margin of error and misleading charts
The news is full of numbers, percentages and poll results every day. But the same number, presented differently, can leave an entirely different impression. This media-literacy explainer sets out, in practical terms, how polls work, what a margin of error is, the difference between the mean and the median, and how charts can mislead.
A number is a powerful argument because it looks precise and trustworthy. That is exactly why numbers are sometimes used to mislead an audience: the same data, presented in different ways, can yield opposite conclusions. Below are the key concepts to keep in mind when reading statistics and polls, drawn from sources. These are not opinions but widely accepted rules of statistical methodology.
A poll: sample and population
No poll asks every single person in a population — it questions a small group (a sample) and extrapolates the result to the whole population. For this to work the sample must be representative, meaning it reflects the structure of the population (age, region, sex and so on). So the first question is: who was asked, how many were asked, and how were they selected?
The margin of error
A poll result is not a single exact number but a range. As Pew Research Center explains, "±3 percentage points at the 95% confidence level" means that if the same poll were fielded 100 times, the result would be expected to fall within 3 percentage points of the true value in 95 of those 100 cases. So figures of 52% and 49% may be statistically tied within a ±3-point margin of error. The smaller the sample, the larger the margin of error.
Not only sampling error
Pew Research Center stresses that the margin of error measures only the error arising from sampling. There are three other important sources of error: noncoverage error (part of the population has no chance of being sampled), nonresponse error (some groups are less likely to take part) and measurement error (questions are misunderstood or answers misreported). So a "±3%" label does not mean the result is perfectly accurate.
Mean and median
The "mean" (average) is the sum of all values divided by their count. But a few very large values pull the mean up sharply. For example, if nine people in a room earn 1,000 dollars and one earns 1 million, the "average" salary comes out at 100,900 dollars — which reflects no one's real situation. The median (the middle value when figures are sorted) is closer to reality in this case. So when income or prices are discussed, whether it is the mean or the median matters.
Correlation is not causation
Two things changing together (correlation) does not mean one caused the other (causation). For example, ice-cream sales and drownings both rise in summer, but neither causes the other — both are driven by hot weather. When you see a report that "A is linked to B," it is reasonable to ask whether a third factor lies in between.
Percent and percentage points; relative and absolute
A "percent" and a "percentage point" are different things. If an indicator rises from 4% to 5%, it has risen by 1 percentage point but by 25 percent. Reports often pick the larger, more striking number. Likewise, "the risk doubled" (relative) may be tiny in absolute terms — for example from 1 to 2 cases per 10,000 people. A change that looks large in relative terms should be checked in absolute numbers.
Misleading charts
A chart can also exaggerate a number. The most common techniques: not starting the vertical axis at zero (making a small gap look like a mountain), cherry-picking the time range (showing only a rising or only a falling period), hiding the comparison baseline, and distorting scale. When looking at a chart it helps to check that the axis values, the period and the source are all marked.
A practical checklist
When you see a number or a poll, a quick check: (1) who is the source — who commissioned and paid for the poll? (2) how many people, when and how were they asked? (3) is a margin of error shown? (4) is this a mean or a median? (5) is it percent or percentage points; relative or absolute? (6) does the chart's axis start at zero and is the period fairly chosen? These questions are not for rejecting the number but for understanding it correctly.
