David Spiegelhalter
I love numbers. As a small boy I used to write down car registration plates, and when it came to my toy cars I’d line them up according to their number in the catalogue. When I grew up I became, surprise, surprise, a statistician.
Despite their mundane reputation, statisticians are sensitive souls who can’t bear seeing numbers abused. So when I listen to the radio or open a newspaper, and someone starts arguing a point with a statistic, I start by being mildly sceuuptical and often end up shouting.
Sometimes the problem is the sheer numerical incompetence of the sub-editors who produce the headlines. It is still not clear whether the classic, “‘We hate math,’ say 4 in 10 – a majority of Americans”, was a clever joke or stupidity.
Numerical idiocies come in many forms, but there’s a cluster around the idea of averages. A favourite sport of mine is spotting politicians who want every school to be better than average; one elected representative was apparently disturbed at a report that suggested 50% of obese people earn less than the average income.
Sometimes the mistakes may be deliberate misrepresentations. For example, the UK’s Sunday Telegraph reported 13,000 needless deaths in 14 worst NHS hospitals. The figure of 13,000 was obtained by taking the total number of deaths in the 14 worst-performing hospitals over an 11-year period and subtracting the number that would have been predicted in an average hospital. But, as with schools and obese people, around half of all hospitals are worse than average. In spite of being explicitly told this was the wrong interpretation, the Telegraph labelled all deaths above average as “needless”. It may not be a complete coincidence that the Telegraph supports privatisation of NHS services.
Of course a number may be true but just not the appropriate choice. The leader of the British Labour Party recently declared himself appalled that the UK had the second-highest youth unemployment in Europe, which is true if you count the bodies. But as a rate per 100,000 young people, the UK is two-thirds down the table.
So what are we to do to protect all these delicate statistics?
Part of the statistician’s mantra is that the imprecision of all statistics should be explicit. So when a rise in the number of unemployed by 38,000 in three months was headlined as “worrying jobless rise”, it seemed a task for a forensic statistician. After a long search of the official website, I managed to excavate “Table A11: labour force survey sampling variability”, which revealed that the 38,000 was accurate to +/- 87,000. So unemployment may have gone up, or it may have gone down, nobody knows.
Perhaps the most common naughty number is when some second-rate study produces a headline of the type “X doubles the risk of Y”, where X is some everyday habit (using deodorant, having a cat, etc.), and Y is some dread but rare outcome (autism, mouth cancer). Almost on principle, I don’t believe these claims. And even if true, as a mathematician I’ve carefully learnt the principle that “twice very-small-indeed is still very-small-indeed” (although I usually say this using a rude term for “very-small-indeed”).
Of course, this abuse of numbers is only part of the problem of reporting science. Others include making huge claims based on experiments on a few rats, cherry-picking exciting results, mistaking association with causation, and so on. A favourite of mine was the headline (again in the Telegraph) claiming Fizzy drinks make teenagers violent. The study asked US teenagers how many fizzy drinks they had and how violent they were. The results showed a positive relationship. But perhaps the causation is the other way round: maybe kids need a nice fizzy drink to cool them down after a spree of violence? It’s no more fatuous than the Telegraph headline.
So what are we to do to protect all these delicate statistics? Constant vigilance is required, and there is increasing attention from blogs such as Full Fact, BBC radio programs such as More or Less and Britain now has a Statistics Authority to tell politicians off for misusing them.
But while the misuse of language, spelling and grammar is generally considered unacceptable, people still seem to be able to get away with abusing numbers. Those who put out naughty numbers should be publicly embarrassed.
