Bias & Statistics

There are three types of lies: lies, damn lies, and statistics 

[Multiple sources of similar quotes]


If your experiment needs a statistician, you need a better experiment

Ernest Rutherford Pioneer of Nuclear Physics

Humor about statistics is nearly as widespread as lawyer jokes – and perhaps for good reason. Playwright George Bernard Shaw touched on a problem with statistics when he quipped: of those who make a habit of eating, very few survive. Even simple averages can be just as challenging because humans can be accurately described as having, on average, one breast and one testicle. It seems disagreement - and claims of bias - reach a peak when statistics are used to defend a particular belief. 


So if straightforward statistical statements have the capacity to confuse or misrepresent can this branch of mathematics ever be used to determine the facts of a matter? We will explore this question from the perspective of human biases; and the finding may surprise you.


To start there are a multitude of statistical traps ready to snare the unwary amateur. For these there are many entertaining books and websites available - a couple of book examples* being: The Art of Thinking Clearly by Rolf Dobelli and The Signal and the Noise by Nate Silver. But we will be looking at the gaffes that are routinely made by professional statisticians and scientists. These errors are distinct from those made by amateurs and they typically flourish from a single blunder: the conscious and sub-conscious pursuit for connections and associations that corroborate the argument they are making (also see our Topics: What is Unbiased Reasoning and the link: Other People's Thinking.

Imagine you wanted to carry out a bias-free socio-economic study of crime rates among dissimilar demographics. Aspects under consideration may include the type of crimes committed and the geographic locations for each of the crimes. But when suspects interact with authorities then policing procedures – both theoretical and actual – might also to be included in the analysis. As suspects respond differently when apprehended then that aspect of the matter could also be recorded. When the crime includes a victim then the study may well include their demographic. Finally, before becoming a crime statistic an event or arrest must pass through a law enforcement system which itself has a number of other issues with statistical implications. Those aspects should also be included.


From the paragraph above numerous permutations of the data could be used to demographically characterize a crime along with its policing and its prosecution. For the conclusions to be manageable a researcher would typically need to narrow down the number of issues. This is where bias could come into play. One can be sure that the biases will be influenced by the culture of the perceiver along with each of the factors detailed in the page The Three Bias Reflexes in this site. 


An approximate answer to the right problem is worth a good deal more than an exact answer to an approximate problem

John Tukey - Mathematician 



Closely inspecting people’s comments about a multitude of topics, two bias issues jump out: Firstly, it is noticeable that both commentators and social scientists select which items they consider to be significant; and secondly, and most remarkably, is the degree of confidence they have that their particular selection is the way to represent the truth. 


In summary any statistics about social science issues will only be as good as the overall topic of bias is recognized and managed. This should include a study of the World of Bias and Topics pages at this site to get to grips with the multitude of problems that lurk in the background, ready to trip up logical thoughts and ideas. 


* This website's editor does not necessarily agree with the opinions expressed in these publications.



Instead of rejecting the ideas of others that you think are biased, try to figure out the different assumptions both of you are relying upon


A conclusion (and a bias) is the place where you got tired of thinking 

Paraphrase of Arthur Bloch – US author, producer – Thinking Allowed


Few occasions render a person more confident about their fairness and open-mindedness than when they describe Bias in others