Even the most rigorously researched statistics are not immune from misinterpretation, and they can often be used in a way that obscures the truth.
Enhance your critical thinking. Read more of our articles on the topic.
It is not only biased statistics that can harm the truth. The biased use of quality statistics may have similar results. There are two main mechanisms through which statistics can mislead: the first refers to the inability of the user of statistical data to understand and explain their real significance (and this inability is aggravated by an individual’s tendency to select statistics that support his preconceived opinions)[1], and the second refers to the biased use of statistical results, with the aim of misleading or manipulating other people. We will further analyse some situations that illustrate both the voluntary and involuntary misuse of good statistics.
Self-diagnosis is a real danger
Worrying about our health or the health of our loved ones is natural. Twenty years ago, going to the doctor when one’s symptoms interfered with one’s activities was the only logical course of action. Currently, however, when any symptom arises, we are tempted to do our own research online. ‘What are the diseases characterised by these symptoms?’ we ask. Sometimes, we even fully self-diagnose. Thus, some of us may easily become convinced that we are suffering from a rare disease from the Amazon Jungle…
What exactly leads to this situation? Among other things, medical statistics, both of good and dubious quality, when studied by an uninformed person without medical training, can lead to a misdiagnosis. A headache does appear on the list of symptoms associated with a brain tumour, but it also appears on that of a simple cold.
It is therefore statistically correct to associate headaches with brain tumours. But at the same time, brain tumours occur in only 0.002% of the population. Moreover, the diagnosis cannot be made on the basis of a single symptom. An overview, studied by a specialist, is required.[2]
The result of incorrectly using statistical medical data for self-diagnosis is unpleasant and complex. For doctors, it often means that, in the first part of the consultation, they have to first fight against the patients’ misconceptions, and this may damage the diagnostic process. For the patient, the misuse of quality statistics may mean a pointless increase in anxiety.[3]
The need for arguments
Michael Shermer, in his book The Believing Brain, argues that people first formulate their opinions and beliefs, and only then find evidence to support them. Thus, we are tempted to choose convenient statistics, and to build interpretations around them that support our beliefs. These can be transformed into a “permanent fixation of a person’s psyche.”[4]
This need to justify one’s opinions and beliefs can be both voluntary and involuntary. In the case of the involuntary mechanism, the subconscious is in control. (We will return to this mechanism in a future article, and analyse ways to neutralise it and move towards a correct way of analysing and thinking).
In the case of the voluntary mechanism, the user of statistical data is aware of his need or impulse to find arguments in favour of his beliefs and ideas. It is similar to “positive thinking”, when people try to see only the good things in life.
But, as Romanian author Andrei Pleşu says, positive thinking is the opposite of thinking. You can’t pretend to think if you prefer to see only what interests you. Extrapolating from this, the analysis of statistical data within the paradigm we build is the opposite of thinking, and for a correct analysis and good understanding, a paradigm shift is necessary, coupled with the willingness to learn and draw the right conclusions.
The mirage of secure investments
The tendentious—and therefore voluntary—use of statistics very often takes place in the stock market. We are presented with some fantastic results, obtained in a short time, with the claim that these results were obtained due to the skill of the investors—the superb statistical results of those who succeeded. But are things really like that? Is the whole truth presented to us? Are these results available to a large number of investors or only to a small percentage of them?
Professors Laurent Barras, Olivier Scaillet, and Russel Wermers studied the activity of 2,076 fund managers (1975-2006), and their conclusion was that “99.4% of these managers did not prove any competence in selecting shares”, 2.4% were competent in the short term and “only 0.6% beat the stock market index in the time interval we studied, a result that teachers described as ‘statistically equivalent to zero’.”[5] Reading these statistics, the mirage of safe investments disappears.
Cropped statistics
There is a possibility that statistics are not being fully but only partially presented. “By chance” an important part of them disappears. Studies are often cited that have shown links between drinking red wine and better heart health. To use this result outside of the big picture (which should also include statistics on the risks of alcohol consumption, or facts proving that people who consider themselves to have only moderate consumption actually consume higher amounts of alcohol than they perceive) constitutes a biased use of valid statistics.
Militant atheists use statistics that show a decline in religious affiliation as proof that religion will disappear and atheism will triumph. However, the big picture shows that although religious affiliation is declining, many people are moving from affiliation with a religious institution to an individual religion, which means that they have not given up believing in God, or given up on Christianity. In Europe, 18% of the population is not religiously affiliated, but a large number of European Christians believe in God.[6] The triumph of atheism does not seem to have been much of a triumph.
Correct use of good statistics
Sources of information must be chosen carefully. Regardless of the author (specialist or non-specialist) of the material we use to inform ourselves, the main goal should be to check the quality of the studies underlying it. It should be noted that correct statistics are not necessarily quality statistics. The size of the sample is also relevant. As Daniel Kahneman writes, “You would not take a drug that boasts a 100% cure rate, without any side effects, if you knew that it was tested on only five people. If all five people recovered without side effects, the statistics are correct, but the problem is the size of the sample.”[7] To avoid error, one must first check this aspect.
When determining the quality of a set of statistics, it is very important to analyse the data that are missing from the equation, and that might require us to use additional statistics to complete the table.
When the statistical results we analyse seem appropriate for subjective reasons, it is necessary to exercise our critical thinking to check whether there are other relevant statistics—even if they may surprise us, or contradict our point of view. The simplest method of identifying them is to analyse what critics have to say about the ideas that we embraced in advance.
Reading what the critics have to say does not mean we are giving in to lies. On the contrary, critical thinking must be practiced to help us, analysing the whole range of opinions, to discern solid arguments from weak ones, strengthening our beliefs and preparing to respond with good arguments in favour of the option we have embraced.
It is also important to mention at the end that, in general, no definitive conclusions should be drawn from a single set of statistics. Knowledge is dynamic and we must approach information with a capacity for synthesis, creativity, and critical thinking, and always be willing to integrate new information that may be provided in the future.
Enhance your critical thinking. Read more of our articles on the topic.
Footnotes[1]„Michael Shermer, The Believing Brain, Times Books, 2011.”[2]„„Mary Aiken, The Cyber Effect: A Pioneering Cyberpsychologist Explains How Human Behaviour Changes Online, John Murray Press, 2016.”[3]„Ibidem.”[4]„Michael Shermer, op. cit., p. 34. ”.[5]„Laurent Barras, Olivier Saillet & Russel Wermers, «False Discoveries in Mutual Fund Performance: Measuring Luck in Estimated Alphas», in The Journal of Finance, 2010, apud. Jason Kelly, op. cit., p. 46.”[6]„”Level of excess drinking of alcohol «is underestimated»”, BBC News, Feb. 27. 2013, https://www.bbc.com/news/health-21586566”.[7]„Daniel Kahneman, Thinking, Fast and Slow, apud. Jason Kelly, op. cit., p. 27.”