The biased sample is a kind of unrepresentative sample, either for quantitative reasons (as is the case with the too-small sample), or for qualitative ones, when its structure does not represent the structure of the real population that is the object of the research.
Enhance your critical thinking. Read more of our articles on the topic.
When researchers do not apply probabilistic techniques and procedures to develop a random sample, there is a risk for their own feelings, expectations and even their most dearly held convictions to influence, even if unconsciously, the structure of the sample. The result hereof is a “biased sample”, whose structure does not reflect the real structure of the studied population. Instead, it is a sample which tends to lean towards a particular segment of the researched population.
Let’s look at possible cases in which the biased sample is invoked in the argument.
Bias by unprofessional limiting of the sample
In sociology, a sample may be biased by containing, to a disproportionately large extent, people from the socio-cultural and professional environment of the sociologist. This is why we cannot say we have done objective research if we apply a questionnaire where it’s most convenient for us—on our family members, friends or coworkers—and then draw conclusions on “all European Christians” or on “European society”.
Individuals who are not trained in research (in other words, amateurs) may limit themselves to a convenient but biased sample when making an argument or a decision. For instance, a citizen may decide to run for mayor or for high prelate of a church, counting on the fact that all the people he met in the last month advised him to do so.
Bias by deliberate limiting of the sample
It may occur that the study is deliberately performed on a biased sample in order for its results to serve certain hidden interests. For instance, a so-called scientific study may recommend that everyone consume certain kinds of dairy products to ensure good bone health, based on a biased sample. The hidden part of the study could be related to a certain company financing the study and providing the sample of loyal and favourable consumers, to increase sales.
This is why, when you come across such studies that claim to be scientific, it is recommended that you check whether those developing the studies enjoy credibility in the scientific world or whether the respective study has good reviews from other specialists or approved forums. It is preferable to take reasonable precautions to protect ourselves from the “inflation” of studies which claim to be scientific in today’s mass media. Given the large amount of information facilitated by new media, we may come across studies that have been tampered with, but we may also discover ways to check them before taking them seriously.
Bias by nonrandom sampling
Sometimes, even if it’s big enough to be representative, the sample is nevertheless biased and therefore unrepresentative of a population’s makeup, because it did not randomly establish its structure divided into certain categories (age, gender, profession, social class, wealth, ethnicity, religion, etc.). For instance, during the 1936 U.S. presidential elections, the magazine Literary Digest predicted, using a large sample of two million voters from among their readers, that Roosevelt would be defeated by his counter-candidate. The Gallup association for public opinion polling, on the other hand, with a sample of just 50,000 voters, but randomly statistically represented, correctly predicted that Roosevelt would be elected. The first sample was biased because it consisted only of readers of the magazine, which were richer voters, unrepresentative of the voting options of the majority of American citizens.
What the biased sample fallacy reveals
This fallacy is mostly committed at the level of general knowledge, of people who do not have sufficient scientific and philosophical training to avoid the pitfalls of non-critical thinking. While an honest scientist refuses to work with unrepresentative samples, an uneducated man will not know how to avoid using such a sample.
For instance, someone’s individual experience, altogether unique and subjective (and therefore “biased”) represents such a sample—that is, a smaller part of the total experience of a family, social class, nation, generation or historical era.
A naïve individual will be the prisoner of his own experience, considering it relevant for all people: “If I wrote a book based on my life, I would get the Nobel Prize!” An ignorant person who despises instruction might boast as follows: “Some might be highly educated, but I went to the school of life!” Some other ignorant person might believe, based on his limited experiences with certain types of fellow humans, that all people are more or less the same as him: “Every person has a price, it’s just that the price is different!”
These kinds of people cannot fathom that there are other types of people, profoundly different from them and the people they have met, that the world is far more vast and diverse, and that not “all that glitters is gold”, as the popular saying goes.
In reality, reading and travelling are the two royal ways in which to expand ones personal experience, enriching it with other people’s experience—the experience of those who lived in other historical eras (by reading) and of those who live in other geographical areas (by travelling). According to cognitive psychology studies coordinated by Jean Piaget, decentering the individual from his own experience and the narrow group he belongs to is done by establishing as many interactions as possible, across the greatest possible distances—which are exactly the advantages of the globalisation process. Those going “out into the world” return less egocentric and ethnocentric, less biased and more objective, and more immune to the biased sample fallacy.
In a nutshell
The biased sample fallacy is some kind of statistically unrepresentative sample: either from a quantitative point of view, as is the case with the small sample, or from a qualitative one, when its structure does not reflect the structure of the real population. In the latter case, researchers do not make professional use of probabilistic techniques and procedures to develop a random sample, which is randomly established within a population. As a result, their feelings, expectations, values or convictions may influence, even unconsciously, the development of the sample. Other times, scientific rigour is deliberately neglected, precisely because there is bias in the selection of convenient results. For instance, a church may initiate online confession, based on a survey of the users of its official website who want this. However, the sample may be biased because it is possible that most parishioners do not access the website and do not agree with online confession.
Enhance your critical thinking. Read more of our articles on the topic.
Dumitru Borţun is a university professor at the Faculty of Communication and Public Relations, part of the National School of Political and Administrative Studies (SNSPA).
