In scientific research, sampling is the primary method used when research cannot be conducted on a one-to-one basis. The facts discovered at the level of the sample are presumed to apply in general.
In order for the proprieties discovered at the level of the sample to be generalised at the level of the population, the sample must be representative of the entire population. This means it must represent it from two different perspectives: First, quantitatively (the volume of the sample must be big enough), and second, qualitatively (the structure or features of the sample must be similar to those of the population). To ensure the first condition, researchers use the laws of statistics. To ensure the second, they resort to random sampling methods, to make sure the sample is not biased. Therefore, in scientific research, the sample is always statistically representative and, at the same time, random.
Unfortunately, at the level of human knowledge, things are not quite the same. We often make the mistake of drawing conclusions based on a sample which is far too small to be statistically representative. This happens when we raise our unavoidably personal experience to the rank of generally valid experience. This fallacy is called the “small sample”. For instance, in my town, which depends on a large industrial company, the robotisation of the production line has led to layoffs and has rendered some of the workers unemployed. Still, I am not entitled to conclude that “Robotisation leads to unemployment everywhere.” If I had the bigger picture, I would know that robotisation creates new areas of knowledge and activity, new industries, and tens of thousands of new work opportunities.
Let us attempt to identify the motivations which may push us to use the small sample fallacy when we want to make an argument.
The desire to impose an interesting point of view on others
As can be seen, the fallacy of the small sample is intertwined with the hasty generalisation. In order to identify this fallacy, we must be aware of the fact that those rushing to assign certain opinions and perceptions to large groups with no correct prior study or scientific research, wish to impose those opinions and perceptions on others.
This is the main reason for rushing to extrapolate an interesting, personal vision on something to a generality. In daily life, and in the media, we come across such ironies as: “Writer X is very well-read and appreciated, especially by the people living in his building.” “Young artist Y is brilliant, according to all his aunts!”
One should not outright reject the phenomenon which generated the notion—equally ironic, of course—that of “cliques”1. This means several individuals group together and set themselves apart from the bigger group, because, at some point, they have come to see things differently than the majority.
At the origin of this notion, the problems causing divergences and separations were religious ones and those forming cliques separated themselves from other members—their brothers up until that moment. This phenomenon of forming cliques was brought about by the initiative one of them took, one who thought that he had gathered enough “votes” for his ideas in order to plant a new church. The motivation was often to be found in the ego of this individual, or even in his stubbornness to impose his point of view on the majority, without any other arguments. Later on, other groups or factions also formed distinct communities (social movements, parties, professional groups, syndicates, and so on). These have started to be called “cliques”.
Ignorance regarding the complexity of a population
We can easily avoid the small sample fallacy if we are aware of the complexity which characterises any population. A human population has a wide diversity of values, preferences, interests, ideals, and goals which animate its members. If we are ignorant about the scientific character a correct sampling requires, than we may naively err—sometimes even comically.
A truly comical situation arose over four decades ago, during a class at the Faculty of Philosophy of the University of Bucharest, in the Pedagogy Department. The teacher, a colourful character, was presenting a lecture after conducting research inside the department, on its own students. It concerned school failure: “I’ve noticed there are three categories of pupils: first—the pupils who are smart when entering school and are still smart when they finish it; second—the pupils who are stupid both when starting and when finishing; and third—the ones who start off being smart and finish by becoming stupid. This is where the school is failing!”
From the back of the room, one of the students stood up: “Wrong, professor! I have a cousin who failed fourth grade, dropped out of high school and made it to Medical School where he was head of his class.” Astonished by such an anomaly, the professor cried out: “Impossible!”
It’s obvious that both characters were using the small sample fallacy: the teacher jumped to conclusions, and generalised based on small-scale research, which was conducted in-house, on that year’s Pedagogy students. The student hurried to contradict the results of research based on a personal case, which might as well have been a singular one. In fact, the “cousin syndrome” is very present in “natural thinking” (as opposed to critical thinking—natural thinking which is not trained to supervise and control itself).
In a nutshell
For facts which are discovered to be true at the level of the sample to be generalised at the level of the population, the sample must be representative of the entire population, when it comes to these two aspects: quantitatively (the volume of the sample must be big enough), and qualitatively (the features of the sample must be similar to those of the population).
In proper scientific research, the sample is always statistically representative and, at the same time, random. Unfortunately, in common knowledge, we make errors by drawing conclusions based on small samples, which are not statistically representative. We raise our personal or collective experience, which is inevitably particular, to the rank of generally valid experience. This is the small sample fallacy, which links to the hasty generalisation fallacy. This occurs, for instance, when someone declares that young people nowadays are “moving away from the church”, basing his view on the particular observation that the church in his town is attended by a very small number of young people.
Dumitru Borţun is a university professor at the Faculty of Communication and Public Relations, part of the National School of Political and Administrative Studies (SNSPA).