Many news programmes, speeches, or press articles refer to at least one poll. When designed and conducted well, polls are an excellent means of measuring public opinion on a particular topic. Unfortunately, not all surveys are well compiled, relevant, representative, or honest.
Enhance your critical thinking. Read more of our articles on the topic.
Polls are not only a barometer of public opinion, but also an effective means of influencing it. The fact that there are sometimes significant differences between reality and the predictions offered by a survey shows that, before accepting any survey as a true expression of reality, one must take into account some significant aspects.
Surveys can help us understand social phenomena or social trends. But polls can also lead us to the idea that we must obey the principles of the majority, or that we are anachronistic if we do not go in the direction that the measurements indicate. Polls can also influence decision makers to go in a certain direction which seems to be supported by the majority of voters. A little caution can help us observe opinions from the right perspective, and can make us understand the world we live in better and the direction in which it is heading more objectively.
Watch out for survey questions!
In order to be able to correctly decode the information from a survey, we need to analyse, as far as we can, the questions that the subjects had to answer. Even if it is impossible for them to be absolutely objective, we must check whether the survey subtly embeds, within itself, the answers that the person who commissioned the survey expects to receive.
Those who design a survey know that a vague or general question is likely to receive more answers than a very concrete one, but vague or general questions do not usually receive sufficiently nuanced answers to help us draw a correct conclusion on the subject matter.
In 2008, Pew Research published a very comprehensive study of how religion is perceived in the United States. This 256-page study brought new perspectives on American religiosity, but also raised many questions about how objective these analyses were. Specifically, the study stated that 70% of Americans (and 57% of Protestants) believed that there are several ways to salvation. In several Christian communities known for their conservative views, such as independent Baptists or Presbyterians, the percentages were even higher than the average mentioned.
This fact has led some Christian analysts to consider that the secret of these surprising figures lies in the question addressed to the subjects. They had a choice between two statements: “My religion is the only true religion that can lead to eternal life”, or, “Several religions can lead to eternal life.” However, the question does not define what “religion” means and, for many respondents, “religion” is often probably held to be equivalent to “denomination”. Thus many American evangelicals responded according to their belief that several evangelical denominations present the biblical teaching necessary for salvation, and hence the conclusion of Pew Research was erroneous.
It is important to note that such erroneous conclusions can also arise when there is a lack of flexibility in the questions. This can mean that respondents can only choose from the options offered, even if their opinion does not overlap precisely with any of the suggested answers.
Correlate the data
The conclusions of some surveys are often modified or nuanced by other surveys, which complement the conclusions provided by the former. For example, it is comforting for Christians to know that 95% of Romanians believe in God, but when they discover that Romania ranks first in Europe in the number of abortions, or that 61% of Romanians follow horoscopes, and 55% believe in curses, many realise that one cannot draw definite conclusions from certain polls, filling the rest of the picture with one’s imagination. In addition, when they say that they believe in God, Romanians do not say it from the perspective of the free-thinker who came to the conclusion that there is a supreme force, but from the perspective of one who considers himself a member of a specific community of faith. Given this, it is surprising that only 54.4% believe in the afterlife and about as many in heaven and hell. That is why a much more detailed analysis of the answers is needed to draw conclusions.
Sometimes the answer to a question involves other resources, of the soul rather than the mind. Sometimes, because they do not want to be judged by the people administering the questionnaire, survey respondents may decide to answer with what they think is better to answer, not with what they actually consider to be true.
Who commissioned the survey, and for what purpose?
It is also important when evaluating a survey to verify the identity of the person who ordered it, the sample that was taken, and the purpose of the study. A pharmaceutical company interested in selling its product might give you a leaflet that says its product is the most sought after by patients, while a bakery will calculate that its bread is the most sought after on the market. A political party or a candidate will be very keen on telling you—before the election—that they are first choice for other voters, and a telephone company will profess that their services are the best on the market.
All this can influence us to choose a certain product, to vote for a certain candidate, or to accept certain principles, because we are told that the majority recommends them. It is therefore important to observe phenomena from different perspectives and to consult as many sources as possible, because, sometimes, the answers are overwhelmingly different.
The Consumption Barometer for 2016 calculated that only 22% of Romanians attended church at least once a month in 2016, while, following a survey, the National Statistics Institute of Romania put forward the percentage of 45%. A central daily paper stated that the real percentage would only be around 10%.
The reality is that a survey that does not allow one to see a detailed presentation of the questions, the sample, the way the data were collected, and the margin of error, is simply a marketing tool. The fact that a product or idea seems to be supported by the majority is not in itself an absolute criterion. We need to do our own research, to analyse, to understand, and not be manipulated—especially not by survey results.
Enhance your critical thinking. Read more of our articles on the topic.