How to game surveys



IT’s always been a question asked by ordinary citizens. How can 1,500 respondents represent the entire country of over 100 million people, around 54 million of whom are of voting age?

Another question that is always asked is, how come one has never been included as a sample in a survey? The answer to this is that because if the sampling is done correctly, then the probability that one person of voting age can be included in the list of 1,500 respondents is approximately one out 36,000, or 0.003 percent.

And the answer to the first question is actually premised on the assumption that every Filipino voter will have equal chance of being selected, regardless of age, sex, ethnicity and location. This can only happen if the sample was chosen randomly, and without any biases. Ideally, if this is the case, then the 1,500 sampled individuals will truly be representative of the entire population, and their responses will in fact capture the opinions of the whole country, or rather, its entire voting population.

But this ideal situation is not practicable, for it is not possible to enumerate all the 54 million voters and assign corresponding codes for each, and then randomly select from these the 1,500 to be included in the survey.

Thus, survey organizations modify the randomization process, and use geographical location as the sampling unit, usually randomizing regions, then provinces, then cities or municipalities, and then barangays. Others may even go down to the level of precincts.

This sampling technique would carry with it the risk of not taking into consideration the bailiwick effect, where certain candidates or issues may in fact have specific localities where extreme responses are expected to occur. A candidate or an issue that is popular in one region or province that is not included in a sample may in fact have an underestimated support in the survey.

As practiced, field enumerators, which refer to the teams that are tasked to conduct the actual survey in the field, do not actually have the names of the voters. What they have instead are fixed points in a sampled area, like a monument or a prominent landmark, and then derive their samples from the n’th household encountered by walking north, south, east and west from the reference landmark.

This practice tends to have a relatively urban bias, or a bias towards residents of town centers. This bias also applies when enumerators rely on handheld applications that are dependent on cellular site signals, for such will further limit the sampled areas only to those reached by those signals.

Thus, one aspect of surveys which can be manipulated to game their results is in the selection of samples. But this would amount to committing what can be considered as a mortal sin in survey taking.An honest survey will always try to minimize sampling bias.

Another important element of the survey is the questions asked, or what are technically referred to as the survey instruments. Survey questions must be structured in such a way that they are not leading and biased, and should be clearly stated. Ideally, negative statements are generally avoided, and each question must contain only one issue or concern.

This is another opportunity to game the results of surveys. Survey questions can be structured to lead respondents to a preferred response, or to make the questions complicated and confusing for ordinary respondents that will force them to respond in an expected manner favorable to one side.

I do not mean to accuse anyone of anything, but one example of a leading and complicated question is when SWS, in its recent survey on the so-called EJKs, asked respondents to agree or disagree with the following statement:

“Marami sa mga pinatay ng mga pulis sa kampanya laban sa ilegal na droga ay hindi totoong nanlaban sa pulis” (Many of those killed by the police in the campaign against drugs did not really resist arrest.)

Aside from using a negative statement, the use of the word “pinatay”(killed) is loaded as it already gives the impression of intent to kill, which is different from “namatay” (died).

A much better way of phrasing the question would have been: “Kung nanlaban o hindi nanlaban ang mga namatay sa kampanya ng mga pulis laban sa ilegal na droga.” (On whether or not those who died during the campaign by the police against illegal drugs resisted their arrest.) Responses would have been categorically scaled as “nanlaban, walang opinyon, hindi nanlaban” (resisted, no opinion, did not resist).

Finally, a more insidious way of gaming the survey results is when the randomized location where the samples will be drawn are leaked to interested parties. An aggravating circumstance is when such is done in exchange for some favors or bribes. Knowing the location where samples will be taken will enable an interested party to saturate the place with information materials, conduct rallies and other campaign activities there,increase visibility by attending social events, or fill the airwaves with propaganda favorable or in some cases detrimental to the other side. This is done days before the actual taking of the surveys to deliberately influence the results.

Surveys, far from becoming merely a barometer that reflect public opinion, have become influencers and shapers of public opinion. As such, they acquire the power to make candidates win. When used to measure the performance of governments, surveys can also be very effective in painting leaders as effective or inept, or as popular or despised. They can even be used for political destabilization.

Surveys are supposed to be mechanisms that have become important in democratic politics, for they enable measurements of people’s sentiments.

But when they are gamed, they no longer become objective measures of people’s opinions, but have become weapons to manipulate and misrepresent their voices. They become instruments by those who would like to undermine democracy to serve their partisan and selfish political interests.


Please follow our commenting guidelines.

Comments are closed.