That was a scene from “The Last Hurrah,” based on one of the most famous political novels of all time. Spencer Tracey asks his flunkies for political intelligence. Today of course, politicians and voters alike prefer a more scientific crystal ball: surveys. What surveys are and how they work, is our task for tonight.
I’m Manolo Quezon, the Explainer.
This brings up the question most people have about public opinion surveys but are often afraid to ask because they think everybody already knows and understands the answer and they don’t want to look stupid. That question is this:
How CAN just 1200 people in a public opinion survey, represent the opinions of 45 million voters?
To help us explain the answer to this most important question, we have Dean Jorge Bocobo, who is probably best known as the blogger behind Philippine Commentary. He is a Harvard University graduate in Applied Physics who has worked, in a different lifetime he says, on quality control technologies for medical diagnostic systems like Cat Scanners and Magnetic Resonance Imagers. He also worked on airline crash investigations for General Electric Aircraft Engines in Cincinnati, Ohio. Nowadays, he describes himself as a reading, writing, ‘rithmetic kind of guy, and our Guest and Explainee for today.
I. Tyranny of numbers
In the 2006 midterm elections in America, every race was surveyed, and part of the campaign ended up being a debate on whether both parties were basing their strategies on surveys that may have been misleading.
We’ve had our share of survey controversies, but each time they come out, they’re big news
This editorial cartoon from 1969 says it all. And indeed as Nick Joaquin, writing as Quijano de Manila in 1963 reported, surveys have gotten bogged down controversially in the past. In that year a leading firm of the time, Robot, agonized over whether to release its survey results ahead of election day: it did, fearing its main rival, Index, which had been publishing survey results on voter attitudes, would print a survey on senatorial votes.
So Robot went ahead, but called its survey results a forecast of the senate election results. Both political parties of the time, the Nacionalistas and Liberals, cried foul. Not least, because Robot ended up being way off the mark. But for over forty years, surveys have become a an integral part of politics in our country.
Why do surveys matter? Because they measure public opinion. And public opinion is the most powerful political force in any Democracy.
Every election or plebiscite is really a large-scale, official Public Opinion Poll where the respondents include all the voters and not just some small fraction of them. Public Opinion during an election is the Voice of the People, Vox Populi, Vox Dei. If Democracy were a religion, the deity worshipped if not always obeyed is the God called Public Opinion. As a result, elections are said to be the only Public Opinion Surveys that matter. Note that the God called Public Opinion does not speak officially in tongues, but in NUMBERS, which we also call VOTES.
In between elections and plebiscites, there is naturally intense public interest and media curiosity about what the God called Public Opinion may be thinking at any given moment, about the economy, or social issues like poverty and hunger, the war in Iraq, the war on terrorism, taxes, prices, wages, impeachment, Supreme Court decisions, etc. It would be very useful and interesting if we could “read the mind” of the God called Public Opinion more often than every 3 or 6 years,
Well, some very clever people — called SCIENTIFIC PUBLIC OPINION POLLSTERS -found a way to conduct unofficial interviews with the God called Public Opinion even when there is no election going on.
Outfits like the Social Weather Stations and Pulse Asia Surveys, Inc, use a tool called Random Sampling to take a kind of blurry snapshot, a highly pixelated photograph of the instantaneous state of Public Opinion. In the typical SWS or Pulse Asia survey, they randomly select 1200 registered voters (300 each from NCR, Luzon, Visayas and Mindanao.) They ask these 1200 registered voters, what their opinion is on a range of issues and questions, like, “Are you for Charter Change” or “Do you think the Supreme Court will rule fairly on People’s Initiative?”
The percentage of the 1200 who answer “yes” to such survey questions, becomes the basis for a generalization and conclusion about the entire population of 45 million voters that is usually turned into a Media Release by the pollster. Thus people will suddenly read headlines or hear sound bites announcing some new statistical discovery, like: 52% of Filipinos Back Chacha” or “67% of Filipinos Doubt Fairness of the Supreme Court”
Pollsters make generalizations and draw conclusions about how the very, very many (45 million) think or feel based on what a very, very few (1200) say in a questionnaire. Not only that. The public opinion pollsters claim that trends in their data have predictive power, that they can for example, forecast the results of elections.
Amazingly the results of elections over many years have proven at least some of their claims! The Public and even the normally skeptical Media have granted at least the “scientific” Public Opinion Pollsters a measure of credibility because of their successful record in the past at predicting election outcomes with uncanny accuracy. Except for some very glaring instances, (like the 2004 NCR Election Exit Poll), SWS exit polls very accurately predict who will win an election even before all the votes are counted and the winner is officially announced, and by what percentage of the voters.
The predictive power of public opinion surveys was also demonstrated in the 2006 US midterm elections. American public opinion pollsters like Gallup Polls were predicting the Democratic victory after tracking public opinion on a range of important issues, from taxes, to corruption, to the war in Iraq and on terrorism
But even as SWS and Pulse are very successful at predicting results of elections or plebiscites, they also do surveys on questions that will not be settled by an election. For example, when the SWS or Pulse Asia carry out self-rated Hunger and Poverty surveys, or when they ask survey respondents whether they think the Supreme Court will rule fairly in a coming Decision, there will be no election or other objective measure to prove their measurement right or wrong.
Aren’t Public Opinion Pollsters taking the name of the God called Public Opinion in vain by seeming to put words in the mouths of 45 million voters that they only got from some random sample of 1200 people?
This brings up the question most people have about public opinion surveys but are often afraid to ask because they think everybody already knows and understands the answer and they don’t want to look stupid. That question is this:
How CAN just 1200 people in a public opinion survey, represent the opinions of 45 million voters? We’ll do some experiments to examine this question, when we return.
II. Manipulating numbers
That was another scene from “The Last Hurrah,” where the Cardinal is told by the monsignor how politics has changed. Today we often hear, surveys have become as abusive an influence on politics as the political machineries of old. But how do surveys actually function?
We think of surveys like it was a kind of magic. But it isn’t magic, is it, Dean?
Dean: Nope. To understand how and why random sampling works, imagine that you are in a contest in which a $1 million prize will be given to anyone who can correctly guess the fraction or percentage of blue balls in this crystal bowl full of blue and red Christmas balls. The exact number of each color ball is in this envelope. You may touch and even attempt to count the balls, but you only have one minute to do so. What is your best strategy to win $1 million?
Since there are thousands upon thousands of balls and you only have 60 seconds the best you can do is to take a small random sample of the balls, say ten of them, like so, and count how many red and blue balls there are.
[Explainee demonstrates by taking three random samples of ten balls each and averages the resulting percentages of blue balls as an estimate of the exact percentage. I’ve tried this now many times and it is amazing how close to 66.67% you can get just by averaging together 3 or four random samples of ten balls each. But on the show, I will produce an “experimental result” which we can be sure will be very close to the correct answer. The exact percentage hidden in the envelop can be revealed after the experimental result is shown and compared].
That’s the science behind surveys. But do the survey folks agree with our explainee, who has strong opinions, or myself, who couldn’t do math to save my life? Our discussion with the experts, when the Explainer returns.
My View
In the Philippines, perception is king, and plausibility is queen. If things are perceived to be, then they are, regardless of reality. At the same time, if something is plausible, it is viewed as probable –again, regardless of reality. Surveys are an attempt to replace “guesstimates” with genuine snapshots of public opinion. Sounds like a crystal ball? It is, but with a difference: it’s scientific, and unless the data’s tainted, it’s accurate.
Next year, our country will be expressing its will at the polls. Elections for the House and Senate, together, will constitute an indirect referendum on our government. Surveys help us figure out public opinion and can be a powerful weapon against efforts to cheat or rig the results. In 1995 and 1998, the surveys were spot on in how the public would vote. In 2004, it was spot on everywhere except Metro Manila. Our discussion suggests the reason why –the data got corrupted. But then, surveys weren’t the only thing corrupted and even sabotaged in 2004.
Next year we have a chance to redeem our democracy. And part of that process of redemption are the surveys. This early on, the surveys have told us that an indirect referendum on the presidency is in the minds of the people. The only thing left to see is if public opinion, and the public’s votes, will be correctly reflected in the results. And that’s not up to the political professionals, but you and me.
