It seems that are as many political polls as there political stories and web sites. But are they equally valid? And how can they project the votes of an entire nation by talking to only a thousand or so people? Read on to learn how to deconstruct a poll to determine its veracity.
There are several points to consider before accepting that a reported result is valid. The easiest is to look at the margin of error. It's "easiest" because it deals with statistical validity. For example, if 50 percent of those polled believe "x" and 47 believe "y" but the margin of error is +/- 3 percent, then there is no statistical difference between the two viewpoints!
If a poll or a news story reporting the results of a poll fail to mention the margin of error, then take the results with a proverbial grain of salt.
Next, how many people were surveyed and who were they? Some polls feature "adults" while others are "likely voters" and yet another set might be "registered voters."
Who answers the questions is important. When the poll asks "who would you vote for if the election were today?" then the respondents should be registered voters! It would be better yet if they also voted in the past election, as past performance is a good predictor of future voting action. But this kind of questioning weeds out respondents, which means more phone calls are required to get the representative sample. More phone calls = more time = cost more.
In addition, the error involved in extrapolating results from small populations to large ones increases as sample size decreases.
Most media surveys are performed at a 95 percent confidence level. This means that the statisticians are confident that if the survey were conducted with the larger audience, 95 percent of the time the results would fall within the margin of error. Be careful if you're trying to compare surveys with different confidence levels or different margins of error.
Also ask yourself: how was the question worded? Professional pollsters know how to phrase a question so that it is neutral rather than one that leads the participant towards an answer. Beware news reports that summarize polling results but fail to provide full text of the questions.
If more than one question was asked, the sequence of questioning is also important. For example, if there has been bad news recently reported about the economy, and people are asked their opinion on the economy before they are asked how well they think the President is doing his job, the Presidential numbers are apt to be lower than if the questions were asked in reverse order.
For more reading, see 20 Questions A Journalist Should Ask About Poll Results
from the National Council on Public Polls.