|

If you think political polls can’t be trusted, think again!

Some years ago, an acquaintance of mine opined that most people don’t trust political polls.

“How do you know that?” I asked, expecting him to tell me that such was the attitude of most of his friends. But he surprised me. Instead, he told me that he had read that in “some survey.”

Well, of course, I told him that the survey to which he referred was itself a poll. Why would he trust that poll but not others? He didn’t offer an answer, but it was fairly obvious that he trusted only those polls that showed results generally in agreement with his own opinions.

I don’t know about that “survey” to which this guy referred, but I do know about a Gallup poll of a few years back that asked Americans about polls in general. The results showed one glaring contradiction: There was a considerable gap between respondents who said our leaders should pay more attention to “public opinion” than those who said our leaders should pay more attention to polls.

How can our leaders discern public opinion other than to pay attention to polls? Mail and calls from constituents aren’t a very reliable measure. The only good option is a good poll — one that is conducted by time-tested, scientific methods.

The truth of the matter is that lots of our leaders, especially those who hold elected offices, pay close attention to polls. Granted, for varous reasons I won’t belabor here, they don’t always heed majority sentiment as registered in those polls, but they keep an eye on the results to see which way the wind is blowing with regard to certain issues.

Many politicians — conservatives and liberals alike — commission their own polls. Many media — conservative and liberal alike — also commission polls. Mind you, not all politicians and not all media can afford the considerable expense of a well-conducted, scientific poll.

In a post  last month about how there are fewer presidential polls this year, I quoted polling expert Nate Silver as follows:

From January through July 2008, there were 558 state polls released that tested the Obama-McCain McCain horse race. By contrast, there were only 329 through the same date this year — about a 40 percent decline.

What’s causing the reduction?

Austerity at news media companies is undoubtedly a part of the problem. Polling isn’t free, nor is it particularly cheap — at least when it’s done well.

Indeed, polling is becoming more expensive because fewer people are picking up the phone and responding to surveys. You have to churn through a lot more calls to get the same number of completed interviews.

More Americans are not reachable on landlines at all. Failing to place phone calls to the roughly one-third of Americans who rely solely on cellphones is becoming less and less acceptable — but calls to cellphones add to the expense of polling.

As for the accuracy of so-called horse-race political polls — which, of course, is the only measure of their trustworthiness — Nate Cohn of The New Republic offered THIS PIECE just the other day:

With [Mitt] Romney trailing by a clear margin, a frontal assault on the accuracy of polling has begun. Some of the skepticism is understandable, but other elements are brazenly self-serving. I’ll be assessing the critiques of the polls over the next few days, but any debate about the accuracy of the polls must start by remembering a central point: The polls are usually pretty good.

(Snip)

Despite recent accusations, there isn’t much evidence suggesting that the polls are systemically biased toward Democrats. In fact, the clearest instance of bias in any direction came in 2010, when the polls systemically underestimated the strength of Democratic senatorial and gubernatorial candidates. Was this because of the unique circumstances of 2010 or because most close races were fought on heavily-Democratic turf, where undecided voters in a tight race are disproportionately composed of Democratic-leaners? It’s hard to say. 

(Snip)

Are the polls getting less accurate? While the pro-GOP bias in 2010 and the possible cell phone issue might lead some to believe the polls are getting less accurate, that’s not yet evident. The polls did tilt-GOP in 2010, but the error wasn’t that much greater than prior elections.

(Snip)

No, the polls aren’t perfect. Polls have been wrong before and they certainly will be again. There is nothing wrong with noting that the polls are imperfect and could be wrong in this election; they might be. It might even be understandable if someone argued that there was a greater chance that the polls are wrong than usual, given declining response rates and challenges with voters relying on cell phones. But asserting that the polls are wrong, simply because the result doesn’t match expectations is a recipe for disappointment. Analysts arguing that the polls are fundamentally inaccurate have rarely been vindicated by the results. If you dismiss the polls when you disagree with their findings, you’ll usually end up wrong. 

Share:

4 Comments

  1. The problem with a lot of polls is the unrealistic sample sizes. Obama won by +6 in 2008, so that is the target sample that some pollsters appear to be striving for.

    There is absolutely no way on earth Obama wins by 6% this year. Remember, the Republicans won the House by +6 in 2010. So these pollsters think there will be a 12 point swing BACK to Obama in 2 years? Yes, Obama is more likable, and more people come out to vote in Presidential years. But fewer people are equating Romney with Bush, unlike what happened with McCain. Plus Obama doesn’t have organizations like Acorn this time around busing people to polling places and instructing them who to vote for.

    He might very well win, but the results will probably be closer to Obama +3, meaning something like Obama with 51%, Romney with 48%.

  2. Neftali: I have no idea what you’re trying to say.

    You say “Obama won by +6 in 2008, so that is the target sample that some pollsters appear to be striving for.” What’s that supposed to mean?

    Sample size is the number of people polled. That has nothing to do with some candidate’s margin of victory in a prior election.

    Polling is a complicated science, as I’ve often said. But I don’t you think understand it at all.

  3. By the way, the smaller the sample size, the larger the poll’s stated margin of error — assuming that everything else about the poll is purely scientific.

  4. One other little thing: Obama’s 2008 margin of victory in the popular vote was 7.3 percent, not 6 percent.

Leave a Reply

Your email address will not be published. Required fields are marked *

CAPTCHA Image

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>