More reasons why I don’t trust Rasmussen polls


As I’ve noted on numerous occasions here, the only political polls I trust are those that have proven records of reliability — no matter whether the results comport with my political biases.

For example, when reputable outfits like Gallup have shown  declines in President Obama’s approval rating in Gallup polls, I’ve reported them. In several instances,  I’ve even predicted such declines. I’ve also compared Obama’s Gallup numbers to those of previous presidents at similar junctures in their first terms.

Note, too, that I’ve almost invariably deemed polls commissioned by Fox News to be perfectly acceptable, because Fox, like most of the major media, employs the services of a reputable polling firm. Granted, the talking heads on Fox sometimes downplay poll results they don’t like, but the hard numbers speak for themselves, and they usually seem legitimate.

The one pollster I’ve criticized most is Scott Rasmussen, for various reasons, including dubious methodology and loaded questions. In disparaging Rasmussen polls, I’ve repeatedly noted that they often produce what are called “outlier” results — that is, numbers that are significantly at variance with those produced by more reputable polls, in the aggregate.

Rasmussen’s “outlier” tendency is dramatically illustrated in the chart shown above, created by Jed Lewison on the basis of data from Pollster.com. (I should mention that the “other polls” designated in light green include surveys commissioned by Fox News. So, we’re not just talking about polls sponspored by the so-called liberal media.)

The question naturally arises as to why Rasmussen’s numbers on generic congressional ballots so often differ from those in other polls. The answer, I suspect, is that those numbers get Rasmussen a lot of mention in conservative media, especially right-wing blogs.

And then there’s the matter of Rasmussen staying away from political contests that might embarrass him.

On that score, Kos has a good piece HERE that deals with Rasmussen’s apparently fearful failure to do polls on the various noteworthy primary races that were decided yesterday. (Warning: The piece includes a mild vulgarity.)

The most curious aspect of this matter is that Rasmussen uses a dubious “likely voters” model in measuring Obama’s popularity, despite the fact that nobody’s going to be voting for or against Obama until the year after next. But when the “likely voters” method might make a little more sense in cases of imminent elections, like those this week, Rasmussen is absent from the scene.

FOOTNOTE: Here’s another instructive example of how Rasmussen’s “likely voters” model skews his numbers:

The most recent Fox News poll shows that 48 percent of respondents approve of the job President Obama is doing while 43 percent disapprove. The Fox poll covered registered voters in general.

The most recent Rasmussen poll on the job Obama is doing shows 44 percent approval and 55 percent disapproval.

That’s a pretty wide variance from the Fox numbers — 11 percentage points on the disapproval side.

 Of course, the Rasmussen poll covered only so-called likely voters, whatever the hell that means.

I rest my case.

FOOTNOTE II: My favorite example of Rasmussen using a loaded question to get the desired result goes back to last year, when he clearly wanted to ingratiate himself with Republicans who were annoyed at the perception that Rush Limbaugh was the real leader of their party.

In a question directed only to Republican respondents,  Rasmussen asked: “Agree or Disagree: ‘Rush Limbaugh is the leader of the Republican Party — he says jump and they say how high.’”

Naturally, the vast majority of respondents said they disagreed. Who wants to admit that they jump at the command of a talk-show host (or at anyone’s command, for that matter)?

The right-wing media dutifully reported that this so-called poll clearly showed that Limbaugh was not the leader of the GOP.



  1. Give it up, dave. This stuff is way, way over your head. I’m serious. You’re out of your element on this subject, as evidenced by your utterly senseless comment.

    Snuss will probably agree with you, but nobody who is sentient will.

    Say what you will. I’m not going to bother arguing with you.

    Perhaps expdoc or Neftali or Mike Carroll or some other conservative who actually walks upright will have a comment or two on this matter. I hope so.

  2. Neftali

    Well…this article goes back 6 years, but it states Rasmussen is the most accurate in predicting the 2004 election. It also says that Rasmussen’s robocalling method is far more accurate than human calls used by other polling firms, something Pat fails to take into account:

    “What happens to the pollster comparisons if we switch to the spread method? Both of the automated pollsters still beat Gallup. Head to head, SurveyUSA missed the spreads by an average of 2.3 points; Gallup missed by an average of 5.4. Rasmussen cleaned Gallup’s clock, missing the spreads by an average of 1.6 points compared to Gallup’s 6.2. Rasmussen also whipped Zogby, erring by 1.0 points compared to Zogby’s 3.2. But the contest between SurveyUSA and Zogby was tighter: The human pollster was off by an average of 3.6 points, compared to the robo-pollster’s 2.5.”


  3. Neftali: That’s good stuff, in reply to which I have several points to make.

    1) Final polls before a presidential election tend to cluster. Differences from one poll to another usually are small. None of the major polls misses the actual results of the election by a wide margin (unless you go back to 1948).

    2) The Slate article to which you linked fails to cite the stated “margin of error” in the various final polls of 2004. Margins of error can vary from poll to poll. Some are as low as 2.5 percent, and some are as high 4 percent. Mind you, I’m not disputing the fact that Rasmussen’s final numbers were closer than those from other polls. But, just out of curiosity, I’d like to see how they relate to the varying margins of error. For all I know, that factor might be meaningless and not in any way take away from Rasmussen’s performance.

    All this stuff is very complicated, of course, and accusations of bias and faulty methods are as old as the polling industry itself.

    You might be interested in this article, which deals with Rasmussen’s rep for outlier results. It’s very fair to Rasmussen and even complimentary in places:

    I still find it curious that Rasmussen’s numbers on Obama’s approval rating are so far afield from those in other polls, including the Fox poll.

    And I stand by my criticism of Rasmussen’s loaded question with regard to Limbaugh and the Republicans.

    If you can find more good material on this subject, pass it along.

  4. Neftali: A question suddenly occurs to me (not that I expect you or any other reader to have the answer):

    If Rasmussen’s numbers usually are more favorable to Republicans than those produced by other pollsters (which, in fact, they are), why doesn’t Fox News commission Rasmussen to do its polling? Instead, Fox uses an outfit called Opinion Dynamics, which rarely comes up with results as favorable to the GOP as those from Rasmussen.

    Just wondering.

  5. expdoc

    “If Rasmussen’s numbers usually are more favorable to Republicans than those produced by other pollsters (which, in fact, they are), why doesn’t Fox News commission Rasmussen to do its polling? ”

    Well obviously because Fox is fair and balanced.

Leave a Reply

Your email address will not be published. Required fields are marked *