Nate Silver on Gallup and its history of volatility--
http://fivethirtyeight.blogs.nytimes.com...more-36284
Gallup Performs Poorly When Out of Consensus
Usually, when a poll is an outlier relative to the consensus, its results turn out badly.
You do not need to look any further than Gallup’s track record over the past two election cycles to find a demonstration of this.
In 2008, the Gallup poll put Mr. Obama 11 points ahead of John McCain on the eve of that November’s election.
That was tied for Mr. Obama’s largest projected margin of victory among any of the 15 or so national polls that were released just in advance of the election. The average of polls put Mr. Obama up by about seven points.
The average did a good job; Mr. Obama won the popular vote by seven points. The Gallup poll had a four-point miss, however.
In 2010, Gallup put Republicans ahead by 15 points on the national Congressional ballot, higher than other polling firms, which put Republicans an average of eight or nine points ahead instead.
In fact, Republicans won the popular vote for the United States House by about seven percentage points — fairly close to the average of polls, but representing another big miss for Gallup.
Apart from Gallup’s final poll not having been especially accurate in recent years, it has often been a wild ride to get there. Their polls, for whatever reason, have often found implausibly large swings in the race.
In 2000, for example, Gallup had George W. Bush 16 points ahead among likely voters in polling it conducted in early August. By Sept. 20, about six weeks later, they had Al Gore up by 10 points instead: a 26-point swing toward Mr. Gore over the course of a month and a half. No other polling firm showed a swing remotely that large.
Then in October 2000, Gallup showed a 14-point swing toward Mr. Bush over the course of a few days, and had him ahead by 13 points on Oct. 27 —just 10 days before an election that ended in a virtual tie.
In 1996, Gallup had Bill Clinton’s margin over Bob Dole increasing to 25 points from nine points over the course of four days.
After the Republican convention in 2008, Gallup had John McCain leading Mr. Obama by as many as 10 points among likely voters. Although some other polls also had Mr. McCain pulling ahead in the race, no other polling firm ever gave him larger than a four-point lead.
It’s not clear what causes such large swings, although Gallup’s likely voter model may have something to do with it.
Even its registered voter numbers can be volatile, however. In early September of this year, after the Democratic convention, Gallup had Mr. Obama’s lead among registered voters going from seven points to zero points over the course of a week —and then reverting to six points just as quickly. Most other polling firms showed a roughly steady race during this time period.
Because Gallup’s polls usually take large sample sizes, statistical variance alone probably cannot account these sorts of shifts. It seems to be an endemic issue with their methodology.
To be clear, I would not recommend that you literally just disregard the Gallup poll. You should consider it — but consider it in context.
The context is that its most recent results differ substantially from the dozens of other state and national polls about the campaign. It’s much more likely that Gallup is wrong and everyone else is right than the other way around.
