RealClearPolitics.com was begun in 2000 and in the 12 years that have followed, it has become the gold standard for aggregators of political stories, punditry and polling data.
Almost every newscast of note refers to the “RealClearPolitics average of polls,” which the commentariat has come to rely upon as a way of smoothing out the inevitable differences between polls and the various methodologies they represent.
With success comes a problem, however, and that is the old rule of “garbage in, garbage out.”
Should “RCP,” as it is known among those who rely upon it, do something to guard its currency against devaluation?
Having built the best brand in the business, ought RCP’s editors to take some role in preserving the quality of its product?
Here’s the problem.
Any media organization can commission a poll. The quality of the results depends upon the professionalism of the pollsters conducting the survey.
Some polling organizations are so routinely lousy over a long enough period of time that they lose all credibility. Think Zogby. Ask yourself when you last saw a Zogby poll being cited as useful data?
Other polling efforts are kept afloat by big name sponsors despite long standing records of terrible bias, like the Minnesota Poll which is simply an occasion of mirth among folks like the bloggers of Powerline who have long tracked that polls hopelessly left-wing bias. The sponsorship of the Minneapolis Star Tribune keeps the Minnesota Poll going despite its transparent hackery.
PPP is a Democratic house which provides useful data if one corrects for the lie of the green.
Strategic Vision is thought to be biased in favor of conservatives, but I and some other conservatives view it’s results with some skepticism.
Rasmussen, by contrast, is generally thought by those on the center-right of the political spectrum to provide accurate results, as is Gallup.
So it goes. Everyone has opinions of pollsters, and RCP uses a blender and produces a broth that most folks find palatable.
Now, however, The New York Times/CBS/Quinnipiac poll of swing states raises a new question of whether obviously busted polls should be included in the RCP average.
The most important of these results –Florida and Ohio—at first surprised and then either thrilled or discouraged observers depending on whether they were cheering for Team Romney or Team Obama.
Then, however, the partisan makeup of the respondents was revealed. (HotAir's Ed Morrissey is always quick to find and publish such key data.)
Turns out the sample Quinnipiac used in Florida had 9% more Democrats than Republicans.
The sample used in Ohio had 8% more Democrats than Republicans.
Not only did these samples fail to reflect the partisan make-up of the 2010 election turnout in either state, they even oversampled Democrats compared to the 2008 turnout –and 2008 was the best year for Democratic turnout in a generation.
It is simply impossible to argue with a straight face that these results are at all useful as predictive of November’s result or even of public opinion in either state as it exists today. But that is exactly what both the paper and the television network did. CBS opened its story by declaring that "President Obama leads Mitt Romney among likely voters in Ohio and Florida." The Times' headline asserted that "New Polls in Three Battleground States Show Obama Edge."
Interestingly, the Times' reporters, Jeff Zeleny and Dalia Sussman, avoided anything like CBS' declaration in their write-up, as though both were aware of the deep flaw in the polls' sample.
The New York Times and CBS are both biased media organizations –the former greatly so, the latter less so but still left-of-center.
Everyone understands this, and arguments about this bias aren’t worth having.
Since these media organizations don’t care about accuracy, neither will they care about a misleading poll carrying their brand name even though individual reporters might refuse through artful writing to pump a lousy poll.
But should RealClearPolitics.com care?
I interviewed Peter Brown of the Quinnipiac University Polling Institute on Wednesday’s show. The transcript is here. Mr. Brown has no defense for the approach they took, other than to repeat that Quinnipiac has a great track record. I gave him opportunity after opportunity to defend the poll's methodology, and he simply could not answer the obvious questions.
When Mr. Brown agreed that a poll of all Democrats would have no utility in telling us anything worth knowing, the game was over, and he sensed it, refusing to provide any guidance on when a random sample crossed over from useless to useful.
Read the transcript and ask yourself why The New York Times or CBS or any “news” organization would use these results as the basis for a front page story?
There is no good reason to do so, no conclusions that can be drawn from the results about how the state is leaning in the fall or what issues are driving the electorate. If a poll of all Democrats is useless, then a poll wildly oversampling Democrats is also useless, though less obviously so.
Indeed, a sample that is ridiculously over-weighted with Democrats coming up with a “result” that President Obama is leading? That is worse than worthless. That is misleading.
So RCP has to decide whether to pollute its “poll of polls” with an obviously biased poll.
It took a dozen years to build a great brand. Will RCP act to protect it?