How Reliable are Those Polls?

Posted: Mar 29, 2006 10:11 AM

We may hail public opinion polls or deplore them, but there is no getting around their importance. When some newspaper announces that only 35 percent of the American people think President Bush is doing a good job, what is the sensible response? We may wonder just how many of the American people were asked (it is usually a few hundred), or how representative the sample was, or even exactly what the question was. But there is no easy way to evaluate the poll, and we have little choice but to go along with the conclusion in the headline.

Politicians, when asked, are usually philosophical about polls, but they are as fascinated by them as everybody else. We have no better means of finding out what the American people think (at least, until they vote on Election Day), so we all accept polls warily, realizing that there is no alternative available. But that's not to say we have to buy whatever conclusion the pollster, or the headline writer, wants us to draw. Take one of the oldest and most famous poll questions of all: "Is the country on the right track, or the wrong track?" Recently, it was reported that 62 percent of those polled (presumably a representative cross-section of the public) currently think we are on the "wrong track." This was promptly interpreted as yet another condemnation of President Bush. But is it?

There are so many things wrong with the question that it's hard to know where to begin. In the first place, it assumes that there are only two tracks the country could possibly be on: the "right track" and the "wrong track." And yet most people would surely have a more nuanced and sophisticated notion of the country's direction than that. Many would feel that the country is on the wrong track in certain respects, but on the right track in others. How can they possibly answer the question honestly?

Besides, whichever track the country is on, it doesn't by a long shot follow that Bush put it there, or can easily move it to some other track. Lots of people think the country is on the wrong track in letting our obligations to Social Security recipients get so far out of line with our ability to meet them, and many even remember that the president practically knocked himself out last year trying to drum up support for his plan to remedy this. Yet everybody who, with Social Security in mind, tells a pollster that the country is on the "wrong track" will find his opinion scored as adverse to Bush.

In fact, people who disagree with one another on just about everything will find themselves lumped together as anti-Bush if they say America is on the "wrong track." Pat Buchanan, at the right end of the political spectrum, may think the country is on the wrong track because millions of illegal aliens from Mexico are flooding across the border. Someone else, on the far left, may think we are on the wrong track because many states won't let hundreds of thousands of convicted felons vote. Yet both Buchanan and the leftist are counted as members of a hugely discontented "majority."

Moreover, certain instinctive defense mechanisms of the public are used to maximize its apparent disaffection. Americans hate to be fooled, and have an innate skepticism, bordering on cynicism, about what politicians tell them. If asked whether they think President Bush is being entirely truthful about something, a lot of people will say no. That isn't to be taken as a flat assertion that he is lying; it is more accurately just a protective stance, assumed in order to defend against the possibility. And yet we are assured that large numbers of the American people believe that President Bush is an inveterate liar.

Of course, that saving skepticism applies even to polls, which is why there is a limit to their influence. We sense that a biased pollster can get pretty much any answer he wants, if he is allowed to shape the question. But unfortunately we cannot resist being influenced, to some degree, by what he tells us the American people think.