May 17 1996

Polls Give Numbers, But Truth Is More Elusive


Opinion polls are big news — especially during an election year. More than ever, this summer and fall, the polling industry will be in overdrive, constantly gauging what Americans think and how they intend to vote. But even the most accurate polls can be very deceiving.

Much has changed since a young man named George Gallup set out to prove that he could predict the results of the 1936 contest between incumbent President Franklin Roosevelt and challenger Alf Landon. In the past 60 years, polls have improved so much that we may be dazzled — and fooled — by their statistical precision.

The more we trust polls, the more likely they are to mislead us. Often, the fault is not in the pollsters but in ourselves: We’re too eager to believe that the numbers add up to truth.

"Slight differences in question wording, or in the placement of the questions in the interview, can have profound consequences," says David Moore, vice president of the Gallup Poll. He points out that poll findings "are very much influenced by the polling process itself."

Consider, for instance, what researchers discovered in a 1985 national poll: Only 19 percent of the public agreed that the country wasn’t spending enough money on "welfare." However, when the question contained the phrase "assistance to the poor" instead of "welfare," the affirmative responses jumped to 63 percent.

The fact that a wording change can cause a 44 percent shift explains how people can make opposite — and equally vehement — claims about what "polls show." The truth is that, at best, polls offer us flat snapshots of a three-dimensional world.

At worst, when they’re funded by partisans, polls may be purposely deceptive. In those cases, faulty polling can come back to haunt those who initially seemed to benefit from it.

In autumn 1994, Republican pollster Frank Luntz declared that each provision of the 10-point "Contract With America" had overwhelming public support. Luntz failed to mention that he’d only surveyed responses to GOP slogans.

Last year, as details emerged about impacts of the "contract," public disapproval mounted — and congressional Republicans who took comfort in their own pollster’s propaganda were brought up short. Six months ago, when Knight-Ridder reporter Frank Greve exposed the polling sleight-of-hand, he noted that "the House GOP’s legislative agenda isn’t just losing popularity; it’s probably shedding popularity that was overrated."

An editor at Congressional Quarterly, Philip Duncan, added: "The revelation that there are gaps in the contract’s appeal might have come sooner if the media had pressed Luntz during the 1994 campaign to document his claim of public support. But all too often, reporters simply pass along results of polls that were designed to influence voter sentiment, not merely measure it."

Regardless of their quality, polls that depict public opinion end up altering it. Poll data "influence perceptions, attitudes and decisions at every level of our society," Gallup executive David Moore writes in his recent book, "The Superpollsters."

"Polling dictates virtually every aspect of election campaigns, from fund-raising to electoral strategy to news coverage," Moore comments. "And, after our representatives are elected, polling profoundly shapes the political context in which they make public policy."

Some polls are skewed by intensive efforts to sway the electorate. For example, in times of crisis, many presidents have been able to orchestrate publicity that spikes the poll numbers — which are then cited as proof that the White House is in sync with the popular will.

While polling seems to offer choices, it also limits them. Author Herbert Schiller says that opinion-polling is commonly "a choice-restricting mechanism." Why? "Because ordinary polls reduce, and sometimes eliminate entirely, the…true spectrum of possible options." Schiller aptly describes poll responses as "guided" choices.

To make matters worse, the narrow range of options presented by pollsters is far from random. "Those who dominate governmental decision-making and private economic activity are the main supports of the pollsters," Schiller observes. "The vital needs of these groups determine, intentionally or not, the parameters within which polls are formulated."

We become overly impressed with polls when we pay too much attention to the answers and not enough to the questions.

[ Next | Previous |