Truth about daily fluctuations in 1992 pre-election polls

Truth about daily fluctuations in 1992 pre-election polls

Bare, John

Pre-election polls drew criticism from all directions in the 1992 presidential campaign, as politicians, news media and voters questioned the accuracy of the surveys. Political operatives complained when poll results showed their candidate slipping; newspapers defended the more controversial and complex survey techniques in detailed explanations to readers; and voters expressed disbelief in the ability of pollsters to tap into the minds of their fellow citizens.

Sometimes the doubts were expressed derisively, as when Newsday prefaced an October editorial with: “If the pollsters set it right for a change, Democratic presidential nominee Bill Clinton will sweep New York City with the force of a tidal wave next Tuesday.(1) Newsday neglected to mention that major polling organizations that have continued to survey voters up until Election Day have been extraordinarily accurate in recent years. Gallup, for instance, predicted the final results to within an average of 1.6 percentage points in the five presidential elections between 1972 and 1988.(2)

On election day, USA Today noted that voters, no doubt confused by a bundle of surveys that put Clinton’s lead over Bush at anywhere from 3 to 19 percentage points, might agree “with President Bush’s view of presidential election polls as nutty.” A Republican pollster told the newspaper that voters “won’t believe (polls) any more because the results are so conflicting.(3)

Candidates themselves, despite the fact that their own campaign organizations conducted countless polls, often said media surveys were not to be believed. Al Gore said he didn’t trust a poll mentioned in the Kansas City Star.(4) Ross Perot’s campaign spokesman brushed aside scientific surveys and said he had more faith in call-in polls sponsored by talk radio shows.(5) On ABC’s Nightline, after President Bush’s performance at the Richmond debate failed to increase his standing in the polls, James Cicconi argued that Bush actually won the debate and said, “I think we’ve seen polls vary from one poll taker to the next and from one night to the next…. I think the real poll that counts is on election day.”(6)

Clinton campaign directors complained loudly when Gallup, in its tracking poll for USA Today and CNN, began in late October reporting the preferences of likely voters instead of registered voters; the change nearly erased the large lead Clinton held over Bush.(7)

In at least three instances, in the midst of the heated three-way presidential race, the behind-the-scenes movements of the Gallup pollsters made news, and USA Today attempted to explain things such as how likely voters were selected for telephone surveys and how screening questions were used to prevent pollsters from always talking to too many wives and mothers–the people most likely to answer phone calls.(8) Standard voter complaints appeared in a cover story that ran under the headline: Polls: Odds are that no two are alike. One voter questioned whether pollsters reached everyday people like her and another voter said “there are a lot of hidden views that don’t come out in polls.”

Members of the media provided support for the notion that polls changed dramatically from day to day. Carolyn Smith of ABC News, in an October appearance on ABC’s Nightline said that “over the last month (women) have been going up and down, all over the place, as regards to Clinton, and that’s the volatility in the race.”(9) As in presidential elections of the past, occasional news reports said “forget the polls” and focused instead on long-watched, mystical trends supposedly capable of predicting the outcome of the presidential contest–women’s hem lines, bellwether districts and the quality of the grape harvest in France. In 1992, one report even linked electoral success to the number of dirt race tracks in the presidential candidates’ home states.(10)

University of New Hampshire political scientist David Moore was one of the few commentators who downplayed poll fluctuations. “I think that fluctuations are overestimated because we take a look at the leads rather than at the numbers for each candidate.”(11) More important, Moore explained, most daily changes that do occur in poll results are not due to magical shifts in the electorate. Instead, polls vary slightly from one day to the next because the results are estimates that fluctuate around an unknown mean.

METHOD

Results of 1992 pre-election polls were examined in two ways. First, the results of a series of polls conducted by a single survey organization were analyzed to determine whether there were dramatic day-to-day changes in the published finding’s of candidate totals.

Second, multiple polls conducted in identical time periods by different survey organizations were analyzed to determine if the results were in conflict. The analysis focused on the totals for each presidential candidate, as Moore suggested is proper, not the margin of the lead.

In considering the first possibility, the results of Gallup’s 33 tracking polls, conducted for USA Today and CNN, were examined to determine whether there was evidence that the daily results for each candidate fluctuated more than could be explained by sampling error. For each candidate, the magnitude of daily change was recorded, and an average was obtained.

From Oct. 1, 1992, through Oct. 25, 1992, Gallup published the results of three-day or two-day rolling averages of 1,000 or more registered voters. After that date, until election day, Gallup presented results of surveys of likely voters. The Gallup tracking poll was selected for analysis because it “received considerably more coverage” than other tracking polls,(12) and because Gallup received some of the harshest post-election criticism, for both the way its interviews were conducted and the way its results were reported.(13)

In considering the second possibility–that multiple polls conducted over the same time period by different survey organizations could present conflicting results–16 cases (from September 1 through election day) in which two or more polling firms conducted surveys simultaneously were examined. For the analysis, the difference between the highest and lowest result in each case for each candidate was recorded. The mean difference for each candidate was obtained to determine whether, on average, candidate totals from polls conducted at the same time by different organizations differed by a margin greater than sampling error.

RESULTS

The results (see Table 1) show that for each presidential candidate, Gallup’s daily results fluctuated an average of about 1 percentage point, a figure well within a sampling error of plus or minus 3 percentage points. (Table 1 omitted)

The largest single daily change was Bush’s increase of 5 percentage points. That change occurred in the October 25-26 poll, when Gallup changed its focus from registered voters to likely voters at the same time it surveyed just 800 respondents, its smallest sample of the entire tracking period.

Overall, the candidate totals in Gallup’s polls were remarkably consistent from one day to the next. Clinton’s daily total changed no more than 3 percentage points, and Perot’s daily total changed no more than 2 percentage points.

The results in Table 2 reveal that polls conducted in identical time periods by different survey organizations produced candidate totals that differed for Clinton and rot an average of 2.8 percentage points. (Table 2 omitted) For Bush, the average difference was about 2.2 percentage points.

The variance was greatest for Clinton, for whom the daily totals differed by as much as 8 percentage points. The results demonstrate that, for polls conducted simultaneously, candidate totals were, on average, remarkably consistent, even though the many survey organizations may have fashioned poll questions differently, selected respondents differently and reported results differently.

CONCLUSIONS

The results generally contradict the conventional wisdom regarding the weaknesses of pre-election surveys, which, as cited earlier, was expressed best by James Cicconi on Nightline three weeks before the 1992 election: “Polls vary from one poll taker to the next and from one night to the next.”

First, candidate totals in Gallup’s tracking polls changed an average of just 1 percentage point from one night to the next. Thus, if poll watchers keep a close eye on the daily results of tracking polls from a single survey organization that is consistent in its methodology and reporting they should expect to observe few drama tic one-day changes in a candidate’s share of the vote. Public opinion may be a moving target,(14) as one polling expert said during the 1992 election, but it does not appear to be moving anywhere in a hurry.

Second, candidate totals differed, on average, less than 3 percentage points from one poll taker to the next, according to the analysis of results of multiple surveys conducted simultaneously by different polling organizations.

Additional secondary analysis of the 1992 pre-election surveys is needed, especially research that follows Michael Traugott’s suggestion of evaluating the “methodological merits” of each nightly survey that later became part of a tracking poll and was reported as part of a rolling average.(15) But these findings are important because they reveal, despite complaints to the contrary, a seat consistency and an overall evenness in the 1992 pre-election survey results published by news media.

NOTES

1. Unsigned editorial, Back Bill Green. Newsday, October 28, 1992, p. 48.

2. Philip Meyer, Only the brave poll right until the end. USA Today, October 27, 1992, p. 13A.

3. Leslie Phillips, Adding up to confusion. USA Today, November 3, 1992, p. 12A.

4. Kirk Longhoffer, Gore: Kansas voters critical. United Press International, October 5,1992.

5. Sara Fritz, Perot Tactics May Be Too Unconventional To Have An Impact. Los Angeles Times, October 7, 1992, p. 13A.

6. James Cicconi, Nightline. ABC News, October 15, 1992.

7. Leslie Phillips, op. cit.

8. Jim Norman, Where those poll figures come from. USA Today, November 3, 1992, p. 4A; Richard Benedetto, Polling ‘likely’ voters narrows margin. USA Today, October 28, 1992, p. 8A; Cathy Lynn Grossman and Jim Norman, Polls: Odds are that no two are alike. US Today, October 19, 1992, p. 1A.

9. Carolyn Smith, Nightline. ABC News, October 14, 1992.

10. Jim Meyers, Bush at the oval. USA Today, September 22, 1992, p. 1 C.

11. David Moore, Morning Edition. National Public Radio, October 6, 1992.

12. Michael W. Traugott, A Generally Good Showing, But Much Work Needs to Be Done. The Public Perspective, November/December 1992, p. 15.

13. Robert M. Worcester, A View from Britain: You Can Do Better. The Public Perspective, November/December 1992, p. 17.

14. Cathy Lynn Grossman and Jim Norman, op. cit., p. 2A.

15. Michael W. Traugott, op. cit.

Copyright Ohio University Winter 1994

Provided by ProQuest Information and Learning Company. All rights Reserved