What Are Push Polls, Anyway?
Karl G. Feld
A quick glance at press stories and official candidate and organizational statements released over the last four years clearly indicates that there is little consensus in the wider political community of what a “push poll” might actually be.
ANOTHER ELECTION is upon us. With it comes the campaign telephone calls to voters for opinion research, voter identification, GOTV, advocacy and, inevitably, for “push polls.”
A quick glance at press stories and official candidate and organizational statements released over the past four years clearly indicates that there is little consensus in the wider political community of what a “push poll” might actually be. Yet, confusing accusations by media and campaigns of push polling activities abound, accusations damaging to the research industry as a whole.
The tremendous growth of Web surveys only adds to the problem. The ease with which these surveys have already been abused to “push” respondents and other data consumers — combined with the ease with which the results can be disseminated — adds a whole new threat to opinion research.
As researchers head into another period of concentrated phone work, short deadlines and intense media pressure, it might be a good idea to review the characteristics of legitimate research versus push polling. The National Council on Public Polls (NCPP) defined a push poll in a 1995 press release as:
[ldots]a telemarketing technique in which telephone calls are used to canvass vast numbers of potential voters, feeding them false and damaging ‘information’ about a candidate under the guise of taking a poll to see how this ‘information’ affects voter preferences. In fact, the intent is to ‘push’ the voters away from one candidate and toward the opposing candidate. This is clearly political telemarketing, using innuendo and, in many cases, clearly false information to influence voters; there is no intent to conduct research.
This definition closely matches those used by the American Association of Public Opinion Research (AAPOR) in its 1996 “Statement Condemning Push Polls,” the American Association of Political Consultants (AAPC) in its June 1996 Declaration Regarding So-Called ‘Push Polling’,” and the Council of Marketing and Opinion Research’s (CMOR) statements on “Political Telemarketing vs. Legitimate Polls and Surveys” and “Defining Political Telemarketing” of February 1999. Each organization has its own variations and additions.
* All three associations specify that legitimate polling firms open each interview by providing the true name of the firm or the telephone research center conducting the interview. AAPOR members have unofficially observed that the candidate or organization sponsoring the survey need not be identified, the position CMOR officially holds in its literature. All three also observe that practitioners of so-called push polling generally provide no name, or in some cases make up a name.
* The AAPC and AAPOR both observe that in a true opinion survey, research firms interview only a small random sample of the population to be studied, typically ranging from up to 1,000 interviews for a major statewide study to as few as 300 in a congressional district. With so-called push polls, the objective is to reach a high percentage of voters, usually at the close of the election. CMOR has defined AAPC’s “high percentage of voters” as more than 1,000.
* The AAPC and CMOR state that interviews conducted by real polling firms generally range in length from at least five minutes for even the shortest of tracking questionnaires to more than 30 minutes for a major benchmark study. So-called push poll interviews are typically designed to last 30 to 60 seconds by AAPC’s definition and 20 to 30 seconds by CMOR’s.
* AAPOR says that “push polls” are usually political telemarketing presented as if conducting a legitimate public opinion poll. CMOR says that push polls are usually conducted by campaign workers or telemarketers, rather than by research interviewers. AAPOR further delineates push polls from research by observing that data is rarely or never saved or analyzed. Legitimate surveys always store and analyze collected data.
Leading pollsters of both parties and all professional organizations with political pollster membership have condemned the practice of push polling, distinguishing it from opinion research. In the words of Republican pollster Ed Goeas of The Tarrance Group, “When political researchers put a survey into the field, they do so using recognized scientific techniques to find out what the public is thinking or feeling. ‘Push polls’ on the other hand, are meant to inform the electorate with no accountability.”
The practice of push polling violates the code of ethics to which members of AAPC, CMOR and AAPOR agree upon joining. The NCPP has issued a statement to the effect that it does not recognize “push polls” as legitimate research.
Journalists, university professors, employees of research firms, staff of local and statewide candidates and even U.S. representatives and senators have made statements about push polling that reflect a lack of understanding of them. The confusion usually comes from the proper use by campaigns and pollsters of survey questions designed to test negative campaign messages. This practice has been called “push questioning” by Christopher Arterton, dean of George Washington University’s Graduate School of Political Management, and “negative polling” by Bruce Blakeman, former vice president of The Wirthlin Group’s political research division.
“Push questions” are widely used throughout the research industry, whereas push polling is not. Push questions — as opposed to so-called “push polling” — are recognized by all the major associations and leading political consultants as a valid and legitimate research tool for the purposes of testing ad messages and examining the collective viewpoints of electorate subgroups.
Political analyst Charlie Cook recently observed that “‘Push questions’ are asked containing attacks on the candidate sponsoring the poll, to test how vulnerable that candidate may be against anticipated attacks from the other party. These are not only legitimate tools of survey research, but any political pollster who did not use them would be doing their clients a real disservice.”
The problem is that the questions used in push polls often sound similar to those used as push questions to test campaign messages in legitimate polls. This is done intentionally to camouflage the true nature of the push poll. As a result, many respondents who are interviewed, opponents who are attacked or journalists covering the race in question often lump push questions and “push polling” together in the same category. Today, pollsters are often accused of conducting push polling when they are, in fact, conducting legitimate research.
Consequences of “Pushing”
Many researchers and interviewing services don’t have such high-caliber support. Often they are trapped between the need to remain silent to maintain client confidentiality and the need to respond to having their good reputations dragged through the mud. The effect on their staff, operations and other business relationships can be devastating.
In the past four years, the research industry has been hounded in different states by various legislators and civic groups seeking to restrict telephone contacts for both research and political telemarketing. Experiences with push polling are cited as the reason for this legislative hazing. Should any of this legislation ever pass, the mandatory disclosure statements will severely bias the quality of data that can be collected.
The blurring of push polling definitions by media and campaigns also does little to improve today’s declining response rates.
U.S. telephone response rates are dropping to a point that threatens the validity of telephone research. This practice, along with continued “sugging” and “frugging,” accelerates the tend, as Democratic pollster Mark Melman observed in a 1996 interview with John Nielson broadcast on National Public Radio’s “All Things Considered.” Michael Traugott, current president of AAPOR, made a similar statement in the wake of February’s controversy between the McCain and Bush primary campaigns, observing that “[Push polls] breed cynicism about politics, and we believe they contribute to declining response rates for polls.”
But the reality is that “push polling” will continue to be used as a campaign tool. Traugott observes in his new book Elections Polls, the News Media, and Democracy that the use of push polls has spread throughout all levels of U.S. politics in the past six years. Research for Larry Sabato and Glenn Simpson’s Dirty Little Secrets: The Persistence of Corruption in American Politics revealed that 35 of 45 candidates interviewed in 1996 claimed they had been victimized by a covert negative phoning campaign. They also detail numerous cases in each party where the technique has been used by candidates’ campaigns and sometimes by advocacy groups without a candidate’s knowledge or consent.
Push polling has been exported to countries like Australia and Israel. Even U.S. presidential campaigns utilize push polling. As the 2000 campaigns heat up, accusations of push polling are again flying.
So what can pollsters do as an industry to protect themselves from the negative effects of these misunderstandings?
Educating political campaigners and consultants about the differences between push polling and push questions may do little to change the behavior of those conducting push polls. Even experienced campaigners in national races have indicated that they care little about adhering to accepted standards and guidelines set by professional organizations. This is clearly not the full answer to the problem.
Professional associations can be used to adjudicate disputes between researchers, candidates, journalists and respondents. The NCPP has already set up a Polling Review Board to be used expressly for this purpose during the 2000 elections. The board members are recognized as authorities in public opinion research. Harry O’Neill of Roper Starch Worldwide, Humphry Taylor of Harris Interactive and Warren Mitofsky, the grandfather of the Mitofsky-Waksberg sampling method and president of Mitofsky International, are its members. Using this panel to clarify definitions of what is and is not a push poll will provide research firms with a widely recognized, third-party defense against the uninformed and slanderous.
CMOR has established the Caller Hotline Research Information Systems, or “CALL CHRIS,” expressly as a forum for consumers who seek more information about a specific interview or surveys in general. Front-line data collectors can refer respondents who do not understand the difference between push polling and push questions to this interactive voice information system or to CMOR’s Web site at www.CMOR.org, seven days a week, 24 hours a day.
Both information sources will provide further clarification by a third-party source of what push polling is and is not. Wider use of the 800-887-2667 CALL CHRIS number will also serve to increase respondent confidence in the legitimacy of a particular survey, head off potential misinformed complaints and make a small contribution to preventing the further erosion of cooperation rates.
Researchers can support the activities of the various organizations which fight on their behalf in the legislative arena. CMOR and AAPC have led successful campaigns in several state legislatures to amend or defeat draft legislation restricting telephone contacts, legislation proposed because of the past six years of push polling activity. Their efforts have advanced legislation against push polling while protecting research in many states, including Iowa, Indiana, Nevada, California, Florida, Georgia and New York.
Consumers of polling information need to be educated through outreach initiatives. The NCPP has taken a first step in this direction with its recent meeting on “Monitoring Polls & Poll Coverage” to establish how it will pursue this educational objective. AAPOR also has taken a stand.
Special attention should be paid to influencing course work at political management and journalism schools. It would also be useful to provide clear talking points to frontline data collectors so they can respond to poorly informed research consumers in a knowledgeable fashion.
There is already a great deal of literature on the poor quality of most online surveys, mostly because of the samples used. But there are also problems with the way some of these “surveys” are written. They’re not surveys at all, but rather attempts to fundraise or smear opponents, much as push polls do.
The Internet has a bright future in politics. Part of that bright future will include Web-based surveys and, inevitably, the World Wide Web’s own version of push polling.
The recent weekly Web “survey” hosted until the beginning of February 2000 on the site of the Democratic National Committee (DNC) is one such example. It was designed to reach the many thousands who would log on to the DNC site. This fits one part of the classic definition of push polling by reaching many potential voters without discriminating among those sampled. It also asked leading questions with slanted wording. Without knowledge of how the data collected was to be used, it is hard to use this characteristic to qualify it as push polling. But the combination of the two features would suggest that the survey was intended to inform and influence, rather than test messages and create data.
E-mail “surveys” have a greater potential to push the electorate. Mass “spamming” of lists of e-mail addresses with push polls is a possibility, though this author has yet to see evidence of the practice.
Web denizens have developed methods of counterattack not available to telephone respondents. The servers supporting political “spam” can be overwhelmed with a mass of e-mailed responses. Web surveys with unscientific samples can be changed by an organized response of the opposition, as happened with the DNC’s survey. Or, most insidious of all, hackers can reprogram Web sites. This offers the illegitimate opportunity to change the results of any push poll that might exist in cyberspace.
Professional organizations are also changing with the times. The NCPP’s Polling Review Board intentionally included Taylor of Harris Interactive. This gives the board an authority on cyber surveys that will prove useful in adjudicating Web push polls, much as it does telephone pushing. In addition, NCPP and AAPOR’s educational efforts are beginning to include a discussion of this growing medium, as the year 2000 AAPOR conference schedule indicates.
Encouraging the various efforts of AAPOR, AAPC, CMOR and the NCPP will mitigate the effects on researchers of the inevitable push polls of the future. In addition, the more researchers use these various resources, the greater credibility they will amass with consumers of research data. This will make it possible for these associations to be more effective. Researchers should therefore make every effort to protect themselves by using the various avenues laid out here, both for the future of the industry and for their own peace of mind.
Karl Feld manages fielding of political polling projects and conducts “research-on-research” for Western Wets Opinion Research Center (WWORC), a Republican data-collection firm.
COPYRIGHT 2000 Campaigns & Elections, Inc.
COPYRIGHT 2000 Gale Group