NCPA - National Center for Policy Analysis

Question Phrasing Can Affect Poll Results

February 8, 1999

Some professional pollsters are pointing out the pitfalls of their trade. They caution that how a poll question is worded often determines the numerical outcome.

A recent survey conducted by the National Law Journal asked "potential jurors" to agree or disagree with the statement: "Whatever a judge says the law is, jurors should do what they think is the right thing."

Analysts had these criticisms of the question:

  • Experts note that on issues of relatively low salience, respondents have a strong tendency to agree with a reasonable sounding statement.
  • When asked to agree or disagree with complex statements, respondents tend to focus on the last clause of the statement -- which would encourage them to look favorably on "doing the right thing."
  • Any statement which begins with the word "whatever" tips the respondent off that the "correct" response is to disregard the clause that follows "whatever" and agree with the second half of the statement.

Critics say that explains why three-quarters of those who responded to the question said they would ignore the judge's instructions if they didn't agree with them.

Yet when the question was phrased as: "Regardless of how they personally view the case they are hearing, jurors should always follow the instructions of the judge concerning the law in the case," 74 percent agreed -- a complete flip from the earlier response.

Poll experts advise readers to ask the following questions: "Who are those being sampled? What is the question format? How is the question worded? How large is the sample? Who commissioned the survey?

Source: Greg Schneiders and Jo Ellen Livingston (both of Frederick Schneiders Research), "Can You Trust the Polls? Well, Sometimes," Wall Street Journal, February 8, 1999.


Browse more articles on Government Issues