What is a better measure of the race between Barack Obama and John McCain: the margin separating the candidates or Obama's percentage of the vote?
That is a question that pollsters are asking each other, and one that will be fundamental for anyone interpreting the presidential horse race this fall.
As Election Day nears, pollsters always face the challenge of interpreting the percentage of people who say they are "undecided" on vote preference questions, but Obama's race will add an extra element of uncertainty this year.
The reason is the so-called Bradley-Wilder effect, named after a series of high-profile races between black Democrats and white Republicans in the 1980s and early 1990s (including the historic candidacy of Tom Bradley for California governor in 1982), in which public opinion polls tended to understate support for the white candidate.
As documented [PDF] in 1993 by Larry Hugick of Princeton Survey Research, the share of the vote won by the black candidates in these races was never more than 1 percentage point greater than their share of the vote in the final poll. Meanwhile, the white candidates consistently outperformed their poll percentages by 4 to 17 percentage points.
"Only black candidates who broke the 50% level in the final poll were victorious," Hugick wrote 15 years ago. To "improve the accuracy of pre-election polls in biracial elections," he suggested a new method: "[Assign] all the black undecided vote to the black candidate and all the remaining undecided vote to the white candidate."
In recent years, however, that pattern has not held. As reported by Scott Keeter and Nilanthi Samaranayake of the Pew Research Center last year, polls in five biracial contests in 2006 were largely accurate. The margins between the candidates predicted the vote, with no evidence of hidden support for the white candidates. "The accuracy of the polling in these five biracial elections," they wrote, "suggests that the problems that bedeviled polling in the 1980s and early 1990s may no longer be so serious."
Also, this year's primary results did not systematically understate support for Hillary Rodham Clinton. If anything, polls tended to underestimate support for the winner in each state, a trend that worked in Obama's favor as often as Clinton's.
But will the general election polling this year be different?
I wondered what pollsters are doing to answer that question, so I e-mailed a handful to ask. The most interesting comments came from Scott Keeter.
I give extra weight to the "evidence based" approach the Pew Research Center uses to allocate undecided voters because it proved accurate four years ago. That approach, as Pew President Andrew Kohut put it in an interview with me in 2006, relies on "measures" of the attitudes of undecided voters, not "upon 'you know I think,' 'I got this feeling,' 'history tells us,' or any of this other stuff where you can let judgments get in your way."
So in addition to repeating their assessment of undecided voters, what steps will the Pew Center take to monitor for a recurrence of the Bradley-Wilder effect?
First, says Keeter, they will continue to "measure the impact of racial issues on voter judgments," as they did in this March report. If they find that racial attitudes correlate with vote preference among decided voters, they may incorporate those attitudes into the model they use to allocate the undecided.
Second, they will watch for "race of interviewer effects." Pollsters concluded that racial attitudes were at work in the Bradley-Wilder effect when they observed that white respondents tended to report different vote preferences depending on whether the interviewer was black or white.
The best known study of this phenomenon -- a 1989 Public Opinion Quarterly article [PDF] based on surveys that Keeter conducted of the Virginia governor's race -- closed with an "unambiguous recommendation: survey organizations must record the race of interviewers and check for these effects whenever they conduct polls in black-white electoral contests." Smart pollsters -- like those at the Pew Center and CBS News -- will be following that advice this fall.
Third, according to Keeter, Pew will keep an eye on the respondents who are hardest to interview. In 1997, a Pew study found that hard-to-interview respondents were less sympathetic to blacks on questions about race -- suggesting that the Bradley-Wilder effect may have been partly due to those who refused to participate in surveys. They could not replicate the same results in a comparable study in 2003 but will be watching hard-to-interview respondents again this fall.
So what's the bottom line? Speculation about Bradley-Wilder is inevitable in the coming months, but the most worthy approach will be based on evidence, not hunches.
