According to recent North Carolina polls, Republican Mitt Romney is:
Winning 30 percent of the African-American vote in opening up a 10-point lead over President Barack Obama;
Getting just 8 percent of the black vote and trailing Obama;
Leading Obama by a whopping 18 percentage points. Or not.
They’re all part of the barrage of poll numbers – some accurate, some not – that are attempting to keep score on the 2012 election. They’re as much a part of modern politics as TV ads and bumper stickers.
But not all polls are created equal.
Over the last decade, technology has fueled an explosion of surveys. While budget constraints have forced many news organizations to forgo regular polling, other groups have jumped in.
“There’s a void that’s being filled by partisan pollsters or ideological groups, new companies and academics,” says Charles Franklin, director of the Marquette Law School Poll and a founder of Pollster.com.
Democratic- and Republican-leaning firms, conservative groups and liberal ones all rely on scientific methods to sample public opinion. But along with the science can come a heavy dose of assumptions, bad methodology or even bias.
“Polls aren’t just a snapshot of public opinion anymore, they’re advocacy or even propaganda and often come with an agenda,” says Chris Fitzsimon, executive director of the liberal N.C. Policy Watch.
Exhibit A for Fitzsimon: Last week’s Civitas Institute Flash Poll.
The John William Pope Civitas Institute, bankrolled largely by conservative Republican activist Art Pope, published a poll that showed Romney leading Obama 53 percent to 43 percent in North Carolina.
“Our regular polling will show whether this Flash Poll is an outlier or a harbinger of a new trend in voter sentiment,” Civitas President Francis De Luca said in a Sept. 10 statement accompanying the poll.
But the poll, by the generally respected SurveyUSA, found Romney winning support from 30 percent of black voters. It also found him getting 60 percent of voters ages 18-34.
For many experts, that strains credulity. 2008 exit polls in North Carolina showed Obama winning 95 percent of black voters and 74 percent of voters ages 18-29.
“There isn’t any way in any state that Mitt Romney is going to have 30 percent of the black vote,” says veteran Democratic pollster Peter Hart. “It would be impossible.”
De Luca acknowledges that raised a red flag.
“I doubted that result was completely accurate, but I thought overall the poll had relevant information in it,” he says. “I could have been more direct and said ‘I don’t necessarily believe these results,’ but I tried to do it in a diplomatic way.
“If I had to do it over, I probably wouldn’t have released this poll because it allows people to try to taint all of our polls. If you do as many polls as we do, some of them are going to be bad.”
He differentiates between the presidential “flash” polls, which he argues are unbiased, and the group’s monthly surveys on state politics and issues.
“I will acknowledge that we ask questions with a bent on our monthly polling,” he says. “We’re trying to find out when politicians frame things in certain ways how they will be perceived by the general public.”
While De Luca stands by the overall accuracy of Civitas polls, Fitzsimon argues that even flawed surveys can influence an election.
“Poll results can actually influence the race by galvanizing supporters or discouraging a candidate’s base,” he wrote in response to the flash poll. “They can take on a life of their own by shaping the conventional wisdom that becomes the dominant media narrative in a race.
“They can affect campaign contributions, as many people want to curry favor with a clear front-runner.”
‘House effects’ bias
Dean Debnam of Raleigh is a Democratic donor. And he’s on the board of Progress Action, a nonprofit associated with a group attacking GOP gubernatorial candidate Pat McCrory.
Debnam also is CEO of Public Policy Polling, a company that started in 2001 and now polls for clients nationwide.
“We have a pretty strong record for being unbiased in North Carolina and across the country,” says polling director Tom Jensen.
PPP’s most recent presidential poll showed Obama up a point in North Carolina, with overwhelming support from African-Americans and a strong majority of young voters.
The firm generally gets high marks for accuracy. Even the Wall Street Journal ranked PPP, along with SurveyUSA, one of the most accurate swing state pollsters of 2008.
High marks also have come from Nate Silver, who analyzes data on his “FiveThirtyEight” blog for the New York Times.
But in June, Silver cited PPP – with Pew Research, Ipsos and NBC/Marist – as having one of the more pronounced “house effects” for Democrats. That is, they tend to show better numbers for Obama than Romney.
Among the pollsters with a Republican house effect, Silver found, were Rasmussen, Quinnipiac, Fox News, the Washington Post and Gallup.
“What Nate determined was that our polls this year have had more Democratic-leaning results than other companies,” says Jensen.
“But that doesn’t mean they’re wrong. There’s no way to know that until we actually have the election.”
Jensen believes some pollsters may be under-weighing minorities and young voters. The proof will come in November.
“It’s going to turn out they were wrong and we were right,” he says, “ because they didn’t have the breakdown of the electorate right.”
Skeptical of robo-calls
Assumptions like turnout color poll results as much as the poll questions themselves. Are black voters more excited than white voters? Are young voters tuned out and likely to stay home?
Then there are the basics.
A poll last month by Descartes Research Group of Washington showed Romney leading Obama by 56 percent to 38 percent in North Carolina, a much wider swing than any other poll.
But, according to the Fayetteville Observer, the poll sampled Democrats and Republicans almost evenly, even though Democrats make up 43 percent of the state’s electorate compared with 31 percent for Republicans.
That poll, like PPP polls and the one for Civitas, use a technology called “interactive voice response” or, more commonly, robo-calling.
“They’re bogus because you don’t know if the person responding bears any relationship to the person you’re (calling),” says Kathleen Hall Jamieson, dean of the University of Pennsylvania’s Annenberg School for Communication. “You may be talking to the baby sitter.”
By law, companies that poll with the method aren’t able to dial up cellphones. And more than 30 percent of American homes have only cellphone service, not land lines.
Still, robo polls, which cost less than live-interview surveys, are increasingly common.
An analysis of more than 500 state polls in the last three weeks of the 2008 campaign by the National Council of Public Polls showed robo polls made up 28 percent of the surveys.
Internet polls made up an additional 18 percent.
“There are reasons to be skeptical of automated polls,” says Mark Blumenthal, senior polling editor of the Huffington Post and founding editor of Pollster.com. But, he adds, they also seem to have advantages.
“When you take an interviewer out of the equation,” Blumenthal says, “you may be getting a more honest answer about who the respondent’s going to vote for and whether they’re really going to vote.”
The answer: Take all polls with a grain of salt.
“Don’t look at one poll,” Blumenthal says. “Look at many. Look at averages.”