Harvard political scientist Gary King has two answers for what pollsters might do to improve future U.S. election predictions.
One is, "They'll just hope for an election that isn't that close, because it will be much easier," said King, who is also director of Harvard's Institute for Quantitative Social Science.
The other is, they'll have to find a way to solve what King calls the fundamental problem of survey research today: Rising mobile phone usage means pollsters can't survey the way they used to.
Few polls saw Donald Trump winning the U.S. presidential election earlier this month. He had a 15% chance according to The New York Times and a measly 2% chance in The Huffington Post. The Republican came out ahead the next day, amassing a majority of votes in the Electoral College, which chooses the president, and beating Democrat Hillary Clinton -- and that has pollsters querying the effectiveness of time-tested techniques.
Random sampling is one of them. That's the statistical research practice of singling out a representative group of subjects from a larger population in order to make observations about the larger group.
"A random sample is not a haphazard sample. It's not the usual meaning of the word random," King said. "It means when you select one, everyone has the same exact probability of being selected."
So when trying to predict results of any election, pollsters try to identify likely voters and then ask them how they intend to vote. Years ago, King said, before mobile phones proliferated, that was doable. Pollsters would dial people up at home, or they'd select them from an enumerated list and then call them.
That's not so easy anymore. In a 2015 study, the Centers for Disease Control and Prevention found that nearly half of households rely solely on mobile phones.
"Ninety percent of the time when you find somebody they say, 'I don't feel like talking to you on my cellphone minutes,' and they hang up on you," King said.
Or when the phone rings, they look at the caller ID screen, see an unfamiliar number and not pick up. In swing states especially, it was a good bet the unfamiliar caller ID signified someone hoping to discover out how they might vote.
"I didn't answer a single pollster," said John Elder, founder of Elder Research, a predictive analytics outfit in Virginia, a Republican-Democrat battleground. (Clinton carried the state with nearly 50% of the votes.)
What pollsters do in such an atmosphere, King said, is target certain groups of people -- like who voted in the previous election -- "and then they can try to find people like that group, not a random sample."
Sharpening U.S. election predictions
To get through to those valuable likely voters, researchers will have to find other ways of collecting information, King said. One low-tech way to predict how people might vote is driving around neighborhoods and noting the campaign signs on people's lawns. "Unobtrusive measures" like this were popular decades ago. If a property in, say, 1992 sported a sign for George H.W. Bush, the owners were most likely not Bill Clinton voters.
"That's not great, but it's not a bad idea," King said.
A better one might be designing a mobile app that would collect information about what news people read, what books they bought from online shopping sites, what movies they watched.
"There are all kinds of ways that people are experimenting with, but to do it in real time, for an election or something like that -- they just haven't figured it out yet," King said.
Perhaps that's because pollsters don't hand out swag. Elder said people are willing to exchange information for something in return. Take, for example, a retailer's loyalty card program that lets customers build up points every time they shop.
"I go and use my Panera card, and they know where I was last and what I got last," Elder said, referring to the U.S. bakery and restaurant franchise. "I go three or four times, and I'm going to get a muffin or something."
Pollsters could do something similar -- trade, say, a free-sandwich coupon for a telephone interview. A sweetener would be an assurance that the information would be used for polling and polling alone -- so no passing it on to party headquarters to build on leads for donations.
"I think a lot of people would do that, and then it might give you more reliable information," Elder said.
A really close call
Though the way polls are conducted in the digital era can certainly be improved, King said, he stressed this about the presidential election: It was extraordinarily tight -- and forecasts didn't have either candidate winning in anything close to a landslide.
Most had Clinton ahead of Trump by three or four percentage points, and in the popular vote at least, Clinton is ahead by 2 million votes and counting -- almost two points. Considering the evaporation of random sampling, that's pretty close, King said.
"There's no real reason the polls should have done as well as they did," he said. "It's actually a miracle they did anywhere near as well as they did."
U.S. election predictions cast doubt on data analytics
Experts weigh in on Trump presidency and future of health IT
Opinion: The tech sector helped create President-elect Trump