News Stay informed about the latest enterprise technology news and product updates.

New surveying methods needed for better U.S. election predictions

The surprise result of the 2016 presidential election has raised a lot of questions about how surveys are done in an era of cellphone-only households.

Harvard political scientist Gary King has two answers for what pollsters might do to improve future U.S. election...

predictions.

One is, "They'll just hope for an election that isn't that close, because it will be much easier," said King, who is also director of Harvard's Institute for Quantitative Social Science.

The other is, they'll have to find a way to solve what King calls the fundamental problem of survey research today: Rising mobile phone usage means pollsters can't survey the way they used to.

Few polls saw Donald Trump winning the U.S. presidential election earlier this month. He had a 15% chance according to The New York Times and a measly 2% chance in The Huffington Post. The Republican came out ahead the next day, amassing a majority of votes in the Electoral College, which chooses the president, and beating Democrat Hillary Clinton -- and that has pollsters querying the effectiveness of time-tested techniques.

Random death

Random sampling is one of them. That's the statistical research practice of singling out a representative group of subjects from a larger population in order to make observations about the larger group.

To survey more people, pollsters could trade items, say, a free-sandwich coupon, for a telephone interview -- plus assurance that the information would be used only for polling.

"A random sample is not a haphazard sample. It's not the usual meaning of the word random," King said. "It means when you select one, everyone has the same exact probability of being selected."

So when trying to predict results of any election, pollsters try to identify likely voters and then ask them how they intend to vote. Years ago, King said, before mobile phones proliferated, that was doable. Pollsters would dial people up at home, or they'd select them from an enumerated list and then call them.

That's not so easy anymore. In a 2015 study, the Centers for Disease Control and Prevention found that nearly half of households rely solely on mobile phones.

"Ninety percent of the time when you find somebody they say, 'I don't feel like talking to you on my cellphone minutes,' and they hang up on you," King said.

Or when the phone rings, they look at the caller ID screen, see an unfamiliar number and not pick up. In swing states especially, it was a good bet the unfamiliar caller ID signified someone hoping to discover out how they might vote.

"I didn't answer a single pollster," said John Elder, founder of Elder Research, a predictive analytics outfit in Virginia, a Republican-Democrat battleground. (Clinton carried the state with nearly 50% of the votes.)

What pollsters do in such an atmosphere, King said, is target certain groups of people -- like who voted in the previous election -- "and then they can try to find people like that group, not a random sample."

Sharpening U.S. election predictions

To get through to those valuable likely voters, researchers will have to find other ways of collecting information, King said. One low-tech way to predict how people might vote is driving around neighborhoods and noting the campaign signs on people's lawns. "Unobtrusive measures" like this were popular decades ago. If a property in, say, 1992 sported a sign for George H.W. Bush, the owners were most likely not Bill Clinton voters.

"That's not great, but it's not a bad idea," King said.

A better one might be designing a mobile app that would collect information about what news people read, what books they bought from online shopping sites, what movies they watched.

"There are all kinds of ways that people are experimenting with, but to do it in real time, for an election or something like that -- they just haven't figured it out yet," King said.

Perhaps that's because pollsters don't hand out swag. Elder said people are willing to exchange information for something in return. Take, for example, a retailer's loyalty card program that lets customers build up points every time they shop.

"I go and use my Panera card, and they know where I was last and what I got last," Elder said, referring to the U.S. bakery and restaurant franchise. "I go three or four times, and I'm going to get a muffin or something."

Pollsters could do something similar -- trade, say, a free-sandwich coupon for a telephone interview. A sweetener would be an assurance that the information would be used for polling and polling alone -- so no passing it on to party headquarters to build on leads for donations.

"I think a lot of people would do that, and then it might give you more reliable information," Elder said.

A really close call

Though the way polls are conducted in the digital era can certainly be improved, King said, he stressed this about the presidential election: It was extraordinarily tight -- and forecasts didn't have either candidate winning in anything close to a landslide.

Most had Clinton ahead of Trump by three or four percentage points, and in the popular vote at least, Clinton is ahead by 2 million votes and counting -- almost two points. Considering the evaporation of random sampling, that's pretty close, King said.

"There's no real reason the polls should have done as well as they did," he said. "It's actually a miracle they did anywhere near as well as they did."

Next Steps

U.S. election predictions cast doubt on data analytics

Experts weigh in on Trump presidency and future of health IT

Opinion: The tech sector helped create President-elect Trump

Dig Deeper on Enterprise business intelligence software and big data

PRO+

Content

Find more PRO+ content and other member only offers, here.

Join the conversation

3 comments

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.

Do you think U.S. election predictions will improve in 2020? Tell us about it.
Cancel
No, unless miraculously the survey companies would be ready to admit that their stat model needs some urgent corrections.
Cancel
I think that the wrongs are not in the sampling method, but rather in the assumptions that are never checked that go into the prediction model, the associations and drivers that underlie the stat model. The learning and validation process is missing in today's practice. In my view, surveys are not just about prediction, they should serve first for learning patterns of behavior. There are a number of techniques that one can use, on the simple side for start. To sum it up: (a) Make sure to include irregular/exception PATTERNS, and that they are multi-variant, (b) Enhance the sampling by importing data from different sources,(c) Keep hands on the pulse (try to find a measure that reflects change, then follow it up), (d) Focus on "error" patterns, where expectations are more likely to fail, (e) Add more variables that measure change, (f) Last but not least, analyze the cost-effectiveness indicators of the campaign including the surveys themselves. I wrote about it in my data-mining forum (in Hebrew, sorry) at http://www.dwh.co.il/forum/4-DataMining/8070-%D7%A9%D7%95%D7%91- - By Home of GT-data-mining.
Cancel

-ADS BY GOOGLE

SearchCompliance

SearchHealthIT

SearchCloudComputing

SearchMobileComputing

SearchDataCenter

Close