News Stay informed about the latest enterprise technology news and product updates.

Presidential election predictions cast doubt on data analysis

Donald Trump's surprise presidential election win has media, pollsters soul-searching on the predictive power of data. Also in Searchlight: Yahoo hack may mean no Verizon sale.

Of the headlines Wednesday blasting news of U.S. presidential elections the day before, this one was striking: "Man Nostalgic For Simpler Era Of 20 Hours Ago."

One reason: It's funny; it's from satirical news outlet The Onion. Another: Implicit in it is the belief -- nearly universal until about 60 hours ago -- that Republican businessman and reality TV star Donald J. Trump would not, could not, win.

Before the votes came in Tuesday night, presidential election predictions bolstered that belief. The New York Times saw Democrat Hillary Clinton with an 85% chance of winning; The Huffington Post had her prospects at 98%. FiveThirtyEight's Nate Silver, who holds a special place in the panoply of prognosticators, pegged them at 71%.

That left Trump with a nearly 30% chance of winning, said Erik Brynjolfsson, a professor at the Sloan School of Management at MIT.

"That's not 0%," Brynjolfsson wrote in an email. "Things with a 30% chance of happening do happen."

Things with even lower chances of happening happen. After the fourth game of the 2016 World Series, for instance, the Chicago Cubs had a 15% chance of winning the title, according to FiveThirtyEight. They pulled it off in Game 7 on Nov. 2, after 108 years.

While the results may not classify scientifically as a shocker -- as Brynjolfsson says, data scientists work with probabilities, not certainties -- they sound a wake-up call for anyone relying on data and for oracle-like predictions about the political or business climate.

Through a glass, darkly

But how, with sophisticated analytics tools and data modeling at people's fingertips, could the voter turnout in favor of Trump not have been predicted as a likelier outcome?

It was a failure on the part of journalists, those responsible for reporting the state of the presidential race to the American public, wrote media columnist Jim Rutenberg in The New York Times. They didn't accurately reflect the outrage of voters who feel disenfranchised seven years after the end of the Great Recession, just as the British media missed signals preceding the vote in the U.K. to leave the European Union earlier this year.

"It was clear that something was fundamentally broken in journalism, which has been unable to keep up with the anti-establishment mood that is turning the world upside down," Rutenberg wrote.

Lots of pollsters took on the blame themselves. The Huffington Post's senior polling editor, Natalie Jackson, said her unwavering trust in polls led her astray.

"I saw no reason to question that the polls would be accurate overall. So I defended and stood by the numbers -- as anyone who trusts their work does," Jackson wrote. "That’s left me eating some crow."

The data modeling used by the HuffPost relied solely on polls, Jackson wrote. It didn't account for economic indicators or the fact that rarely has a political party held the presidency for three consecutive terms.

'Hidden costs'

Analytics can be extremely powerful, said John Elder, founder of predictive analytics outfit Elder Research, but it can falter in an area like politics, where human emotions and values play big. There are also what he calls "hidden costs." Here's how he explained it: Trump voters who felt that the media has a deep-seated bias for liberalism, and thus, the candidate's Democratic opponent, likely didn't want to be judged -- by them or anyone else. So they didn't answer.

"If you're a Trump voter, you knew what the media thought of you," Elder said. "And if you were in a university or in a professional setting, there was a high social cost to identifying as a Trump voter."

Robin Young, host of the National Public Radio program Here and Now, explored whether lying was a factor in the presidential election predictions: Did people give false information about how they voted? She talked to Republican pollster Whit Ayres, president of North Star Research, on WBUR-FM, a Boston NPR station.

"I don't believe that there's a consistent pattern of lying, but I do believe that many of us -- and I include myself in this -- interpreted the polls at the end as leading to a Clinton victory."

Trump, Ayres said, got the right voters in the right places -- the swing states of Florida, Michigan, Pennsylvania and Wisconsin -- giving him a plurality of votes in the Electoral College, which chooses the president and vice president. Clinton won the popular vote.

Good data, good read

Brynjolfsson said the major lesson is this: Data matters, but data quality matters more.

"Think hard about whether the data you are putting into the model is accurate, complete and without systematic flaws," he wrote.

Better data models could help pick up on hard-to-detect signals in future elections, Elder said. For example, getting more particular information about people -- whether they like, say, NASCAR or ballet, or what their hobbies are -- to sniff out how they might vote could increase accuracy. But it won't be easy, especially in an age when the use of landline telephones has declined in favor of mobile phones, and people check the ID of callers before they answer.

"People are on to pollsters," he said. "When I was growing up, the phone rang -- you picked it up. You answered it."

CIO news roundup for week of Nov. 7

The presidential election predictions dominated -- here's what else made headlines.

Yahoo hack could thwart deal with Verizon. In an SEC filing Wednesday, Yahoo revealed that some employees were aware, when it took place in 2014, of the data breach that affected over 500 million users. "The Company had identified that a state-sponsored actor had access to the Company’s network in late 2014," Yahoo representatives wrote in the filing. Yahoo disclosed details of the hack in September, two months after Verizon agreed to acquire Yahoo's core business for $4.83 billion in cash. Following the security incident, "Verizon … may seek to terminate the Stock Purchase Agreement or renegotiate the terms of the Sale transaction," Yahoo warned investors. The hack did not have a "material adverse impact" on Yahoo's business, the company added in the filing.

Forrester forecasts cloud market to reach $236 billion by 2020. In a report highlighting the top 10 trends in cloud computing in 2017, market research firm Forrester predicted that large enterprises will be adopting cloud-based services in a big way next year. The report forecasts that the worldwide public cloud market will reach $146 billion in 2017, and rise to $236 billion in 2020. Cloud service providers will design security into their products and offer more cost saving options for enterprise customers, according to the report. "Cloud computing has been the most exciting and disruptive force in the tech market in the last decade, and it will continue to disrupt traditional computing models at least through 2020," Forrester analyst Charlie Dai wrote in a blog post. Companies like Amazon, Microsoft, IBM, Google, Salesforce, Oracle, CenturyLink and SAP will also help accelerate the adoption of private cloud, Dai wrote.

WhatsApp adds two-step verification. WhatsApp started rolling out a two-factor authentication feature for its beta users on Android and Windows Thursday. The optional feature was added to bolster users' account security, according to the company. Users are required to enter a six-digit passcode, which they will be asked to confirm each time they try to verify their phone number with WhatsApp. "If you have two-step verification enabled, your number will not be permitted to reverify on WhatsApp within 7 days of last using WhatsApp without your passcode," the company stated on its FAQ page. Providing a valid email address during sign-up allows WhatsApp to send users a link to disable the two-step verification in case users forget their passcode, according to the FAQ page.

Assistant editor Mekhala Roy contributed to this week's news roundup.

Next Steps

Check out our previous Searchlight roundups on Microsoft Teams work tool,  the software giant's foray into immersive computing and Gartner's recent CIO gathering in Florida.

Dig Deeper on Enterprise business intelligence software and big data