agsandrew - Fotolia
The International Institute for Analytics released its annual list of predictions and priorities. Their big prediction for 2018: Companies will experience the pros and cons of what they dubbed the age of the algorithm.
Tom Davenport, co-founder of IIA, fellow at the MIT Initiative on the Digital Economy and the president's distinguished professor of information technology and management at Babson College, and Bill Franks, chief analytics officer at IIA, said that algorithms are nearly ubiquitous and will proliferate in the enterprise.
The good news is that the age of the algorithm will mark a level of analytics maturity for the enterprise: Algorithms will be easier to access, easier to use, and even self-learning, making it possible for companies to interrogate and take advantage of their data like never before. The bad news is algorithm ubiquity will also lead to complexity, impacting technology buying decisions, enterprise architecture strategies and even how job titles are perceived.
IIA's 2018 predictions were accompanied by corresponding priorities, providing companies with advice on what analytics hurdles to expect in the age of the algorithm and how they can be avoided.
Prediction No. 1: The age of the algorithm arrives
Algorithms have become ubiquitous and will play a more prominent role in day-to-day corporate activities next year, according to Franks. They are now easy to deploy, easy to access, embedded in applications and can be rented in the cloud, eradicating the careful choices analytics experts used to make about which algorithms matter and how to expend precious processing power. "Today, we're basically able to go ahead, tee up and test a whole range of algorithms and then pick the best," he said.
Priority: Automate algorithmic testing
Data scientists should embrace tools that automate the testing process as a timesaver. Franks was quick to point out that using these tools, which companies can build or buy, doesn't negate the need for data scientists. Problems still need to be well defined, data still needs to be prepared and results of the testing process still need to be interpreted.
Prediction No. 2: AI projects grow, but disillusionment rises
Franks and Davenport disagreed on the latter part of this prediction. Davenport said almost every company he talks to has a portfolio of concrete AI projects underway; he believes companies have moved away from "moon shots" to smaller, easier-to-get-at problems, and that they tend to be bullish on the technology. But Franks believes enterprise AI in 2018 will follow a pattern similar to big data, which started out hot and then fizzled before finding its footing. He sees the lack of AI skills as the biggest contributor to enterprise disillusionment with AI. The talent dearth is especially acute for the more complex forms of AI, such as deep learning. That prowess is even scarcer than data science skills. "As demand rises, that's going to be a challenge," he said.
Priority: Incorporate AI into your plans -- rationally and incrementally
The key is to be strategic in how AI is incorporated into the analytics roadmap by making it an extension of what the business is already doing with analytics. "About 90% of AI has a statistical underpinning," Davenport said, which means the foundation for AI isn't new to data science departments. The roadmap should include top priorities, including AI, according to Franks. Doing so will give the company a contextual understanding of how aggressive it needs to be with artificial intelligence.
Prediction No. 3: Hybrid analytics
The proliferation of analytics products has given rise to freedom of choice -- open source or proprietary tools, on premises or the cloud, or, increasingly, a mix of both. But with added choice comes added complexity, Franks said.
Priority: Allocate resources to determine the right technology mix
Companies will need to think hybrid analytics if they want to remain competitive, according to Franks. In 2018, analytics professionals should start to consider not only what a tool does, but where it best performs and how well it integrates with other technology. And in partnership with IT, they should start building flexible analytical architectures, making it easy to snap in or get rid of tools. Added flexibility will ensure a level of resilience if startups go under, if more established vendors stop supporting a product or if the analytics team makes a bad investment. "The architecture can be an insurance policy," Franks said.
Prediction No 4: Beware the so-called data scientist
The title of data scientist has become watered down. Reporting tools and data preparation tools often have models baked in, making them easier to use and accessible, according to Franks. "People who traditionally would not have had any ability to claim being an analytic professional/data scientist suddenly, in theory, can say I use tools that do this so, therefore, I'm going to put that on my card," he said.
Priority: Inventory your analytics skills and define job titles
Davenport suggested companies collaborate with HR to inventory and classify the analytics, data science and AI skills they have in-house. Doing so will help to distribute talent appropriately and pinpoint potential skills gaps. An example of classification is to implement a categorization and certification process for data scientists, with junior data scientists taking on simpler tasks such as regression analysis and senior data scientists taking on complex tasks such as developing new algorithms.
Prediction No. 5: Blockchain is a roadblock for analytics
Blockchain, an immutable distributed ledger, won't have a profound impact on enterprise analytics programs, according to Franks. But it will pose performance issues when analyzing transactional data. Data on the blockchain is distributed and not centralized, it's repetitive and compressed, and, unlike SQL databases, it isn't designed for analytics. Data scientists will have to extract data out of the blockchain format and "make it into something friendly," Franks said.
Priority: Prepare to analyze blockchain data
Franks said understanding how blockchain works and the new issues the technology presents is a good first step. He suspects that, at least in the short term, analytics professionals won't have a lot of direct interaction with blockchain data. Instead, they will build "enterprise analytic views" into the data to do the analysis. "Those views can then be updated on a somewhat frequent basis and made into an extract or it can run live," he said.
Prediction No. 6: Analytics are widely applied to improve data
When Davenport mentioned ditching moon shots for easy-to-get-at AI projects, one example he had in mind was applying AI to the data preparation process. For some companies Davenport has talked to, machine learning algorithms took months and sometimes just weeks to find duplicated records in multiple databases -- a project that would have taken humans years to complete. "We hear a lot about what analytics and AI can do, but you don't hear so much about this one, even though it may be one of the quickest ways to value for a lot of organizations," he said.
Priority: Modernize your organization's data process
Davenport likened the process to a 12-step program. "Step one is to admit that you have a problem," he said, which is easier said than done. Next, he recommended companies find data that's important but problematic to the business. "Often that's customer data or supplier data," Davenport said. Then begin using technology that can help with the data prep process, such as probabilistic matching tools. GE is a powerful case study, for anyone interested.