360 Guide:

AI technologies affect all corners of business, IT


News Stay informed about the latest enterprise technology news and product updates.

Measuring ROI for AI investments? Put on your venture capitalist hat

For all the intense interest in AI investments, the ROI for AI is not well-understood. Rather than make bad guesses, CIOs should treat AI projects like venture capital investments.

Artificial intelligence refers to everything from robotic process automation software to actual metal-legged robots,...

but the technologies under AI's big umbrella are not all equal. The scope, application and level of maturity of the various AI technologies can differ wildly, making it impossible to generalize how to measure return on investment.

AI technologies aren't typical tech investments in another way: They hinge on data to drive the algorithms that help these intelligent machines solve problems. Finding the right blend of data can be a challenge, adding cost to an AI project. Indeed, experts suggest that rather than trying to calculate the payoff of AI investments, CIOs should adopt a venture-capitalist mindset and prepare themselves for a lot of failure and a little bit of success.

"When you think of databases, that's a much more mature technology. There is a ton of ROI-types of data that's available with respect to, 'What does that investment look like?' and 'How do I get that return?'" said Bharath Kadaba, vice president and chief innovation officer at Intuit Inc., a business and financial software company in Mountain View, Calif. "Whereas the AI and machine learning world is relatively less mature: You can fall into signing up to solve a problem that's hard to solve, and you can put money behind it and you may not get any clear results."

Think R&D, develop projects in parallel

Interest in AI abounds, but actual deployments of the technology are still relatively few. In a recent Forrester Research survey, 58% of business and technology professionals said they're researching AI, but only 12% said they are currently using AI systems.

Mike Gualtieri, ForresterMike Gualtieri

The limited use of the technology provides an opening for CIOs to create competitive advantage and even carve out new revenue streams. "Until artificial intelligence in applications becomes the norm, you have an opportunity to get out in front of the trend and use AI to facilitate more efficient business processes and, of course, better, more individualized customer experiences," Mike Gualtieri, an analyst at Forrester Research, wrote in "Artificial Intelligence: What's Possible For Enterprises in 2017."

And AI projects don't have to require a lot of talent or a lot of money. "Cloud companies -- Amazon, Google, Microsoft, IBM, HP [Hewlett Packard Enterprise] and their cloud -- they have what's called pretrained models," Gualtieri said in an interview with SearchCIO. "So, they have AI sort of baked into the APIs already."

But AI investments do hinge on data models that have to be tuned, and that tuning takes time and often requires additional and/or different data to get a result, Gualtieri said. "You may see a business benefit, but it's not until you actually work with the data and use the algorithms that you'll know if it will work."

A more realistic way to look at AI investments, according to Gualtieri, is to see them as research and development (R&D) projects. "When you're doing this stuff, you have to think like a venture capitalist," he said. "Venture capitalists will invest in 10, 12 companies. They believe they're all going to be successful, but statistically, they know that only two or three are going to be fabulously successful."

By treating AI technologies as R&D efforts -- including those AI investments with relatively proven technology -- CIOs will have the time to experiment and the space to develop multiple projects in parallel, which Gualtieri said is key. "Otherwise, if it's sequential, if you just have this one idea and you spend three months and it doesn't work, well, now, you need to find the next idea, which can put you behind the eight ball," he said.

Indeed, Gualtieri said bigger companies are making use of their innovation labs, which are funded for the express purpose to experiment without proving an ROI, to oversee and pursue their AI investments.

AI investment advice

CIOs may want to turn their attention to Amazon Lex as a point of entry for AI investments. The Amazon service, powered by the same conversational engine as the company's Alexa bot, introduced a chatbot interface API last fall. The service makes building an application on the Amazon Echo platform fairly straightforward for application developers and designers, according to Mike Gualtieri, an analyst at Forrester Research. "Amazon has created an API that lets companies create an experience that's specific to their business domain," he said. It's not good for everything, he warned, but no AI experience is needed.

Innovation labs push the envelope

Intuit's Kadaba heads up an innovation team that experiments with cutting-edge technologies, including AI, and develops projects that give the software company a competitive edge.

Not all of the company's R&D efforts in AI are under his purview, Kadaba explained. Many projects are spread across the business units where off-the-shelf machine learning and AI technologies can be tested and where the outcomes for tech investments are clearer. His team comprises a group of inventors who tackle "the hard problems where the ROI is not clear," he said.

Bharath Kadaba, Intuit Inc.Bharath Kadaba

Kadaba, who has a doctorate in computer science and a career that includes a mix of emerging tech and operations roles, leads a team of 85 engineers, data scientists, designers, product managers and program managers. "We take what I call an outside-in view and then decide on a collection of projects to invest in that are risky, exploratory, but have high-impact potential," he said.

Take one of the newest features in TurboTax, which served 34.2 million customers in 2017 as of April: The ExplainWhy button, introduced in 2016, gives customers a highly personalized, natural language explanation of deductions and information on how a return was calculated. Kadaba said developing the button took years and likened it to a Google search problem, because "every customer's tax situation is different," he said.

That button uses a customer's personal data, but it also relies on anonymized customer data and machine learning to construct "knowledge graphs," a Google term that refers to a back-end map of the relationships between data points. "Google builds a knowledge graph in terms of how the world works," Kadaba said. "We were able to do the same in the world of tax -- that's the way to think about it."

The knowledge graph is what enables the software to provide a personalized explanation of the software's accounting to customers. And, according to Kadaba, it provides a differentiated product experience to the market. The work has since been patented by Intuit.

"In the innovation world, we have to have the confidence that we have the right people who can crack the code," Kadaba said. "And when they do crack it, the advantage that we get is a step-function improvement to what is it that we're able to deliver."

Boards are already on board

Graeme Thompson, CIO at Informatica Corp., a software development company in Redwood City, Calif., said an ROI measurement is, in general, becoming outdated. That is especially the case for AI investments.

"If you're trying to theorize what's going to happen 24 months from now, you're kidding yourself," he said. "We have to do things in shorter, discrete bursts of effort where you cannot necessarily measure the return, but you can figure out whether your hypothesis was correct."

Graeme Thompson, InformaticaGraeme Thompson

Experimentation -- not revenue projections -- is key to the success of AI projects, Thompson said. With untested technologies, it's critical that the IT department be able to quickly change course when necessary. Experienced members of the C-suite and boards of directors already get that, he said. They lived through the big ERP deployments and watched as the vaunted ROI of the technology tapered off quickly.

"Whereas if you can use AI and machine learning to provide a better customer experience, that benefit lasts forever," Thompson said. "And most folks understand that."

He recounted a memorable moment with his chief financial officer regarding an Oracle investment. "The comment came back from the CFO, 'You guys are great at these ROI calculations, but the only part that's right is the 'I.' You pretty much know how much it's going to cost, but you've got no idea how to quantify the benefit. So, let's just stop pretending that we can, and let's figure out how to do these short experiments instead,'" Thompson said.

Next Steps

AI implementation impediments? No problem

From big data to AI: How analytics have changed

Machine learning in the ER

Dig Deeper on Enterprise business intelligence software and big data

360 Guide: AI technologies affect all corners of business, IT

Article5 of 8

Up Next