Calculating ROI on IT projects is useful but not sufficient

Calculating ROI on IT projects is necessary and useful. ROI is only as good as the assumptions that go into it, however -- including a company's track record on project delivery.

The formula for calculating ROI is straightforward, and ROI is a tried and true metric. Nevertheless, CIOs using...

ROI to gauge the value of an important IT project might want to keep in mind that numbers do lie -- or at least they don't tell you everything you need to know.

"Just because you see a deal with a high ROI doesn't mean you are getting a lot of money back or that it is a better deal than a project with a lower ROI," said Glenn Clowney, director of technology assessment for the Case Study Forum and president of ROI-Calc Inc., which makes Web-based applications for showing ROI.

Clowney was speaking to an audience of CIOs at the annual user conference of Information Builders Inc., the business intelligence provider. To drive home the point, he analyzed the ROI realized by several customer companies on their respective investments in ROI-Calc's WebFocus software. Among the companies: a large relocation services company showing an ROI of 196%, a major retailer/manufacturer with an ROI of 415% and a large cargo company with an ROI of 424%.

At face value, the cargo company with the 424% ROI seems to come out the winner. ROI, however, needs to be looked at in the context of other factors, Clowney explained, including:

  • The size of the initial investment
  • The time it takes to break even, or payback period
  • The time required to produce the return

Net present value (NPV), which measures the net benefit of a project in today's dollar terms, is a critical metric.

All three companies using WebFocus boasted fantastic ROI (it was a vendor conference after all), but the value of each project to the company did not correlate exactly with the ROI yield. The major retailer/manufacturer with a 415% ROI broke even in two months and realized a whopping net benefit of $8.1 million in three years. The relocation services company with the 196% ROI realized a net benefit of $3.9 million in 14 months. And the cargo company? It broke even in six months and realized a net benefit of $3.7 million in five years.

"Investors want your payback period to be short, your ROI to be high and a net present value of greater than zero," Clowney said.

That is, investors usually do, said Craig Symons, principal analyst at Cambridge, Mass.-based Forrester Research Inc. and an expert on deriving business value from IT investments.

"When times are good, and the economy is growing and your business is growing, what might be more important than payback period is the size of the return and maximizing that return regardless," Symons said. "When the economy is in a tougher situation and money is constrained, you might be looking for the quick wins: Where can I invest a little and get something back quickly?"

Financial metrics mean nothing if you don't keep score

Estimating the ROI of a project always begins with a deep understanding of the status quo, or "as is" state, Clowney said. The next step in the business case is analyzing what the investment will do for the organization over time -- and that can be tough.

"If you don't ask the 'why' question at least four or five times, you won't get to the underlying reason for the investment," Clowney said.

Business cases often miscalculate the TCO, because people do not think through the costs. A simple example is the printer that sells for $70 on sale and will end up costing more like $600 over a period of three years, when toner, paper, trips to the store and so on are factored in.

You can come up with all the financial justifications you want ... but what is important is that you actually keep score.

Craig Symons, principal analyst, Forrester Research Inc.

Organizations also tend to overlook the cost of the changes that a big project, such as an ERP implementation, will exact on an organization's systems, employees, vendors, partners and education programs, Clowney said. Employee turnover in the wake of a change -- or as a result of not making a change -- can be very expensive, as can such "hidden costs" as repair, documentation, buying spare parts and phasing out the old system.

"At the end of the day, ROI is only as good as the assumptions that go into go into it," Forrester's Symons said. One of the biggest omissions he said he sees IT groups make when they calculate ROI is that they fail to adjust for risk, including their own ability to execute on a project. Track records are ignored, he added, despite the fact that most IT organizations have a "pretty good history" of their success rates. Research shows that 30% to 70% of IT projects do not deliver on whatever set of metrics IT chooses to use, he said.

"Yet, I can guarantee that every time they build out a business case, that business case is assuming that the project execution is going to be flawless," Symons said.

Having a project management office, or PMO, or processes in place that keep track of the IT department's history on projects, as well as the benefits those projects have brought to the business, is ultimately more important than numbers.

"You come up with all the financial justifications you want, whether it is ROI or NPV or internal rate of return -- pick your metric -- but what is important is that you actually keep score," Symons said.

Let us know what you think about the story; email Linda Tucci, Senior News Writer.

Dig Deeper on Return on Investment (ROI)

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.