News Stay informed about the latest enterprise technology news and product updates.

Agile business intelligence is still a work in progress for most CIOs

Petabyte data warehouse? Check! Scalable to thousands of users? Check! Every business intelligence (BI) feature imaginable? Check! Agile BI?

“One thing we do not yet know how to do well is agility,” Boris Evelson told me at the recent Forrester IT Forum in Las Vegas.

Evelson, a principal analyst at the Cambridge, Mass.-based Forrester Research, has been covering BI for some 30 years. Over that time, scalable, powerful, stable BI has become a reality at companies with enough money and know-how. “I don’t want to say it is a commodity, but we know how to do that,” he said.

On the other hand, agile business intelligence — the ability to react faster to the ever-increasing speed of business change — remains “challenge No. 1,” Evelson said. It’s the subject of every conversation he has with clients these days, he told me, and it was the centerpiece of his talks at the conference.

Guidelines for an agile organizational structure

One of the reasons agile business intelligence remains elusive for most CIOs, Evelson said, is that BI software is different from almost any other enterprise application. With ERP or CRM, for example, once the requirements are defined and the software either procured or developed in-house, IT can expect a shelf life for that software of 12 to 36 months, with minor modifications. “With BI, if you do that, when you roll out the first iteration, it is already too late. The world changes way too fast,” he said.

Given that CIOs can’t do much about the pace of change, how do they get to agile business intelligence? In Evelson’s view, it’s a combination of using an agile software development methodology — which relies on prototyping rather than specifications — and on an agile organizational structure. Not that either is easy to do, especially the organizational-structure part. CIOs and their BI experts understand that silos are bad for BI, he said. But so is centralization, because “shared services are anything but agile.” What’s needed is a middle ground. Not middle as in wishy-washy, but as in a nuanced set of guidelines for handling BI. That requires a hard-nosed discussion about which apps need to be in a central area (mission-critical ones, for example) and which nice-to-have, ad hoc apps should stay where they are.

In-memory analytics, mobility

There are also plenty of technologies that can make a BI environment more agile. Forrester has a list of about 20, Evelson said, from cloud and mobility for BI infrastructure and delivery to inverted indexes and in-memory analytics, an approach he believes is suitable for as much as 90% of BI efforts.

“Think of in-memory as Excel on steroids. It has all the flexibility of Excel but also the power of traditional BI tools, like virtualization,” Evelson said. QlikTech International’s QlikView and Microsoft’s PowerPivot take an in-memory approach to BI.

Of course, the flexibility these tools provide also represents a “huge danger,” Evelson hastens to add. IT cannot control what users do in Excel, and the same is true for in-memory tools: One person’s analysis of customer profitability is not going to be the same as another’s. “You have to be smart. If it is a mission-critical app where nothing less than 100% accuracy is good enough, then in-memory analytics is not the right choice.”

Not surprisingly, at companies where there is more business ownership of BI, in-memory analytics are being adopted “left and right,” Evelson said. At IT-centric companies, not so much, he said.

The quick fix

Technologies help facilitate agile business intelligence, but for CIOs, finding the organizational structures and methodology is the tough nut to crack, in Evelson’s view. In the meantime? Users will gravitate toward instant gratification.

“Traditional BI is why spreadsheets are still the most ubiquitous, best BI tool out there. You have a question, and as long as you have spreadsheets, you can get your answers,” Evelson said.

Is agile business intelligence as hard as Evelson makes it sound? If you have blazed a path to agility and are willing to talk about it, I’d love to hear from you.

Join the conversation

1 comment

Send me notifications when other members comment.

Please create a username to comment.

This article caught my attention since I've developing Agile BI techniques since 2004. As an Agile BI practitioner, trainer, and coach I get the opportunity to work with a lot of companies across multiple industries in this area. What I am finding is that many CIOs are observing the power of agility in their software development initiatives and wish to see the benefit of agility in their BI initiatives. This article describes Barry Evelson as saying "agile software development methodology — which relies on prototyping rather than specifications..." If he really says this, then I take strong exception to his point of view. Agile development (including Agile BI) is NOT about prototyping. It is a focus on the early and continuous delivery of high-priority, production-quality, working features to business users - and adapting to their feedback in order to build the right thing. In other words, we work in short iterations (2-4 weeks), delivering small increments of business value, and evolve the solution in close collaboration with our business users. The benefit of Agile BI for IT leaders is that they meet the actual needs of business users early in a project cycle rather than those long 12-36 month horizons that we've seen in the past. One last thing. Agile DW and BI has been tried and proven to work well. We have to adopt some different approaches for handling large data volumes, dealing with legacy systems, and evolving systems that are in production. My book, [A href=""]Agile Analytics: A Value-Driven Approach to Business Intelligence and Data Warehousing[/A], is now available on Safari (rough-cut) and will be on shelves in July.