This content is part of the Essential Guide: Big data exploration and analytics for CIOs: Oh, the places you'll go

Essential Guide

Browse Sections

The history of business intelligence and analytics and what comes next

A look back at the history of business intelligence and analytics reveals where data analytics has been, where it stands, and where it’s going.

“Business intelligence” and “analytics” are terms as common in our world as peanut butter and jelly. In deference...

to our global readership, to avoid any possible local or regional bias, we should perhaps additionally consider fish and chips, borscht and potatoes, hummus and falafel, and likely many others, but I don’t want to belabor the point.

As best I could determine in my look back at the history of business intelligence, the term was first used by H. P. Luhn in an article entitled “A Business Intelligence System,” published in an IBM research journal in 1958. Luhn defined BI as “the ability to apprehend the interrelationships of presented facts in such a way as to guide action towards a desired goal.” Sounds straightforward.

During the next 30 years, we evolved the original concept through various stages of maturity: decision support systems (DSS) and executive information systems (EIS) were quite in vogue during the 1970s and 1980s. In 1989, a major milestone was achieved when Gartner Inc. analyst Howard Dresner described business intelligence as “concepts and methods to improve business decision making by using fact-based support systems.”

Most of the work done throughout this period was focused on technologies, standards, processes and tools to support the collection, storage rationalization and retrieval of data and the creation of reports. Data warehouses, data marts, data dictionaries, and extract, transform, load (ETL) processes became ubiquitous. Think about this stage as the beginnings of transforming data into information and the use of information to help drive (primarily operational) decision making.

Fundamental shift toward analytics

And then, everything began to change, largely driven by a 2,500-year-old discipline: statistics (think “analytics”). According to Wikipedia, evidence of the use of statistics can be found as far back as the 5th century BC, and more formal writings on the subject date back to “Manuscript on Deciphering Cryptographic Messages,” written by Al-Kindi in the 9th century. That work provides a detailed description of how to use statistics and frequency analysis to decipher encrypted messages.

Tom Davenport, a visiting professor at Harvard Business School and senior adviser to Deloitte Analytics, is generally acknowledged as a leading global authority on business intelligence and analytics. In “Competing on Analytics: The New Science of Winning” (Harvard Business School Press, 2007), Davenport and co-author Jeanne G. Harris describe fundamental shifts in how analytics are being used. Their work shows us how top-performing enterprises are using data-driven analytics to inform competitive strategies, literally referring to these best-in-class capabilities as “secret weapons.” Diverse examples cited include Amazon, Barclay’s, Capital One, Harrah’s, Procter & Gamble, Wachovia and the Boston Red Sox. Clearly, when done right, the benefits are inarguable.

And still, according to a recent Gartner study, 70% to 80% of business intelligence projects fail. Expenditures of hundreds of thousands, millions, even tens of millions of dollars invested in business intelligence and analytics projects commonly yield questionable business benefits. Given that we have been working on these challenges as an industry for roughly 50 years, and as a civilization for more than 1,000 years, why can’t we get this right?

Here’s a list of some of the more common pitfalls:

  • Lack of business involvement and support, resulting in questionable agreement on objectives and benefits accompanied by insufficient or nonexistent business sponsorship;
  • Siloed data across lines of business and or geographies, often accompanied by insufficient governance around ownership and access to needed data, often making the understanding and resolution of data inconsistencies difficult to understand and sometimes impossible to resolve;
  • Big-bang scope, multiyear timelines and expanding scope of deliverables that almost guarantee suboptimal, too-commonly catastrophic results;
  • IT’s assumption that if they build it, the business will come, often stemming from an unenlightened reaction to lack of business buy-in and involvement;
  • Poor operational performance based on overly complicated IT solutions that deliver too little, too late.

I could go on, but I think the point is made. There are four important dynamics at work here that we, as the leaders of our industry, now need to accept, embrace and aggressively respond to:

  1. The common reasons for failures in business intelligence or analytics programs are not entirely unique to the programs themselves. Look back at the list above and remove BI/analytics from your perspective and think about ERP, CRM and financial management, for example, and the relevance of the list of challenges is about the same;
  2. Advances in business enabled by technologies like mobile computing, social networking and social media, unstructured data, geo-spatial, biometrics, sensor-based technologies, consumerization of IT and cloud computing -- all of these are driving an explosion in the types and volume of information at an unprecedented pace. We generally call this “big data.” Does that mean that what preceded this period was based upon “little data”? And if we are so challenged by “little data,” OMG;
  3. Much of the new types of data enabled by the advances in technologies described above is available in real time. Business intelligence and analytics disciplines are rapidly evolving from looking at history to forecasting the future to looking at the present to inform action now -- for example, optimal customer offers, real-time supply chain management, proactive customer service, etc. If we are so challenged by batch approaches to information management, how will we step up to really exploit the value of real-time action and reaction?
  4. Enterprises that can successfully embrace and realize business value from exploiting the rapid changes in information types and in the incredible volume of data available from an increasingly diverse set of sources will widen the competitive differentiation gap. It will become, if it has not already become, the competitive differentiation chasm. Enterprises that cannot get with this program will likely be marginalized, or worse. CIOs who cannot successfully lead their enterprises through the successful enactment of these programs will likely be at the “or worse” end of the range of possible outcomes.

And so, my esteemed IT colleagues, friends and countrymen, please heed my words as a wakeup call. First and foremost, ensure that your priorities are properly connected to your business. Ensure that you can articulate the true business value and the right business case for properly sponsored and appropriately supported business intelligence and analytics initiatives.

Do not be shy about communicating with your business partners around the competitive differentiation chasm and the benefits and threats associated with living on either side of that gap. And get your IT organization in order; ensure that you have the right bench strength and the right staff with the right skillsets to successfully implement these programs. These are not your father’s MIS programs.

Are we there yet? Clearly not, but we are on the journey. Future writings in this column will endeavor to provide continued insights into how to think about these and related issues. I strongly encourage questions and additional dialogue around how to use these insights to lead your enterprises in formulating the most efficient and effective strategies that leverage IT to drive the creation and growth of sustainable business value.

Let’s continue the conversation.          

Dig Deeper on Enterprise business intelligence software and big data