It might have seemed like just another buzzword, but before 2011 was out, the need to mine big data was topping many a CIO agenda, upending long-established practices for data management. By the time the calendar turned to 2012, all eyes were on big data analysis -- how to make sense of this wealth of information to maintain, enhance and expand the business.
Accordingly, attention also turned to business intelligence (BI). Based on a survey of CIOs at 178 organizations worldwide, the CIO Executive Board of Washington, D.C.-based Corporate Executive Board Co. (CEB) found that CIO spending plans for 2012 favor BI projects. CEB Executive Board Director Shvetank Shah called this BI focus a "mega-trend" -- IT focusing on information-related projects.
"[T]he real action right now is in business intelligence, as well as [in] collaboration and anything at the customer interface -- from either understanding the customers' patterns, performing customer service or empowering salespeople through IT to be better at their sales jobs," Shah said. "All of these line up with information and analytics. What remains to be done for IT, increasingly, are those information projects, not the process projects."
Whatever your industry, your company's ability to manage and mine large data sets will be critical to its success going forward.
Let us know what you think about the story; email Karen Goulart, Features Writer.
Table of contents:
What effect is big data having on enterprise CIO data management strategies?
The ability of most existing IT systems and organizations to capture, communicate, aggregate, store and analyze data is no match for big data's many dimensions. Depending on the industry, data sets can range from a few dozen terabytes to multiple petabytes. And, taking the variety of information being collected from social media and context-aware mobile information to databases, metering data and video -- as well as the velocity at which data must be processed -- into consideration, the chances of big data analysis success decreases. The software development cycle for large-scale data processing alone is not the same as it is for traditional BI, according to Boris Evelson, Forrester Research Inc. analyst and BI expert.
In traditional BI projects, business requirements come first. "You talk to business users, define the requirements, put them down on paper, architect them and implement. Big data is mainly about the fact that I don't even know enough about what is out there to give you my requirements," Evelson said. People need to explore. Requirements could coalesce quickly, but it's just as likely that initial theories will not be supported by the results and more exploration will be needed, he said.
How are traditional BI strategies evolving with the need for big data analysis?
Traditional BI strategies are evolving to adapt to the demand for quick access to data by incorporating technology beyond traditional on-premises systems. Some IT leaders like Jim Szatkowski, vice president of technical and data services at Schaumburg, Ill.-based Distribution Market Advantage Inc., are adopting cloud solutions to supplement their BI strategies in response to the desire for anytime, anywhere information. In Szatkowski's case, he is using a BI Software as a Service (BI SaaS) product from PivotLink Corp. that allows customers of the national food distribution system to access supply chain data in real time and custom filter thousands of lines of data in minutes.
James Richardson, BI research director at Gartner Inc., said current BI SaaS products have matured in terms of features, scalability and support, making them a viable option for the enterprise. In some cases, cloud-based BI tools are replacing on-premises BI, but businesses more commonly are using them to augment resident BI systems for urgent, tactical needs.
"Business users are often frustrated with long deployment cycles, high costs, complicated upgrade processes and the IT infrastructures demanded by traditional BI solutions," Richardson said. "SaaS and cloud-based BI offer a quicker, potentially lower-cost and easier-to-deploy alternative."
What challenges does big data present to CIOs?
One of the greatest challenges of big data is not just the amount of information the term implies, but the incredible variety of that information. Data quality has always been something of an issue for the CIO creating or tweaking a BI strategy, but the size and scope of big data only exacerbates the problem.
If data quality is relative and getting data to a point where it is acceptable for use remains a BI imperative, what should be the CIO's plan of attack? Bill Hostmann, distinguished analyst at Gartner Inc., advises his clients to establish data quality metrics by surveying their enterprises' "thinkers and deciders" on data quality issues related to their BI programs. Gartner uses a simple survey tool that measures users' level of satisfaction with their BI data on four fronts (timeliness, relevancy, accuracy and consistency), as well as their ability to use the data to make business decisions. The results should be tracked quarterly because the definition of data quality, again, is always changing.
As BI shifts from an effort controlled by the IT department to an activity practiced by users at all levels of the enterprise, what defines the quality of data also is a moving target. For CIOs, the question should be this: What quality of data is good enough for the task at hand?
"It doesn't matter what they, as the providers of information, think data quality is," Hostmann said. "What matters is the level of satisfaction of the person who is using that information to analyze or make decisions on the data. Their perspective is what counts."
What opportunities does big data present to CIOs?
For all its intrinsic headaches, big data can position CIOs to be the champions of their companies. Take Christopher Perretta, executive vice president and CIO at State Street Corp. The Boston-based financial services company has a reputation for being a leader in technology, and as he crafts his IT strategy, Perretta knows he has a legacy to uphold. He knows that these days, the cutting edge rests on managing large data sets.
The push to unlock information at the financial institution is driving the construction of "a whole new set of tools to handle very large data," Perretta said, including private clouds for processing and analytics. His IT organization relies on a chief scientist to keep tabs on the newest technologies and on a seasoned chief architect to identify projects that make sense for the business. And you won't get any argument from him on whose job it is to make data accessible to the whole company -- or as it puts, act as "the linguist" to the digital enterprise.
"Occasionally I look at my business card: It says chief information officer, so when I was looking around for someone to do this, I figured it must be me," Perretta said.
Gartner Research analyst Yvonne Genovese, who with colleague Mark A. Beyer co-authored a report on extracting value from big data, said this is a time for CIOs to shine.
"This is an opportunity for CIOs to return to the boardroom," Genovese said. "Business leaders believe there is information locked up that can help them make better decisions. They want fact-based decision capability, and they are looking to IT to get them there."
What is the role of IT when it comes to big data analysis?
With the pressure on to make sense of big data, IT leaders often try to step into the role of data analyst. Expecting the architects and developers who installed BI technology to analyze data -- usually as a means of maintaining control of the domain -- is the wrong approach, according to John Weathington, president and CEO at San Francisco-based management consultancy Excellent Management Systems Inc. He notes that BI data analysis and technical development might seem related but they are completely separate disciplines.
Still, if control of the domain is important, a CIO might consider tweaking the IT staff's skill set. One option is to add data analysts; another is to cross-train the organization's IT professionals in data analysis skills. These solutions, of course, involve money and time. A third viable option is for IT staff to partner with data analysts in other parts of the company. This will have an added bonus: showing a spirit of collaboration between IT and the business.