Guide to big data analytics tools, trends and best practices
A comprehensive collection of articles, videos and more, hand-picked by our editors
Big data is like a fashion trend: You can't turn around without seeing yet another magazine article, research report or talking head pontificating on why you can't live without it. In this case, however, the trend-pushers have a point. Case studies at the Gartner Business Intelligence Summit added to a growing body of evidence that a company's ability to collect, process and analyze large data sets could well mean the difference between...
success and failure. Leadership -- on the IT and business side -- is key.
In part one of this two-part SearchCIO tip on developing a big data analytics strategy, Gartner analyst Doug Laney laid out the steps CIOs must take to make IT departments big-data-enabled. Here, Laney and colleague Mark Beyer explain why big data represents a rare technology breakthrough and offer three basic business rules for leaders to ensure that their companies don't get left behind.
Basic business rules for capitalizing on big data
Define the value of big data for your company. In 2008 and 2009, simultaneous improvements to storage, memory, processing and network technologies triggered big data's rise in popularity. These improvements are often characterized as the three V's of big data -- volume, variety and velocity. "Everything got bigger and faster all at once," Gartner analyst Mark Beyer said, referring to this as an "alignment of the four planets of technology."
It's a moment in technological history that happens once every 15 years or so, according to Beyer; when it does, companies stumble to figure out how to take advantage of the rare occasion, often seeing it as an opportunity for more data analysis. This is where businesses are now with big data and why the fourth V -- value-- has begun to emerge.
Read about successful big data projects
"We've come around to the idea that big data isn't all about these three V's," Laney said. "It's also about recognizing that there are opportunities to really innovate with this information."
Big data analytics can not only help businesses create new efficiencies, make better decisions and edge into real-time processing, it can also inspire business transformation. Some of the most successful manipulators of big data have figured out how to turn the data they've collected all along into a new source of revenue.
"Dollar General is a great example of an organization that has implemented a self-funding data warehouse," Laney said. The retailer grants access to anonymized transactional data (see sidebar) for a fee via 1010data's Software as a Service data warehouse product, he said.
Take an inventory of company data sources (yes, it is business' job). Laney said he's "concerned, flabbergasted, dismayed" that businesses have a better accounting of physical assets such as tables and chairs than they do of their data. His research on what he calls infonomics -- the economics of information -- encourages businesses to think otherwise.
To get started on this business rule, Laney suggests taking an inventory of data sources -- internal and external -- available to the business. The list could include operational data, such as point-of-sale or machine-generated; commercially syndicated data; dark data, which refers to data collected, used once and then archived, such as email; and public data, such as weather reports or census information.
Doug LaneyAnalyst, Gartner Inc.
The Climate Corp., a startup from two former Google employees, uses public data and big data analytics to build its flagship product -- crop insurance policies for U.S. farmers based on weather predictions. Unlike other insurance companies, The Climate Corp.'s policies are individualized and dependent on a variety of factors such as crop type, field location, soil type and relative maturity of seed planted, according to the company website. Some data points used come from open data sources, such as weather and forecast reports, as well as soil samples, Laney said.
Adopt and adapt good big data ideas from other industries. "One of the questions I get as an analyst is, 'What are others in our industry doing with big data or analytics?' That's not an outright terrible question … but why do you want to be second?" Laney said.
For example, everyone's stuck on the story of Target outing a pregnant teenager with its use of data and analytics; it's a fun story to tell, Laney said, but can the story be mined for innovative ideas that can be applied to businesses besides Target? Probably not, he said. Rather than seeking out trials and successes from similar businesses, look at the successes from businesses in other fields and how those successes might be adapted. "This is where innovation comes into play," he said.
Take the show House of Cards, launched in the U.K. (see sidebar). To create the U.S. version of the show, its parent entertainment company used viewer data -- and lots of it -- to determine the storyline, the director to hire and the leading actor to cast.
"You may not be in the entertainment industry," Laney said, "but what kind of data can you collect about the behavior of your customers or users over time that could help you craft new offerings that specifically and expertly target their interests?"