From data gathering to competitive strategy: The evolution of big data
A comprehensive collection of articles, videos and more, hand-picked by our editors
Never mind the predictions about the end of days. The end of Christmas returns is nigh. After a year during which companies funneled millions of dollars into fumbled attempts to build big data technology, some experts predict 2013 will mark the start of this disruptive technology actually disrupting whole industries, from the $60 billion weight-loss market to education K-12. One of my favorites waiting in the wings: a women's fashion...
retailer that, based on the online apparel bought and returned, knows enough about a customer's body type to tell her those skinny jeans she's just clicked on will make her look fat. (Conveyed in the nicest possible way, no doubt.)
"You can still buy them, but the technology is available to predict the item may not be the best fit for you and to recommend another style," said Anil Kaul, CEO at AbsolutData, an Alameda, Calif. marketing analytics firm working on this very solution for a retail client. Still buy them? Yo, big data analysis, I worship the algorithm you whir on, girlfriend!
Kaul, who has a doctorate in marketing from Cornell and over a decade of experience in marketing and quantitative modeling, believes that for all the hype about big data technology and big data analytics, he's seen only a handful of companies -- Yahoo, Google, eBay -- "do new stuff that could not have happened without big data technologies."
At many other companies, the so-called "big data analysis" under way is more akin to traditional business intelligence (BI) with bigger data sets. At others, the efforts have centered on building the big data infrastructure. "People fall in love with the technology because it is fantastic technology, but they forget it is there to serve a business purpose," Kaul said.
Build it and the business will not necessarily come. The familiar caution for CIOs -- it's the business problem, stupid -- is especially applicable in big data analysis projects, because the technology is still immature, Kaul said. There is no "one best set of technologies to use." Better to pinpoint the business problem first and mix and match what's required to solve it specifically, rather than build a platform for all analysis possibilities. AbsolutData, for example, has a client with 3,000 servers dedicated to using proprietary big data to develop a special type of social network. "What they want to build is something very revolutionary, but we had to take a step back and pare it down to focus on being able to deliver a business result in six months," he said.
BI versus big data analytics
Jeanne Johnson, lead partner for global BI and U.S. data analytics partner at KPMG LLP, is singing from the same songbook to CIOs about big data technology. "The methods and some of the tools are an absolute paradigm shift from traditional BI tools but the reason for doing big data and big analytics is the same -- to get value out of information," she said. However, the methodologies do represent a big departure from the "rules-driven" discipline of BI, in which certain sources of information are expected to provide certain insights on a weekly or regular basis, she added.
Big data analytics is about "discovering patterns, as opposed to realizing expectations. In traditional BI, you're working for your data, is how we put it. With big data analytics, we like to say the data is working for you," Johnson said. KPMG is advising CIOs to keep honing their BI programs but also jump on the "big data continuum" now, with the aim of learning as they go rather than waiting for the technology to mature and try to ramp up fast. BI organizations have to take baby steps with big data analysis before they can run with it.
The disruption big data analytics poses for BI organizations, however, extends beyond IT. Like Kaul, Johnson believes big data analysis will spur a major shift away from business process automation and optimization. The Six Sigma principles that served as a core source of competitive advantage to companies must be augmented by "information capability," Johnson said. "There is a huge opportunity, which started in the retail and the business-to-consumer industries, but which we see spreading to other sectors."
New business models on the back of big data technology
And what disruptions are on the horizon! "There are companies being built right now on the whole premise of big data," Kaul said. He's a customer of a company called FITbit, which tracks one's daily activities -- how much one walks, sits, sleeps -- on little devices that sit in a pocket or clip on a belt or a bra. "They are collecting a lot of personal data, and I am paying them to collect it," he said.
With enough data collected from enough people, FITbit could predict the optimal lifestyle regimen for Kaul. "What if they could show me that weight loss is not connected to the 30 minutes of exercise I make myself do every day but instead to how many steps I take every day? What works for me may be different for you." A service like that has the potential to upend the weight loss industry.
Think of the disruption to education once a company comes along that can build and leverage a database of student learning styles and outcomes, Kaul added. "A company can start building a model that says, 'Based on your last so many lessons and outcomes, we can predict what will be the optimal course of learning for you as a student for this material.' I can have a classroom of one, a customer of one," Kaul marveled.
These companies do not exist to collect data per se. Rather, they provide information-rich products and services people are willing to buy, Kaul noted. But their business models are based on big data technology and analysis. And that's a big change coming.