Big data exploration and analytics for CIOs: Oh, the places you'll go
A comprehensive collection of articles, videos and more, hand-picked by our editors
IT departments and the business do not see eye-to-eye when it comes to defining big data success. That's according to a newly released survey of more than 300 IT and business professionals from technology consultancy Wikibon. More than half -- 54% -- of the respondents who identified themselves as working in IT said they realized the "full value" of their big data projects, but only 18% of business respondents reported the same thing.
By submitting your personal information, you agree that TechTarget and its partners may contact you regarding relevant content, products and special offers.
Definitions of success, of course, are subjective, said Jeff Kelly, an analyst for the Marlborough, Mass.-based consultancy and the report's lead author, but "we don't take the responses at face value." The significance is in the marked disparity between the two sets of responses. IT is responsible for setting up the big data infrastructure and, once complete, considers the job successful. The business, on the other hand, "doesn't care about hardware," he said, only insights. "It's a warning to the entire market -- vendors and the enterprise," he said. "If we don't overcome this disconnect between IT and the business so that the business gets value out of big data analytics, these projects won't be successful."
Here are two additional trends Kelly plans to follow closely:
- Data warehouse vendors will feel the burn. Hadoop and NoSQL products probably won't replace the enterprise data warehouse, but they are competing for workloads. Because of that, "you're going to see spend start to level off in the database space," Kelly said. He's hearing anecdotally that businesses want to reduce and even freeze spending with established database players like Oracle. And, according to survey results, more than 60% of respondents said they've either moved or are experimenting with moving applications off traditional databases to a big data technology.
- The public cloud gets some love. More than 55% of respondents said they are using the public cloud somewhere in their data analytics stack. Of those, almost 80% reported that use as proofs of concepts (POCs) or pilots. "The big question is what's going to happen when a large group of POCs decides to go into production," Kelly said. Will businesses bring those projects in house or will they expand their deployment in the cloud? He predicted, "the public cloud will play a much bigger role in big data analytics projects," but it's going to take time -- 10, maybe 15 years.
OLTP + OLAP = intelligent apps
If Saum Mathur's vision of the future is correct, keeping your analytical database separate from your transactional database is poised for extinction. The vice president of information management and enterprise architecture for Hewlett-Packard IT shared his prediction at last month's Vertica Big Data Conference in Boston.
Today, CIOs have to invest in two different types of databases to tackle online transaction processing (OLTP) and online analytical processing (OLAP). But Mathur believes the technology will soon converge. In the future, he said, "applications are all going to be intelligent applications." Imagine, he said, if business processes could leverage analytics for operational efficiency and productivity -- by themselves.
"I see that as the next step -- bringing two worlds together," he said. Evidence he's onto something? Look no further than Apache YARN (Yet Another Resource Negotiator), introduced with the release of Hadoop 2.0 to open up HDFS data to processing systems beyond MapReduce, or HP's Trafodion, a beta technology that layers OLTP on top of Hadoop.
Big data lights up Buenos Aires
When thinking about big data, Hans Koolen, director of consumer IT at electronics giant Royal Philips, keeps things simple. "Think of classic data as your EDWs [enterprise data warehouses]," he said at last month's Vertica Big Data Conference. "This is slightly exaggerated to make the point, but your warehouse is used to manage your bottom line. Big data is used to manage your top line."
Koolen said Philips is thinking about how big data will help grow revenue, build new products and create competitive advantage. Last year, the company replaced 91,000 of Buenos Aires' conventional street light bulbs with LEDs as part of its "lighting as a service" program.
"It gets really exciting when you know that every light bulb is addressable," Koolen said, which means every light bulb can be remote control operated and is programmable. "That's all enabled through the use data," he said. "And not from an EDW perspective, but from a really big data perspective."
The sounds of big data?
Here's a unique big data use case: IBM Corp. and James Murphy, a musician and music producer, are transforming big data into music -- in real time. The project is called "US Open Sessions," and it leverages streaming data from US Open tennis matches, translating data points into electronic-sounding music notes.
The project may be new, but the data collection is not. IBM has been gathering tennis match data for years and serving it up through its SlamTracker app, a predictive analytics and interactive data visualization platform for today's diehard fans. This year, data points include ball and player movement, no doubt essential to creating a beat in tennis music. Interested in hearing what big data sounds like? Listen here.
More from The Data Mill
Consumer messaging apps make their mark on the enterprise
IT architecture needs a good dose of digital intelligence
Chief digital officer vs. chief data officer: Who will win the day?