The big data conversation is changing. What used to be a debate largely about semantics has turned into a discussion about business value and necessity. Experts call this a sign of maturity, but that doesn't make getting a big data project off the ground any easier. Today, it's still common to hear CIOs and IT departments question how to get started. The answer, according to experts at Nemertes Research Group Inc., isn't about getting all your ducks in a row.
"Don't get wrapped up in having the right governance, the right buy-in, and cogitating endlessly over what the perfect goals are," said Johna Till Johnson, president and founder of the Mokena, Ill.-based consultancy. "Get a rough and ready consensus of what you think you might need and go out and get a proof of concept using open source tools and get it out there," she said.
Johnson and her colleague John Burke, CIO and principal research analyst for Nemertes, recently led a three-hour course on how businesses can build a big data initiative. In part one of this expert tip, Burke and Johnson provide six ways to break out of the traditional IT mind-set and get moving on a big data initiative. Part two focuses on how data stewardship helps CIOs ensure the success of a big data initiative.
1. Show, don't tell, and think local
One of the first things to keep in mind is a classic: A picture -- or a prototype, in this case -- is worth a thousand words. And there's no reason that prototype can't tap data already on hand.
"A good place in IT to think about that is your log data," Burke said. Log data is generated so rapidly that businesses tend to limit how much they're collecting and, even then, are barely capable of managing and analyzing the stuff they do have. That makes log data, and internal systems in general, ripe for big data project fodder.
Start by "thinking of a question you would love to have the answer to," Johnson said; it might be low-hanging fruit, but it can also quickly showcase the power of big data.
2. Think IT informatics?
One of the best ways to design a big data prototype is to figure out a question that requires combining not only data on hand but also different kinds of data already being collected.
"Don't go crazy trying to find new sources of data," said Johnson.
Instead, take a page from health informatics. It's not uncommon for hospitals to have whole teams whose job is to figure out what kinds of disparate information could be pulled together to help uncover new patterns or new insights.
"One of the things I would suspect … is that having the equivalent to medical informatics within your own organization is going to be key," she said.
Companies have a 22% higher success rate with a defined big data budget.
Johna Till Johnson
3. Add it to the budget
Big data is so new that there's no set of best practices to follow, Burke said. So figuring out who owns big data and how to fit it into an organization is not easy. Still, there are some emerging trends that point to greater success. One is to devote a line item in the budget to big data -- even if no actual resources are being devoted to the initiative this year.
"Companies have a 22% higher success rate with a defined big data budget," said Johnson, quoting a statistic from the consultancy's big data benchmark research published in June.
4. Get a sponsor
Having someone high up in the organization focused on the big data initiative can make a difference, according to Burke.
"If that single person is a C-level person or a senior vice president, that is the most successful strategy for running a big data project," he said.
A high-ranking advocate provides the attention and support a big data initiative will demand, Burke said. This is not the person to pick your next storage or data management platform, but he or she can help with the big picture and keep people focused, he said.
5. Invest in new tools, new talent and a new start
Another emerging trend is to realize a training budget alone doesn't go far enough.
"We started our research this year with the thinking that the training budget was going to be critical to the success of big data," Burke said, "but it turns out not to be."
Big data is bringing new tools into the mix that do require training, but right alongside with introducing new tools, CIOs also need to break down old mind-sets.
"The people coming in the door that know SAS and SPSS tend to be older, tend to be more expensive and tend to have a particular worldview in how to do analysis that's not necessarily in line with big data," Johnson said.
Maneuvering those employees out of their comfort zone might not be the best answer, Johnson said. Instead, think about bringing in new talent to help bring that new worldview to the company. That's easier said than done, but finding good talent now could prevent headaches in the long run. "We're finding, across the board, that retrofitting your existing systems by training is not as effective as refreshing from the start," she said.
6. Form a tiger team
A big data initiative should not be siloed as a segment of the IT department. Burke and Johnson advise forming what they call a "tiger team," or a group of members from across the IT organization who analyze how a big data program could affect everything from infrastructure to analysis.
More on big data
How well do you know big data? Find out by taking this quiz
Read our guide on the evolution of big data
Learn how big data can provide intimate customer service
Representatives from across the business should be included, too. Look for people who need to "understand the technology as well as understand the value proposition," Johnson said. "The more groups within an IT organization involved in big data, the higher the success. And the more groups across the company -- from sales to marketing -- the higher the correlation of success."
Go to part two for Nemertes' four pointers on what data stewards need to do to ensure the success of a big data initiative -- from rethinking their approach to data quality to casting a skeptical eye on product offerings from stack vendors.