Sergey Nivens - Fotolia

Manage Learn to apply best practices and optimize your operations.

Without a data-driven culture, big data projects will fail

Looking for a big data project success? Remember this: Culture is king.

Tara Paider, associate vice president of IT architecture at Nationwide Insurance based in Columbus, Ohio, has some advice for data experts eager for big data project success: One of the biggest reasons big data projects fail is neither the technology nor the quantity of the data. It's people.

Case in point: A regular part of the job for Nationwide's insurance agents is to make sure customers don't jump ship when their premiums are due to go up. Working from a list of premiums expected to change in the next 30 days, agents pick up the phone and connect with their best customers to explain the changes. Data from a new customer data analytics program, however, found that doing so sometimes had a negative effect: Rather than help agents hold on to customers whose premiums were changing, the calls actually caused attrition, Paider said.

"It was hard for [the agents] to wrap their heads around the fact that what the data was saying was different from what they'd seen and acted on for the last 20 years," she said to Gartner Business Intelligence and Analytics attendees in Las Vegas.

And so Paider and her team began producing more nuanced lists, giving agents access to only those premium notices "where calling and touching base with the customer would have the right impact," she said.

Tara Paider, associate vice president of IT architecture, Nationwide InsuranceTara Paider

It's a story as old as the CIO position itself. ERP consolidation projects often failed, for example, because of people -- not technology. Big data, it seems, is no different. But big data is upending more than business process workflows; it's having an effect on everything from enterprise infrastructure to the org chart. To ensure that people are excited by rather than threatened by what the data is saying, experts often tell companies that they simply have to create a data-driven culture. But that's easier said than done.

As Paider pointed out, a strong corporate culture -- of any ilk -- may begin with executive sponsorship, but it doesn't end there; it also requires frontline employees to get on board, which can create additional hurdles. "It was the hardest thing to get past -- the 'this was the way we've been doing it for 20 years or 30 years, and we know best.' That's our biggest challenge," she said.

Moving toward a data-driven culture

Paider's view is not unique.

Research from PricewaterhouseCoopers cited culture -- or rather, the wrong culture -- as one of three roadblocks holding businesses back from exploiting data and succeeding in the information age. The study, produced in conjunction with Iron Mountain, found that three out of four businesses are extracting little to no advantage of any kind out of their data.

But how does a company move toward a data-driven culture, which Gartner defines as businesses that use data "to organize activities, make decisions and resolve conflict"? At Nationwide, one of the steps was to appoint a chief data officer (CDO), to whom Paider reports.

"Having a CDO was a good message that we want to leverage data to make decisions," she said. "I don't think it's the only way to change the culture, but you need someone who has that voice around data in both the business and IT to drive the right behaviors."

Four elements of a data-driven culture

In a research note titled, "How to establish a data-driven culture in the digital workplace," Gartner analysts Alan Duncan and Frank Buytendijk offer these suggestions on how to build a data-driven culture:

  • Lead by example. CIOs should make a conscious effort to communicate to employees how they're using data to make decisions. "In meetings, in presentations, in all daily interactions, executives need to show they are looking for the right data to base decisions on," the report states.
  • Hire data-driven people. CIOs can send a signal that data is playing a prominent role by hiring candidates who think in data terms.
  • Create more transparency. Make it easier to access data. Along with that, CIOs should also make information governance policies more transparent.
  • Conduct data-driven performance reviews. In a data-driven culture, data should be used in every aspect of the business -- from hiring to goal setting, according to Gartner.

Another culture fix at Nationwide was to open up data to the business. New, unstructured data sources, such as geolocation, voice and social media data, promised additional insights into customer behavior and additional opportunities to serve customers better. But to help the business find those insights, IT needed to invest in big data technology and commit to giving the business access to the data. "Traditionally, we thought about data projects as linear, much like app dev projects," Paider said. "But data projects are messy, and you really don't know what you're going to find in that data ... until you get in and start to touch and feel the data."

Jeremy King, CTO and the head of WalmartLabsJeremy King

Opening up company data was key to building a big data culture at another data-driven enterprise. Jeremy King, CTO and the head of @WalmartLabs based in San Bruno, Calif., said his team centralized data onto a single Hadoop system to give internal customers access to the data they needed to run experiments. Before providing access, King's team had to build processes that cleansed and tokenized the data to protect personally identifiable information. But the company also had to remove bureaucratic barriers that made accessing data difficult, a step that eludes some companies. "I've talked to so many companies that have done the early work on setting up Hadoop or a big data architecture, but they don't let anybody have access to this system," he said during his presentation at Strata + Hadoop World last fall.

That kind of red tape can limit creative thinking and experimentation, King said. Even providing access to just a portion of data can be limiting at a company like Walmart. "If you're only using a subset of data, you have a hard time making a determination of whether this is going to work at Walmart scale. So we want everyone to have access from the very beginning to test their theories," he said.

Micheline Casey, advisory board, ClearStoryMicheline Casey

Granting access to the entire anonymized data set has paid off. In at least one case, doing so spurred on a startup-like moment when, in a matter of hours, two engineers conceived of and developed the prototype for an advertising optimization platform that connects online ad impressions with offline sales. The prototype eventually became Walmart Exchange.

"I implore you, unless you have a system that lets people have access to data, these types of magical moments are not able to come to life," King said.

How to build a data-driven culture

To start building a data-driven culture, CIOs have to find use cases that are compelling to the business. Micheline Casey, former chief data officer at the Federal Reserve who is now serving on the advisory board for big data analytics company ClearStory Data, experienced resistance to big data projects not because of a lack of interest or buy-in, but for a much more pragmatic reason: money. "The Federal Reserve is a government agency, and, so even though it doesn't take money from Congress, it has a set budget," she said. And a small budget at that, which created competition for funding dollars.

"Particularly for companies that are new to big data ... identifying and prioritizing the right projects and use cases to show value builds credibility within the organization," she said. "And it's that credibility in many organizations that gets the additional dollars."

Gideon Mann, head of data science for the CTO office, Bloomberg LPGideon Mann

Gideon Mann, head of data science for the CTO office at Bloomberg LP, echoed Casey's point. "When you think about grassroots change toward a data driven and machine learning/big data kind of approach, it's a series of incremental trust building," he said.

That's why experts, including Mann, suggest introducing the business to big data by solving a small, discreet problem. "You want to find one example that can be an exemplar," Mann said. "There, you have a very concrete problem, and you're going to have a very applied solution for that problem."

As the business sees quick wins, as trust in the methods grows -- as the culture becomes more data driven, IT's strategy will have to change. At Bloomberg, for example, the big data value discussion is over. Now the challenge is figuring out what exactly needs to be built on top of machine learning and big data platforms.

"You have to make sure you're closely aligned with product and you're [building] more of a general system than a particular solution to a small problem -- because the small problem might change," Mann said.

Let us know what you think of the story; email Nicole Laskowski, senior news writer, or find her on Twitter @TT_Nicole.

Next Steps

Blackjack depends more on data than luck

Dell's CDO on building a data-driven enterprise

Why culture can keep you relevant

Dig Deeper on Enterprise business intelligence software and big data