According to Harvard University professor Gary King, big data isn't necessarily about the data -- it's about the...
analytics. The problem is, analyzing massive amounts of data to extract business value is no easy task and typically leaves the door wide open for poor data interpretation -- and backwards decision making.
In her recent CIO Matters column, Big data, bad analytics, Executive Editor Linda Tucci delved into why research focused on applying empirical methods to social science research doesn't always work, and shared how King's team suggests remedying bad information analytics.
It's highly essential to have real-time analysis for having effective results.
Our SearchCIO readers had quite a bit to say about data gathering and interpretation, highlighting the importance of hiring math experts, selecting proper business intelligence (BI) tools and implementing real-time analysis for valuable intelligence.
As a prompt, Tucci asked, "Let's assume your company is working on a big data project. Are you confident it has the right analytics to derive business value from it?"
Only 37% of readers polled felt confident in their information analytics program, while some admitted, "No, we don't have the right analytics in place." Here are two not-so-optimistic responses to Tucci's question:
- "Not yet, but we are working on it."
- "At this moment, we are not confident at all. There is still long way to go on big data analytics implementation."
But several readers among those polled are currently tackling information analytics. Here are their thoughts and stories:
- "We've taken the steps to assess our business intelligence needs, find a BI solution to fill those needs and implement the BI solution throughout the entire company. From the CEO to low-level employees, everyone has access to how their efforts fit into the company's goals and strategies. This is due to the fact we selected a BI solution with real-time data feeds, dashboards and collaboration. It's changed our company."
- "We had many successful analytics projects based on statistical analysis and data mining before going to big data. We want to now apply this on a massive amount of data."
- "The off-the-shelf 'big data' analytics technology is still in its infancy, but in-house solutions are not the best answer to this kind of project challenge. According to some proof of concept (POC) I've carried out to test-market solutions, HP Autonomy and IBM Content Analytics have the best answers to face this challenge, but both require some customization (in my opinion, HP Autonomy is the 'top' solution for Big Data)."
- "For big data analytics, one algorithm is not a silver bullet! For example, the Centers for Medicare & Medicaid Services data released for pharmacy drugs and mobile health is complicated. After applying regression, I looked at the bias in coefficients. The results were dates of services that had patients paying nothing or less than the total cost of drugs. But we needed to quantify 'less.' What about $10 less or $20 less? Only then can patient behavior can be interpreted because Medicare patients who had to pay the least (or largest price differential) would be filling the medications often, hence the need for segmentation. I suggest CIOs apply mathematics like differential calculus and Taylor's Power Series to get the proper, actionable results."
Speaking of applied mathematics, readers stressed the need for understanding numbers and equations for effective data interpretation:
- "Here in Mexico, hiring professionals with seniority in statistics and math is increasing very fast. This situation answers the questions of the issue here. Companies must hire statistics experts in order to lead with big data and social media data."
- "You are applying a metaphysical equation: Multiplying the unknown with the uncertain!"
- "Focus on mathematics and prepare to be surprised by the variety of data; results may be a bit different than what you thought the math equations would do."
Information analytics initiatives
iPad analytics app speeds decision making
Prescriptive analytics take the spotlight
Tom Davenport's analytics how-to's
King's research team experimented with sentiment analysis by word count, focusing on social media posts containing certain keywords related to "jobs" to identify a correlation to the monthly unemployment rate. They found that their analysis technique wasn't always successful, especially when Steve Jobs passed away and his death dominated the news.
Errors like this happen all the time, so what can be done to ensure better information analytics? King suggests computer-assisted reading, verbal fact-checking and reframing the analysis of social media censorship. Readers offered more advice:
- "It's highly essential to have real time analysis for having effective results."
- "Big data is job one, but a lot of smarts must to go into the analytics part."
- "Having supplied BI solutions since 1998, we've noticed the first requirement for success is that the company application users get [provides] the answers they need with chosen solutions. Solutions must allow users to have the situation under control and then react accordingly. While one has to begin from somewhere with regard to big data, specific know-how and mature software are necessary. But I am not sure about existence of the latter."
Do you have advice for CIOs looking to turn big data and information analytics into business value? Share your tips in the comment section below.