FotolEdhar - Fotolia

Hadoop security, Spark top list of 2015 Data Mill columns

We've rounded up the top five most popular Data Mill columns in 2015, which touched on bimodal IT, automation and Apache Spark.

Big data, analytics and business intelligence were top of mind for CIOs and senior IT leaders in 2015, according to just about every CIO priorities survey out there. But if this year's five most popular Data Mill columns prove anything, it's that CIOs and IT leaders are having more nuanced conversations about these topics than ever before.

Introductory stories about Hadoop, for example, gave way this year to discussions about practical concerns, such as Hadoop security. Even the topic of Hadoop itself is starting to feel old hat, as readers gravitated toward stories about Apache Spark, the new big data kid on the block.

Data Mill, itself, also changed in 2015, as made clear by this list of top columns. Rather than focus strictly on all things data, the column branched out to include IT strategy topics, shining a light on how CIOs can talk digital with the business and explicating bimodal IT, a prescription for marrying traditional IT responsibilities with the new expectations of the digital enterprise.

Without further ado, here's our countdown to the year's most popular Data Mill columns:

Robots in the workplace

Businesses should think augmentation, where humans and robots work together, rather than automation, where robots work alone. That's what Tom Davenport, professor of information technology and management at Babson College and a well-respected analytics thought leader, argued in this story, "Automation vs. augmentation: What's best for business." While it may be easier for businesses to automate, it's smarter for businesses to augment processes, Davenport said. To help businesses get started, he provided five augmentation strategies.

Hadoop security attracts attention

In 2015, Hadoop hype transitioned into Hadoop practicality. Topics such as security became a more pressing concern, suggesting the open source, big data technology is pushing its way deeper into the enterprise. In this story, "Hadoop security: Don't build your data lake without it," former analyst Jeff Kelly provided insight into why Hadoop security can be so complex.

Framework for digital discussions

For CIOs and IT leaders, it's not enough to walk the walk when it comes to digital transformation. If they're going to gain the support of the business (a key to success when going digital), they'll also have to talk the talk. In this story, "Systems of engagement are ground zero for digital transformation," Geoffrey Moore (of "Crossing the Chasm" fame), provided a two-part framework on how CIOs and IT leaders can guide that conversation.

Gartner touts bimodal IT

Businesses expect the head of IT to not only maintain legacy technology but also to provide a flexible infrastructure for experimentation. In this story, "Gartner: Bimodal IT needed to navigate centralized/decentralized debate," analyst Kurt Schlegel explains why CIOs should take a closer look at building an IT department that operates at two speeds.

Big data speed with Spark

A year ago, the top Data Mill column focused on Hadoop 2.0 and looked at how the Hadoop stack continued to evolve. In 2015, the most popular Data Mill column was still big data-centric but had little to do with Hadoop. Instead, readers expressed curiosity about a new open source, big data technology known as Apache Spark, which was spun out of the UC Berkeley AMPLab. Spark processes data in memory, which paves the way for faster, iterative analytics, an attractive characteristic for big data analytics experiments. In this story, "Is Apache Spark the next big thing in big data analytics?" Databricks co-founder Patrick Wendell explains why Spark should supplant the Hadoop stack's original data processing engine MapReduce, which processes data sets in batches.

Welcome to The Data Mill, a weekly column devoted to all things data. Heard something newsy (or gossipy)? Email me or find me on Twitter @TT_Nicole.

Next Steps

More on big data and analytics:

Seven big data failures you should avoid

Big data is not about the data

Big data in a bubble is no good

Dig Deeper on Enterprise business intelligence software and big data