Data lake governance is crucial to big data management

Data lakes, which are storage repositories and processing systems like Hadoop that can hold massive amounts of raw data, have become vital to big data management as companies look to exploit ballooning big data of various shapes and sizes for actionable business intelligence.

Download this free guide

PDF: What does "digital transformation" really mean?

What exactly is digital transformation? You may hear the term often, but everyone seems to have a different definition. See how our experts define digitization, and how you can get started in this free guide.

Start Download

• Corporate E-mail Address:

  You forgot to provide an Email Address.

  This email address doesn’t appear to be valid.

  This email address is already registered. Please login.

  You have exceeded the maximum character limit.

  Please provide a Corporate E-mail Address.

By submitting my Email address I confirm that I have read and accepted the Terms of Use and Declaration of Consent.

By submitting your personal information, you agree that TechTarget and its partners may contact you regarding relevant content, products and special offers.

You also agree that your personal information may be transferred and processed in the United States, and that you have read and agree to the Terms of Use and the Privacy Policy.

Such companies include Colony Brands, a direct retail marketer in Monroe, Wis. According to Steve Cretney, Colony's CIO, the majority of the company's hundreds of terabytes of storage sat idle in cold storage, while only a small fraction was used for business analysis. While historically a company might have approached this discrepancy using a data warehouse, Cretney went one step further: He moved Colony's data warehousing function to the cloud -- and opened up the company's data with a data lake.

Like Colony, other companies are turning to data lakes as a cheaper and more flexible way to leverage complex data when compared with traditional data warehouses and extract, transform and load tools, say the experts that Senior News Writer Nicole Laskowski spoke with for the cover story of this month's CIO Decisions.

"Data warehouses and databases … are too expensive and too constrained by storage and performance," said Phil Shelley, adviser and director at DataMetica Solutions in India. But with tools like Hadoop, "we can bring in all of the [historical] and new data, in near-real time, without the use of legacy ETL tools, into one single place," he said. Plus, data lakes' lack of data structure offer flexibility to data scientists because they can analyze data without a predefined schema, according to Joe Caserta, founder and president at Caserta Concepts in New York.

But despite the many benefits of data lakes, they also come with risks -- including losing meaningful context or valued customers -- if they are not properly governed. Data lake governance includes not just data cataloging and indexing and metadata management, but also overall governance, such as pinning down who is in charge of ingested data and properly defining data.

Dive into our cover feature to find out how CIOs can implement the right amount of big data management and data lake governance while still leveraging the flexibility data lakes provide. Plus, learn more about the semantic database, a type of data lake that uses the same standards and technology that is used to create Internet hyperlinks.

Email Francesca Sales, site editor, or find her on Twitter @Fran_S_TT.