Andres Rodriguez - Fotolia
Of late, data and data ownership have been on my mind. There are several things causing me to ponder data:
- Data privacy: There are new and pending laws about what we do to keep our client data private. I have some clients who have told me that their data can never enter certain geographies.
- Data volumes: We are collecting and swimming in massive amounts of data. Many of us now simply plan on expanding our storage capacity and costs every year.
- Information security: If our data is an asset to us, it might be an asset to others -- and those others might not have altruistic intentions with our data.
- Application modernization: As we update and improve our legacy applications, we uncover interesting issues like replicated data (which requires synchronization), data validation and which data represents reality. In practice, this asks the question, who owns what data?
In other words, data management is a big deal. And over the past few months, my focus has been on data ownership and governance. Here is why.
My software engineering teams and operations teams have been in a bit of a clash over ownership of our data. The conflict started when the software engineers -- as they developed new code -- began creating their own data stores. The members of the operations teams felt that the data belonged to them -- all the software engineering teams had to do was request a data store and operations would take it from there. But, countered the developers, why should we wait for operations to create support for data that belongs to the application we developed? In an age of "DevOps" and rapid deployment, why require anyone to navigate through the operations processes?
To be honest, I was a bit torn, as I could see both sides of the dispute. We wanted to be agile, nimble and fast, but I also did not want to create new data stores -- attached to an application module -- that replicated data that already existed. Why not just use that data and avoid the issues of data synchronization and validity?
I did what I always do when I have no idea what to do -- I pretended to have a clue by telling the teams that I knew exactly what to do about data ownership but felt that they were best served if they figured it out for themselves!
Core vs. application data
I locked the warring teams in a room in something of a steel-cage death match and told them they could not come out until they had figured it out. To my surprise, it did not take them very long to develop their plan. They did this by creating something of a data governance manifesto. Their manifesto identifies these principles:
- Our data is an asset and must be both available and owned.
- Our data falls into two categories:
- Core data. This is data that is universal and must be governed centrally -- our examples include customers, orders, identities, roles, permissions, products. The operations team provides and supports core data.
- Application data. This is everything else and its use and ownership are local. Software engineering teams own and manage this data.
- Users interact with applications. Applications consume services. Services respect the data and don't do anything to hurt or corrupt the data. Only the services can touch databases.
In other words, we classify our data into these two general categories (core and application) and clearly define the owner of each category. This provides us a loose-coupling that supports standards and standardization while also allowing the flexibility that the development teams need in order to be nimble. If an application needs core data, the operations team makes sure that the core data is available and consistent.
We still have plenty of work to do on our data -- retention periods to reduce data volumes, information security to protect our data and data privacy to make sure that our customers retain some control over what we do with their data -- but I feel as if we are on the right track with what has been our most contentious element of data management.
About the author:
Niel Nickolaisen is CTO at O.C. Tanner Co., a Salt Lake City-based human resources consulting company that designs and implements employee recognition programs. A frequent writer and speakeron transforming IT and IT leadership, Niel holds an M.S. degree in engineering from MIT, as well as an MBA degree and a B.S. degree in physics from Utah State University. You can contact Niel about his column on data ownership at firstname.lastname@example.org.
Did you enjoy this column on data ownership by Niel Nickolaisen? Here are some of Niel's recent columns:
Don't use BP tools as a dodge for bad workflows
Mobile is forcing CIOs to investigate next-gen security tools
The urgent case for consolidating your app portfolios