Foundations for Architecting Data Solutions

Chapter 6. The Meta of Enterprise Data

Meta is an interesting word that tends to be used in different ways, but in the context of this discussion, it’s information that provides higher-level context and descriptions about some related information. To talk about a domain outside of data, we can look at the video gaming and e-sports world, where it’s common to hear people talk about the “meta” of a game. In the video gaming context, this term is used to refer to a strategy and approach to the game within the game. People who understand the “metas” and can utilize them are far more successful in winning in video games then those who play for fun.

In the world of data, metadata is the information and strategy surrounding data and the use of that data. This metadata will allow additional visibility into things like what data is being stored, what it’s being used for, and who’s using it. In some cases, metadata can be more important than the data itself, and it’s not uncommon for the metadata to outlive the actual data. Effective metadata management is a critically important part of building data solutions, so in this chapter, we talk about planning for and managing the implementation of a metadata solution in your data applications.

It’s worth noting that managing metadata is probably one of the most difficult challenges to address when designing a data solution. This can be attributed to the vast number of sources, ingestion options, storage systems, and access patterns involved in ...