EVENTS

A common pattern in data lake and lakehouse design is structuring data into zones, with Bronze, Silver and Gold being typical labels. Each zone is suitable for different workloads, different consumers, and different use cases. This prompts the question: which layer is best suited for applying data quality rules and actions? My answer: all of them.

In this webinar, I’ll expand on my answer by describing the purposes of the different zones, and mapping the categories of data quality relevant for each by assessing its qualitative requirements. I’ll describe Data Enrichment: the practice of making observed anomalies available as inputs to downstream data pipelines, and provide recommendations for when to merely alert, when to quarantine data, when to halt pipelines, and when to apply automated corrective actions.