Make money doing the work you believe in
One important thing about any of these flows, but especially with medallion so nicely divided into three layers, is that in practice teams end up creating at least four if not five layers. Something like:
-Bronze: raw data
-Silver: cleaned data (e.g. flattened jsons, data type casting)
-”Curated silver”: where teams might apply some business logic or standardization if planning to combine data from multiple domains (these were maybe transformation sql views in the “old” world)
-Gold: the nice star schema with facts and dims aka the storage model aka the old warehouse schema in sql world
-”Platinum”: essentially mart views for reporting, aggregations or anything really for downstream consumption
I am not refuting your article to be clear, I think it was really well written Chris! I am writing down some thoughts on the ideal design vs real implementation from what I have seen working in the “old” sql-warehouse world before and now during the medallion architecture standard. As someone who loves to have things well defined, it really annoys me how much variability there is in teams applying the same idea (can you tell my background is in consulting? :D )
