Interestingly costs were mostly ignored in the last years. We had credits, plenty of credits. But now GCP has reduced them and adapted their pricing to the current situation.
Therefore costs become an optimization item. Reducing costs initially is really not that hard, but to find the queries that generate these costs is harder. Therefore I really like Mastheads approach to make these visible.
And big plus - since the work on my metadata, then don't need to access my real data.
1) would a very good solution. It's adding the graph cool pieces to an existing data lake. From there anyone can do what she wants (Athena, Quicksight, Import into Redshift/BigQuery). Would love to see that.
BTW saw your talk today at AWS Summit in Berlin. Good job there.
Therefore costs become an optimization item. Reducing costs initially is really not that hard, but to find the queries that generate these costs is harder. Therefore I really like Mastheads approach to make these visible.
And big plus - since the work on my metadata, then don't need to access my real data.