I'm the CPO at a SaaS data catalog and analysis hub company called data.world - I was at the Snowflake Summit when the Looker acquisition by Google was announced - the guys at their booth seemed pretty happy ;)
With Google snapping up Looker ($2.6B) for Google Cloud, Salesforce's much bigger purchase of Tableau is a clear sign that the big guys see buying BI tools is a good way to expand the reach of their offerings into more of the business. We talk to companies every day that have made massive investments in data warehouses, viz tools, and high-paid data scientists, but they still aren't agile enough because they can't tie it all together so their people can find and use the right data when they need it. I read an article once about the failure of self serve BI and the reality is that you just end up creating more sprawl. People need tools to reduce the clutter and sprawl and stop the endless chain of emails trying to figure out what table to look at or query to use or if your source is still updating.
We built our data catalog and analysis hub for exactly this, and it's extremely validating to see the big guys like Salesforce and Google investing in expanding the user base of big data tools and I really hope we can be part of the solution of sorting it all out!
I’m one of the cofounders at data.world and you definitely make good points… We encourage all users to post a license with their datasets but, like open source, not everyone will maintain or honor these. This is definitely something that we feel we can help the open data community with and encourage even more. (As an aside, in general you should note is that a lot of data scraped from websites is something that you’ll need to be careful with).
When it comes to data quality, the world is a messy place and the data that comes from it is messy too. Most professional data scientists spend an inordinate amount of time cleaning datasets, doing feature engineering etc… it’s part of their job description. We’re trying to eliminate some of that repetitive work by making sure that people can comment on, contribute to and give some signal back on the quality of the dataset. We also think a dataset is more than just the data... on data.world you can upload code, Notebooks, images, etc... anything that helps add context to the data.
Finally, when it comes to size... ML definitely needs it. However, there's a lot of interesting data out there thats still very complicated, very useful but not that big. Most datasets in the world are well under the terabyte size (or even 100s of GBs in size). We're rapidly expanding the size of datasets we support because we want that stuff too but we really want to help people understand all the data in the world!
With Google snapping up Looker ($2.6B) for Google Cloud, Salesforce's much bigger purchase of Tableau is a clear sign that the big guys see buying BI tools is a good way to expand the reach of their offerings into more of the business. We talk to companies every day that have made massive investments in data warehouses, viz tools, and high-paid data scientists, but they still aren't agile enough because they can't tie it all together so their people can find and use the right data when they need it. I read an article once about the failure of self serve BI and the reality is that you just end up creating more sprawl. People need tools to reduce the clutter and sprawl and stop the endless chain of emails trying to figure out what table to look at or query to use or if your source is still updating.
We built our data catalog and analysis hub for exactly this, and it's extremely validating to see the big guys like Salesforce and Google investing in expanding the user base of big data tools and I really hope we can be part of the solution of sorting it all out!