i’m one of the co-founders at MotherDuck. our team is building the UI in collaboration with the team at DuckDB Labs.
this is a first release. we know there are going to be tons of feature requests (including @antman’s request for simple charts). feel free to chime in on this thread and we’ll keep an eye on it!
meanwhile, hope you enjoy this release! we had lots of fun building it.
Co-founder of MotherDuck here. Let us know if you have any feedback or questions -- we're super excited to release this as a valentine's day present to all those data explorers out there :-) Hopefully you agree it's as eggscellent as we do!
dbt folks actually brought this idea to us because it just makes so much sense -- no translating of sql between dev and prod.
highlights from panel brainstorming from winnie @ dbt:
Local development: should be fast, representative but not comprehensive of prod data, and independent of cloud services if I want that.
The alternative has been that people typically use a schema in their production platform for development, CI workflows, etc. Costly and cloud-dependent.
DuckDB and its role in facilitating the MDS-in-a-box pattern has been a game changer for local data development, which ramped up the ability to share data, modeling work, and analysis the way we share open source tooling, but syntax remained an issue for deployment.
over my career i’ve seen countless cases where the complexity of a distributed system introduces as many (or more) reliability concerns as it solves. when everything works well, it can be beautiful. but when things go wrong, they can quickly become a cascading mess of hard-to-debug issues.
Just to agree with @orthozerox, it's not a 'killer feature' but about lowering the friction.
Lots of users:
(a) mentally align better with SQL than pandas APIs, regardless of whether they know both or not
(b) want decent performance on their analyses, which they aren't getting from pandas, and won't get from many OLTP-databases they're using over-the-wire
(c) want ease of accessing parquet and csv locally and remotely with minimal development overhead. it's super simple with duckdb.
Nonetheless, some of the other things you pointed out are some tradeoffs. We're building a serverless cloud capability at motherduck on top of duckdb in order to address some of these and optimize compute and storage based on data locality, bandwidth and the need for collaboration amongst multiple people.
I'd love to understand the shape of this data and some of the types of queries you're performing. It would be very helpful as we build our product here at motherduck.
I have no doubt that there are situations where the cloud will be faster, especially when provisioned for max usage [which many companies do not]. However, there are a lot of these situations even where the local machine can supplement the cloud resources [think re decisions a query planner can make].
Feel free to reach out at ryan at motherduck if you want to chat more.
Agreed. But how many executives will agree to take these fine-grained actions to achieve value from the data? How many data teams are able to build up a strong-enough argument to convince them?
I've worked on many product-led-growth initiatives in the software industry. The software industry is probably the biggest 'believer in data' there is -- many scientific-forward minds who understand the value. However, even in the software industry, it's really hard to convince folks that if you make 5 improvements that net 1% conversion gain each, you can dramatically improve revenue.
Correct, much data has no value. The cost for storing the data, maintaining the data [in the day-and-age of privacy requirements especially], and combing through the data is often much greater than the value obtained from the data itself.
The expertise we need in the industry is people who understand applications in-and-out and make great decisions on what data is worth keeping for present and future applications. And what data is needed to be kept, but only in aggregates (or anonymized, which reduces costs of maintenance)
Would love to hear your feedback!