YES!! Normalization was the BEST PRACTICE since DBMS was invented in like the 70s, and somehow, we just totally forgot about it the past 10 years ago for OLAP. 10x more expensive storage, impossible schema evolution because of data backfilling, the extra pipelines you have to build and maintain, and the cost scales with the data size and your business growth. I was just talking with a buddy of mine about this and how to run JOINs on the fly without denormalization pipelines using StarRocks in this case.
Maybe they should rename it to their migration options page. Or maybe I'll just ask ChatGPT what the best alternative is...
Still, pretty useful stuff, but it also feels like Rockset had been moving a little too slowly in recent years, but congrats to them on finding a new home.
Funny that I hadn’t heard of them in the database space till they showed up at the top of ClickBench. Makes me wonder what other interesting projects I’m missing out on in China.
Apache Kylin was actually China’s very first top-level Apache project. It did come out of eBay, but the work all originated in China. It’s a really cool solution to query acceleration.
Very cool to see this! Kylin has been a super fun project to b a part of. A unique approach to OLAP/query acceleration that's pretty much the best way to deal with huge volumes of data.
It's great to see them collaborating with Women Who Code to get the word out and grow our open source family. If anyone is interested in learning more about the project, check us out here: http://kylin.apache.org/
Thanks for sharing this! Apache Kylin is an awesome project. It has been around for a few years and has been getting a lot of attention in Europe and Asia, but still seems to be relatively unknown by a lot of folks I talk to in the US.
If people are interested in getting involved with the community, or just want to check Kylin out, you can find the page for the project here: http://kylin.apache.org/
Kylin is a really cool project, and how it's using OLAP for faster analytics on large datasets/big data is interesting - especially given how old the technology is. The Kylin community has found some creative ways of breathing new life into OLAP. If you'd like to join the project you can learn more here: http://kylin.apache.org/
Kylin is a super cool community. Very supportive and doing some neat stuff with OLAP. If anyone is curious and wants to join us, you can read more about the project here: http://kylin.apache.org/
Very cool, thank you for sharing. I've been a part of the Kylin community for over a year now and it has been a great experience. The project is really pretty cool. I know some people think of it as just another OLAP engine - but it is way more than that.
If you're part of a team working with huge datasets, take a look at it, and if you're looking for a great open source project to get involved with, join us!
I know that the team behind the initial creation of Kylin launched a commercial product they're calling Kyligence (https://kyligence.io/), and they seem to have gained a fair bit of traction in Asia. I've also seen them showing up more and more here in the United States.
Well, for one I’d really love to see even more improvements in recognizing supply usage. As near as I can tell there is nothing out there that’s doing the job. Most inventory management still requires someone in the lab to track and report. There’s a lot of barcode-based solutions where folks scan stuff out, but people are so busy that this either never gets pick-up in the lab or not enough people commit to it and you might as well not even bother with the system.
Something that could track the removal/use of products without needing so much handholding or constant verification would save so much time and avoid so many urgent orders.
Also, something that ties together ELN work and related supplies alongside inventory, and that adjusts inventory levels and helps with reorder alerts at particular thresholds. That would smooth a lot of the pain around planning. Like if something could alert me that a reagent/consumable/chemical was out or low before I was about to commit to a project it’d sure avoid a lot of urgent orders which can be a huge cost driver due to shipping costs or having to pay more than average because only one vendor doesn’t have something on backorder or they’re the only ones who can ship in time to keep things moving.
HappiLabs is right that pricing is all sorts of messed up across the sciences. That anecdote about labs in the same building having different pricing is all an all too common story I hear from plenty of folks in the lab.
I see time getting wasted every week, and not just from folks managing the lab. Often, as soon as the weekly planning meeting ends even scientists will get out their laptops and start bouncing between VWR, Fisher, Sigma, etc's. websites to figure out where the best prices are, how long it'll take to get what they want, and shipping costs. The fun part is when the stuff arrives and they realize their inventory count was off and they're actually out of something they thought they had in stock...and then it's back to the same old websites.
It's good to see YC investing more towards solving this problem. I know Quartzy (YC S11 - https://www.quartzy.com/) has been working on this problem too, a bunch of labs I work with are using them for this same issue. They have a ton of partnerships with suppliers which has allowed them to consolidate all those vendors into one place. This has solved a lot of that price hunting, but I think there's plenty of room to expand with more automated inventory management since that's really at the heart of a ton of supply issues.
Honestly, thinking about it, there's a lot of stuff in the lab that automation would help with beyond the experimentation part which gets the majority of attention at the moment.
Working with labs, I've found it extremely eye-opening how archaic the supplier relationship is. A big challenge appears to be with the sales model and its reliance on contracts, opaque pricing, and exclusive relationships.
This is, of course, made even more complicated because not all supplies are created equal. Even simple plastics can have slight variations that are invisible without further testing which can impact experiments. You have to be careful, and many of the scientists I work with are superstitious when it comes to buying products, not wanting to risk their experiments.
An alternative I see a ton of labs using these days is Quartzy (YC S11) (https://www.quartzy.com/), which also provides great price alternatives, and also evaluates against other parameters people care about when it comes to their experiments. They have an entire team dedicated to vetting these products so scientists can order with peace of mind. Quartzy also provides some very useful supply ordering and inventory tools that are a huge boon for lab ops.
There's a lot of opportunities for disruption in the lab supply space. It's definitely needed.