A blast from the past. When Taskrabbit was acquired by IKEA, I built several tools that went through the whole catalog via various crawling approaches. One tool was to estimate how long it would be to put each item together for an initial training set.
I am excited about the Rails defaults where background and cache and sockets are all database driven. For normal-sized projects that still need those things, it's a huge win in simplicity.
Ario | Onsite in Palo Alto, CA | Full-time | heyario.com
I recently started at Ario where we are building an app for parents that uses AI to make things easier. We are targeting saving them one hour every day.
Previously, I co-founded Taskrabbit. That was somewhat similar but now it's time to get LLMs in on the action!
It's in the app store, but still early in the game, so we are looking for engineers to build it out. We are using Python and React Native.
We have to work on the scaling. There is lots of web scraping and LLM calls that we need to make sure works under load. I'm sure there will be quality improvements as well.
Just came to say, I still think this is the best balance between the many factors of running a dev team. I keep trying to recreate it in every tool I use.
Airbyte acquired our Reverse ETL company, Grouparoo, 1.5 years ago. There is so much to solve making just the Extract and Load work well and so much value that comes from that, we have been busy there. I'm excited to circle back to publishing next year.
I like how the article notes that the stuff we were talking about with Reverse ETL (mostly activating your data in SaaS systems like Salesforce, Zendesk, etc) is one important part of Publishing. But we are also seeing traditional use cases like file uploads and new fancy stuff like vector databases.
I built a system for TaskRabbit that scraped all the IKEA products from a variety of sources and ran algorithms to determine their category and predict how long they would take to be assembled. Then there was a Mechanical Turk sort of system for human input. When combined with real-world feedback from the Taskers, it was pretty good.
For better or worse, I've personally been through the entire catalog multiple times.
At Grouparoo, this is a primary use case. We have a UI that engineers use locally. This helps gets things right. It outputs a JSON configuration that is checked in. When that is deployed, it does all the syncing.
There are probably some nuances one level down. Things our users have told us they can do in these areas that, to my knowledge, Hightouch doesn't do:
* Combine data from different sources to define a model. We'v seen using Postgres as a source of truth and supplementing with Snowflake data, for example.
* Add tags to contacts in mailchimp, zendesk or make lists of them in customer.io, Pardot, etc based on segmentation. I believe Hightouch Audiences is more like a filter.
* Full workflow with branches, PRs, test suite in a repo. I saw Hightouch added git syncing to a known branch yesterday and it looks cool, but it's not the full workflow yet.
I'm certainly trying to keep it in the friendly-competition area, especially on this thread :-)
Congrats on the launch! Hightouch looks great and this need is real. Things seem to be going well, so I don't think I'm taking too much away by mentioning that we have been been working on Grouparoo, an open source alternative that solves similar pain points.
A few differences: git developer workflow focused (branches, CI, PRs, etc), ability to self host, segmentation in destinations (tagging people in mailchimp based on rules, for example)
We just finished up the Open Source Data Stack conference, which is all about this topic. Feel free to check out the reply.
Specifically, open source approaches to the modern data stack where the trend is picking the right tools for the job that revolve around the warehouse central data store.
The pieces discussed were around getting data in (Snowplow events, Meltano ELT), transforming it (dbt), reporting (Superset), getting it back into tools (Grouparoo Reverse ETL), and orchestrating things (Dagster).
It's certainly not production situation, but I quickly made this [1] a while back to help with "watching" certain queries for debugging. It uses a polling approach.
We can use any query to bring in and are considering various ways to process it after that. That's in the near term roadmap.
The main kind of "processing" that's done now is using all these properties to calculate cohort membership (High Value users) so that all these tools can use it: Zendesk to route tickets, Marketo to trigger a campaign, even the product to change their dashboard.
There are usual suspects for sources. Databases (MySQL, Postgres) and data warehouse (Snowflake, BigQuery, RedShift).
Though data out is more common, we can also bring in data from any given SaaS tool as well. For example, we have a Mailchimp _source_ that will pull in people as they signup through their form.
There is a plugin model and a few Typescript interfaces to implement to be either a source or a destination.