Given that this is a declarative language, have you given thought to creating a means of generating the (imperative) DDL needed to alter tables if the DBML is updated? My team has been storing DDL in our GitHub repo and we have a basic script run by our CI/CD tool to deploy that DDL upon push to master, but it would be great if that could be extended to handle cases where the DDL is modified.
Major League Baseball (MLB.com) | Principal Data Engineer | New York | ONSITE (with flexible remote work up to 2 days a week)
Major League Baseball is hiring a Principal Data Engineer to report directly to me, the VP of Data Engineering, at our New York City office. We work with a diverse set of baseball fan transaction and interaction data, helping to connect fans with the game of baseball! I oversee a team of Data Engineers and BI Engineers and we're looking for someone with significant experience with building and maintaining data pipelines to join our team. More info here - https://grnh.se/c075d4691
I'd argue that AWS's more directly comparable service to BQ is Athena.. Athena is AWS's managed Presto service, but it's positioned as 'serverless' as you don't need to plan capacity in advance, and very 'coincidentally' priced at the exact same $5/TB that BQ is priced at. Has anyone compared Athena vs BQ performance? I work at an AWS shop so we've been kicking the tires with Athena and thus far are pretty impressed, but I do wonder if the exact same workload would perform better on BQ, which has been around much longer and is truly built from scratch for running as a 'serverless' scalable query engine.
Using EMR on AWS makes it as easy as a few clicks in the AWS console to spin up a cluster running Spark and Zeppelin. Plus you get direct access to any data stored in S3 on that same AWS account..
I've seen demos of a tool called Datameer which seems to offer very similar functionality (an Excel-like interface for configuring a job on a small set of data, followed by submission of that job to a Hadoop cluster as a MapReduce job). How does DataNitro compare to that?
This is great, the next feature I'd love to see would be a way to update existing cards via email. The use case: I create a card by forwarding an email to my trello email address. In the meantime, the email thread continues and I have a few more updates on it. Now I want to update the Trello card with the updates from that email thread. If I just forward that email to Trello, it is now creating another new card instead of updating the existing card. Would others find this feature as useful as I would?