> (But in case he’s listening, please add Rusts discriminated enums, it really hurts to move from a language with them to a language without them.)
I'm not a programming language expert, but my understanding is that discrimitated enums is jut a way to support sum types. Dart now has the sealed class modifier [1], that fills this role.
This looks really nice. I was reading about this and realized it had similar ideas to this [1] Phd thesis from Jon Gjengset. I checked his twitter [2] and it was based on his work indeed.
Depending on the cost per database, a service like this could allow one to build SAAS apps with Single Tenant model [1] in a really simple and elegant way.
I always liked this model since it gives the best isolation and not having to code with multi-tenant in mind really simplifies development and security. But maintaing an infra for this is hard and costly.
Dan Luu impact at Wave.com is noticable already. Since he joined in March, Wave is pushing one great blog post per week (in contrast to 1 in 2021, 0 in 2020 and 3 in 2019).
I tried creating this culture of blogging but failed, would love to hear strategies that worked to achieve this.
`OrioleDB implements default 64-bit transaction identifiers, thus eliminating the well-known and painful wraparound problem`
I thought that PG's 32-bit transaction identifier and wraparound issue was due to the current MVCC implementation that keeps old versions of rows in page tables, so there is the need to vacuum to clean up the dead rows. So my assumption was that using another approach for MVCC, like the UNDO log, this problem wouldn't exist (since page tables would only have the latest version).
I once started building as a side project something similar but focused on querying cloud resources (like S3 buckets, ec2s, etc... discovering the biggest file from a bucket was trivial with this). I abandoned the project but someone else built a startup on the same concept - even the name was the same: cloudquery.
I built it using the multicorn [1] postgres extension and it is deligthful of how easy it to get something simple running.
They mention this in the article. But to sum up, each connection in PG is handled by its own OS process. Postgres behind the scenes is composed by multiple single-threaded applications.
This comes with the advantes for Pg developers (and us!) that they don't need to deal with tons of data races issues, but the trade off is that memory wise, a process takes way more memory than a thread.
Comparing Fauna with DynamoDB and Redis gives the wrong impression. Trade-offs are totally different.
However, the product in question (Upstash), looks very interesting: serverless storage with a pay as you go model with a maximum price cap is an excelent idea.
I would love to have more options in this space, especially in GCP to use together with Cloud Run.
Apache Drill is an interesting project, from all the MPP engines that appeared a few years ago, it was the most similar one to BigQuery (the first public version) and the most flexible.
However, the competion was fierce and each Big Data vendor (MapR, Cloudera and HortonWorks) was pushing its own solution: Drill, Impala and Hive on Tez. Competion is always a good thing, but it fragmented the user base too much so no clear winner emerged.
At the same time, Spark SQL got sufficiently better to replace these tools in most use cases and Presto (from Facebook) got the traction and the user base that none of these projects had by being vendor agnostic (and its adoption by AWS in Athena and EMR also helped boost its popularity).
If you have mongodb enterprise you can use Mongo bi connector and connect it with any bi tool. I recommend metabase since it is open source and very good for slice and dice and reports.
The mongodb bi connector is a postgres with foreign data wrapper to mongodb. So even if you don't have a mongodb subscription, you may be able to pull this off using the OS version https://github.com/EnterpriseDB/mongo_fdw
In Germany, Takeaway [1] bought most of the competion and the small players quit (like Deliveroo) since they couldnt compete... well looks like this is actually working:
The client handles horizontal scalling, checkpoiting, shards split and shards merges. Using just the SDK, you have to build this yourself (unless you are using Kinesis for use cases that dont need it to be done correctly).
And in the second paragraph of this documentation:
"These examples discuss the Kinesis Data Streams API and use the AWS SDK for Java to get data from a stream. However, for most use cases, you should prefer using the Kinesis Client Library (KCL) . For more information, see Developing KCL 1.x Consumers."
Because the official client from AWS is written in Java.
It is possible to write clients in any language, however, it is not that simple especially when you need to handle logic of scalling out or in your kinesis stream (that will split or join shards) and when you have multiple consumers in the same consumer group (you will need a distributed locking mechanism and a logic to steal locks if one consumer dies).
I migrated a couple of projects from Java, TS and Go to Rust and honesly, I couldn't be happier.