It tells the history of computing by following J.C.R Licklider. As one of the directors of ARPA, he was responsible for funding research labs to work on computer research. He had a major impact on which projects got funded, and in-turn which systems are now being used 60 years later. I honestly love this book so much. If you love computers and history, its a must-read.
I'm just going to throw out another problem since you guys are here :)
The current batch process joins some tables before sending data to elasticsearch. This means that the debezium connector doesn't write all the data I need into kafka. I was thinking I could create a materialized table in ksql with infinite retention for the other postgres tables I need to join on. Then when I stream in an update for the data I want in elasticsearch, I can join on these tables.
The issue is that a Stream-Table join only gets triggered when the stream changes. This means that when the data in the tables change we will not see these updates in elasticsearch.
I guess my only option is to join everything in our app and then produce the full message to the elastic sink topic?
I just started using kSQL at work to build a data pipeline from Postgres to Elasticsearch. This will be replacing our error-prone batch processes. So far it has been a really nice experience, however I don't have it running in production yet.
Little bit concerned with how we will update the schema of the streams in production. At some point we will need to drop a stream and recreate it with a different schema, and I'm afraid we will miss some incoming data during the downtime. Confluent just released a migration tool, but I'm pretty sure this only works for adding/dropping columns and not changing the datatype of a column.
I have been using CDK at work and it is fantastic. It feels like React for Infrastructure. I used to write raw Cloudformation at my last internship, and being able to write Typescript instead feels like moving from assembly language to C.
At my school there is a 2 unit class that you must take along with the intro DS&A course that teaches bash, vim, git, testing, etc. It was definitely the class that helped me the most in my internship and also made me a Vim user.
This is not an answer but on a similar note, has anyone watched the latest Joe Rogan Experience with Daryl Davis. He is a black man who convinced Klan Members to leave the KKK. Amazing podcast. Maybe the only way to stop toxicity/polarity is to show real conversations with real people.
Look up "edit distance"... it is one of the first problems an Algorithms course will discuss that can be solved by dynamic programming. But ya, no way anyone is coming up with that unless they have seen it before.
I think the fact that Swift has first-class automatic differentiation is an even bigger deal than the fact that it is strongly-typed. Here is an interesting write-up about it...
It tells the history of computing by following J.C.R Licklider. As one of the directors of ARPA, he was responsible for funding research labs to work on computer research. He had a major impact on which projects got funded, and in-turn which systems are now being used 60 years later. I honestly love this book so much. If you love computers and history, its a must-read.