Throughout history, people have believed that new inventions would solve complex social problems. The telegraph was supposed to end war, the telephone to cure loneliness, and the automobile to put education within everyone’s reach.
Techno-Optimism Archive collects clippings from old newspapers and magazines that capture moments when new technologies were met with high hopes and confident predictions about the future.
Author here. Thank you all for the comments. I take full responsibility for stupidly using an image for posting the code snippet. Sorry for that! Also, the article was originally posted almost 2 years ago (and "resurrected" with the recent migration to Medium). This is why a fairly old DuckDB version is referenced there. Some of the issues I observed are now gone too.
Obviously, many things have changed since then. We've experimented extensively and moved back and forth with using DuckDB for our internal cloud processing architecture. We eventually settled on just using it for reading the data and then handling everything else in custom workers. Even using TypeScript, we achieved close to 1M events/s per worker overall with very high scalability.
However, our use-case is quite distinct. We use a custom query engine (for sequence processing), which has driven many design decisions.
Overall, I think DuckDB (both vanilla and WASM version) is absolutely phenomenal. It also matured since my original blog post. I believe we'll only see more and more projects using it as their backbone. For example, MotherDuck is doing some amazing things with it (e.g., https://duckdb.org/2023/03/12/duckdb-ui) but there are also many more exciting initiatives.
I think just as with full-text search, vector search, if supported, will be full of tradeoffs for general purposes databases.
The view that everything needs to support direct input for generative AI is short sighted. There are other use cases as well. Even if ultimately these will become just building blocks for whatever AGI there comes. Horses for courses
I feel that logs are around for so long that it's easy to take their capabilities for granted and not go much further. This is another example there's more that can be done. It reminds me rfc5424
At LogSense.com we actually tackled this problem too and came with automatic pattern discovery that pretty much converts all logs into structured data. I actually just posted it here: https://news.ycombinator.com/item?id=20569879 I am really curious if this is something that you consider helpful and any feedback is very welcome
There were 41k flights until May 2018 [1]. I was not able to find the current number but with 350 models delivered so far and 4 average flights per day that would sum up to ~500k flights in total at most (but more probably half of that due to ramping up the deliveries).
With 2 accidents the rate is now 4/million. That's at least an order of magnitude difference more than the average of the modern jetliners [2]
Of course, with two events it might be just bad luck but the similarities are concerning, as pretty much everyone observes here
> Predictable runtime behavior (zero cost abstractions and no garbage collector).
I see a lot of people considering Rust/Go because they are looking for something much more modern than C++ and at the same time not running on JVM due to the above
Techno-Optimism Archive collects clippings from old newspapers and magazines that capture moments when new technologies were met with high hopes and confident predictions about the future.