1. You have given up on any sort of consistency.
2. The database implementation will never be able to optimize a lookup workload which is pretty close to pessimal. Not caring about 99% is only viable if can actually avoid the other 1%.
what is a document? How is an ORANUM or a bignum not a document?
One motivation for creating documents is that modeling document contents as relations requires the creation of a bunch of primary keys which no natural definition. A simple document might be an ordered collection of paragraphs, [p23, p57, ...]
Modifying such things is difficult. In fact, the most effective way of structuring modification seems to be OTs based on document offsets. What Google docs does.
Been doing this for nearly 40 years. No sign of doom. What I think you're discovering is just that there is a large amount of low-aspiration work out there. That is as true of FANG as anywhere else.
IMO specializing in any particular bit of trendy tech. is a recipe for disaster. Trends come and go. First-principles CS and being able to communicate it continues to be extremely valuable and satisfying.
PathQuery was started by Warren Harris. At Metaweb, Warren did a rewrite of MQL in OCaml which we never shipped because of the Google acquisition. He spent much of the first year at Google working on a prototype of PathQuery in Haskell. The prototype was rewritten in C++ and used in the first KG-serving infrastructure sometime around 2012/2013.
I am a huge fan of Frank McSherry's work and don't necessarily agree with the premise that DD is somehow failing. However,...
Batch data processing is very well understood, cheap and getting cheaper every year. So, if you can afford to boil the ocean every night, DD is a tough sell.
The addressable market, customers with problems which can only be solved with DD (instantaneous exactly correct answers) is probably small right now.
>Deep reading has in large part informed our development as humans
As a statistic, reading is only significant in the past 200 years or so. Given that there was a similar kerfuffle a couple thousand years ago when we shifted from memorizing to writing things down, I doubt that this is too significant.
Literacy is also shifting. Growing up with memes and Instagram and Tiktok has produced a generation that is vastly more literate in design, photography and film.
>I can't wait until I have enough money to be financially independent and never have to do this nonsense again.
There are a lot of people operating on that plan. Mostly it doesn't work out for them.
Play this forward ten years with a family to support and a mortgage which committed you to personal inflation and a track record of ...? You can absolutely fool people for a week or a year, but a decade of non-experience will stick out like a sore thumb. Experience compounds, and the one you describe is compounding down. Do this for ten years and you will be much less accomplished than you could be.
Concurrency is hard because we invented a bunch of theoretical machinery (semaphores, mutexes, condition variables,...) and embedded it in languages such that most of the programs one can express are bad: racey or deadlock prone. Compare with modern sequential languages which have near-perfect denotational semantics: anything which compiles means something.
If you want practical guidance, my best is to avoid all of the concurrency constructs that are taught in schools. Use log-structured single-writer multiple reader, wait-free, processes and shared memory.
Structure and Interpretation of Computer Programs is still the best single book on computer programming: the most important concepts (closures, continuations, macros, transformations,...) for the least expense in complexity.
To what you learn from SICP, rust and C++, add just one thing: how to exploit a physical stack efficiently. Scheme->C++/rust will be easy. C++/rust->Scheme can still be quite difficult depending on how distracted you got by the complexity.
As a practical matter, most system software is written in C. Being able to read the Unix kernel has been valuable for nearly 40 years. You'd be surprised what one can accomplish with just functional decomposition.
Don't despair. Datalog is alive and well. Yes, it does compose beautifully. However, with composition solved, you'll discover that a naive implementation of relational algebra will suffer from spurious cross products.
"Technical debt" is the difference between the code that you have and the code that you would write if starting over. Does your final code reflect what you learned while writing it?
"Technical surplus" is your ability to learn and implement that learning as code.
The "bullshit" part is complaining about debt while not being able to run a surplus.
Upgrade aversion is the natural consequence of the automatic upgrade subscription business model. If developers don't have to sell the next version and the customer can't easily leave, ...
So, imagine how things are going to work out when you have to explain to a VP that you have absolutely no idea why the site was down for 4 hours, but you have now added logging such that when it happens again you might be able to debug the problem...
Log as if your life depended on being able to debug a single occurrence. If this causes a lot of output, you probably need to add more assertions to the code.