I mean, the matrices obviously change during training. I take it your point is that LLMs are trained once and then frozen, whereas humans continuously learn and adapt to their environment. I agree that this is a critical distinction. But it has nothing to do with “meaningful internal structure.”
> Previous generations of neural nets were kind of useless. Spotify ended up replacing their machine learning recommender with a simple system that would just recommend tracks that power listeners had already discovered.
“Previous generations of cars were useless because one guy rode a bike to work.” Pre-transformer neural nets were obviously useful. CNNs and RNNs were SOTA in most vision and audio processing tasks.
There is no epistemological collapse. Access to accurate information has never been so fast nor so easy. To be sure, lies are spread on the internet - but people believed all sorts of bullshit before the internet. Those who want to claim there is a crisis don’t have a principled argument as to how things are worse.
A neat list, but there is a bias toward authors who published one controversial (and not necessarily good) argument that got a ton of rebuttals. Williamson is the worst offender.
The article is rife with references to a mediocre tv show, but it doesn’t contain a single example of the principle as applied to a firm. The author is an expert in something, but it isn’t business.
It’s interesting to consider whether a simulated rainstorm is in fact possible. Not a crude numerical simulation like those used for forecasting, but one fine-grained enough to accurately predict the trajectory of every drop.
In the regions where it works (PNW, Quebec, etc) we could easily build more. The hurdles are regulatory. The regulation isn’t baseless - a dam will affect the local ecosystem adversely. But that’s a tradeoff we choose rather than a fundamental limitation.