I'm afraid I'm even balking at the word "pioneering" in context with US frontier labs. They are probably doing a few new things, right, but they are not blazing any trails for others to follow along, the Chinese are.
They can only pay so much because of the expectations about the generated value. Those might turn out wrong, as any expectation, but value is very much in the picture.
Not the same person here, but a data point nonetheless. Before supplementing D3 I had a cold basically every year, sometimes twice a year. Since I brushed up my levels I average 1 cold in 6 years.
To me it appears as if the study using the constructed dataset was a completely different one than the one that was concerned with AI.
For the AI study real data from "3.4 million people who submit 4 million job applications to 1,700 job postings across 150 employers and 11 industry sectors" was used.
I wonder how personality forming it is, being a curious kid growing up hacking on computers. If you don't get what you expected, it's almost never the computer's fault - it means you did it wrong, and need to reconsider. There's no excuses and no dumping responsibility on anyone or anything else.
I have the feeling it probably teaches you something, or at least it should. Something not too unlike epistemic humility, maybe.
It would still degrade it's effectiveness, which is what they claim to want. Exaggeratedly: If it wasn't so, you'd just need fundamental math in the training data, as everything else can be derived.
The question is: If biological, computer security, and ML research are so bad, why do they even train on the relevant data?
The only answer that makes sense is they wanted the model to be competent and usable in these fields, just not by you, which is why they had to bolt on a badly functioning crippling device after the fact.
I blame it on the art world being full of pseudo-intellectual little shits, so of course they get off on the Marxian esthetic of throwing around signalling phrases like "late capitalism". They live and die by pretense, there just isn't anything more than that to these "people". All facade, nothing inside. The average LLM has more soul than these types, I fear.
There is actually a way to get really amazing sample efficiency out of a learning setup, and that's engineering in a load of appropriate inductive biases, which personally I am convinced evolution has done for us. Explains a big chunk of the "how are brains so sample efficient" problem really easy, but unfortunately without handing us an easy way to replicate it, which makes it unpopular. Also, it's something that we don't really want to do in the same way evolution has, as all those biases do even further reduce sample efficiency for all the things for which they are not appropriate.
Thoughtful comment, and I'd like to add another angle: However meaningful it is to say the community is divided, I also think that individuals are "divided" on the question as well.
I can speak from myself as an example (although n=1): I am incredibly open to machine learning and the advances it brings. On the other hand I am extremely conscious of the fact that the current LLMs do often write bad code, which becomes especially obvious once projects go much beyond "private toy" size. For me personally the consequence is that I try to make my projects even more modular, and the modules even more clearly delineated. With proper guidance, current LLMs work mostly fine when working on isolated modules. That doesn't mean they don't sometimes fail even there, but they also on rare occasions come up with surprisingly clever solutions when you let them loose on a code base, as long as the problem you want them to fix is mostly isolated in a couple of modules.
So long story short, you can be all for LLMs and still be conscious of their shortcomings, and that just vibe coding applications you intend to let loose on unsuspecting customers is probably a bad idea, and possibly outright immoral. We have already seen more than enough examples of vibe coded applications dumping sensitive user data into the lap of anyone who is inclined to pick up a stick and prod them a little bit.