I'm surprised it is that low. Are not all top AI labs "cheating" and workaround LLMs's low sample efficiency by hiring people to generate more data points - similar problems with answers, so they can train models on those and improve scores? A good benchmark for general intelligence probably should be a complete black box, no sample data given/leaked at all.
As Steve Eisman said: Mag7's businesses becomes capital intensive and they also seems to lack a strong moat, that will make investors more reluctant to buy keep buying their stock.
What does it mean, Is it like when somebody used some coding agent to develop a feature and later input prompts and a resulting PR can be used for training by a presumption that final PR was a correct implementation of a prompt?
> these types of training data might be key to teaching models to reason beyond what they could gather from static data.
I was under impression that every time LLMs try to be truly novel and they need to assume things in the area where they didn't have enough data points that there were trained on, results are not good, has that changed?
That is informative, I was suspecting that is how models improve their performance on some convoluted "non-googlabe" benchmarks like SimpleBench, that is how, they just got the taste of those those questions from publicly available samples and then hired people to generate similar questions and provide answers for them.
I wonder if extracting those static reasoning chains make sense given a Rich Sutton's "The Bitter Lesson" and Geoffrey Hinton's "People should stop training radiologists now.". I guess until participants make money they won't stop, not sure if they do, so far it is more about expectation of profitability as I understand.
> Given the enormous cost of training, would it be worth training new models then?
I'm not an expert, but is this even an option? I mean models must be refreshed with latest knowldge base periodically even without algo/design improvements, otherwise the lag become too noticeable and it will hurt users and their use-case.
> The reported 2025 figures include $7.5 billion in cost of revenue, $19.18 billion in research and development, $5.73 billion in sales and marketing, and $1.57 billion in general and administrative expense
Does training of new models go into RnD or cost? And subscription plans' subsidies, are those cost or sales and marketing?
"Now you see why, when I meet a founder, the first thing I ask about is their growth rate."
This obsession with growth instead of progress or value rubs me the wrong way, given the trend for enshittification of services sooner or later, also remind me of a video I watched yesterday when a founder gives examples of private equity people trying to force growth no matter what:
https://www.youtube.com/watch?v=k4vNIsVY-0Y&t=412s
I thought usually founders try to pivot till they run out of money. I wonder if that is good or bad for a serial entrepreneurs if they decide to shut it down instead of pivoting?
I was also wondering if it will result in Helium shortage and even higher prices the next few years, then/if it will be all over, new fabs might also come online and it will be the opposite - a glut. I believe that follows the historic pattern of boom and bust cycles of chip production.
What type of projects you work on, in particular how rich it is in novelty, non-googlable data points and non-trivial project-specific deviations from industry standards?