I am using a model that runs on a 100 GB300s, near AGI, all bets are off, $10B training run on a 1GW cluster, and it can't realize that when I told it to "please implement v2 of feature X" that I mean delete v1, not support v1 and v2 together, in a weird Frankenstein's monster of the two. Sorry, but I think my job is quite secure for the forseeable future.
If there really was a "simple" solution to Fermat's Last Theorem, Andrew Wiles wouldn't have achieved the important result he did, ending up making connections across disparate fields of math.
The LLMs "sweeping up" easy, or previously missed, results seems like a net negative. It's probably better for humans to struggle and come up with new tools than to just "clean up" low hanging fruit that doesn't add much value to the field.
From CoreWeave, at current prices (~$2.46/hr spot to ~$6.16/hr on demand) would correspond to $22M–$55M.
The dataset is really where the cost is though - they used LVD-1689M - 1.6B images of curated web data from roughly 17B instagram images. This probably cost a huge amount of hours in human annotation, compute for algorithmic filtering, etc and not to mention probably a 20-50 person team working on this model.
You might want to change assumptions about how expensive these models are.
I remember saving up for a year to buy the ATI Radeon 9600 XT (I think it was $200 MSRP) so I could play the game on high settings. Now we can play it inside a virtual machine on a crappy laptop. What a journey
Always pains me to see all this innovation and cool software work being applies towards making a machine that extracts money from retail investors and makes a few people extremely wealthy. What a depressing and colossal waste of time.
Next time there is a fire at your house I will say "he's an adult who should have been careful playing with dangerous things like fire, we shouldnt waste society's money and resources on saving his house"
That part should be non negotiable - the tech companies should be funding the energy infrastructure. My bet is that they'd happily do it. They are starved for space and energy, not capital.
Can someone explain to me why local governments are so against datacenters? It seems like a golden opportunity to build electric infrastructure that's paid for by corporations and if AI is a bubble at least that infrastructure will remain and continue to provide cheap power.
And the reason for _that_ is because of the callous way American society accepts the deaths of thousands of people who die due to the Healthcare Industrial complex (of which Brian Thompson was a key member of). Just because those deaths don't happen with guns doesn't make them any less important.
We will be old, not boomers. Boomers are a special generation at a special moment in world history - they made decisions based on the limited amount of knowledge they had about how the world works and while some think those decisions have doomed us forever, I remain optimistic.