It didn't seem like nonsense to me. (Recently graduated undergrad with a math degree; probably could have gone to PhD). It seemed like the AI was cycling through a bunch of different possible approaches to tackle the problem. Eventually it finds one and makes more progress there until it reaches the solution
I've pursued a similar strategy via CDs found in resale shops. I've gotten a good collection going, but there's a very specific selection of CDs popular enough for people to buy but not so good that they kept it.
> For centuries, communities gathered around the fire to hear the same saga recited for the hundredth time. The listeners didn't grow impatient with the familiar opening formulas, the elaborate genealogies, the detailed descriptions of weapons and weather.
I generally agree with the broad strokes of this post, but this description gives me pause. How do we know that listeners didn't grow impatient? Though I suppose it would take a good deal of compression to answer this :)
He's retired, so I'm guessing more about the clout. Or even just "love of the game"? He had a fairly popular tweet thread a couple years back where he wrote out 80 tips for competitive programming -- that feels less likely to be clout based
Cool project! I have a couple of questions that would be nice in the writeup:
* How did you generate your example problems? Did you take an existing benchmark? Or did you have LLMs generate the problems?
* Do you have any thought to adding a second "base programming language" to alter? I'm not sure that there's enough variation as there is. (Another thought would be to generate 4 or 5 different new languages, each quite different, and then run the benchmark on each of those languages? I'm not sure how much the fact that it is randomly generated each time matters that much?)