Type in a prompt, the more details you add the better. The prompt can also be vague and the result is more random.
The generated plan may serve as an initial draft for a plan. Second stage could be to improve on the underdeveloped areas or cut down on the over engineered areas.
These are zip files, containing mostly json files, some markdown and a csv file. The tasks gets assigned a UUID. Is this a format that makes sense?
I'm using LlamaIndex so it's not tied to a particular LLM provider. I prefer using OpenRouter with the new Gemini 2.0 Flash. And I also like Ollama with Llama3.1 where I can inspect the logs as the code is running.
I'm using Luigi for a DAG representation of the data being exchanged between the many agents.
Instead of e, I have tried other constants such as pi, but it doesn't look as good.
I guess there is another constant that makes the distribution look even nicer.
Seeing the examples while having the editor visible. That's a good idea. I haven't explored this direction, since I had my phone (with tiny screen estate) in mind.
Drafts for a such a UI are much welcome. However I'm probably too lazy to code it though.
There are 100 tasks that is hidden from the public, that is only exposed, when running on an offline computer. So the solver has no prior knowledge about what these tasks are about.
Humans can try the 800 tasks here. There is no time limit. I recommend not starting with the `expert` tasks, but instead go with the `entry` level puzzles.
https://neoneye.github.io/arc/?dataset=ARC
If a model jumps to 100%, that may be a clever program or maybe the program has been trained on the 100 hidden tasks. Fchollet has 100 more hidden tasks, for verifying this.
The short story. I needed something that could render thumbnails of tasks, so I could visual debug what was going on in my solver. However I have never gotten around to make the visual inspection tool. After having the thumbnail renderer, mid january 2024, then it eventually turned into what it is now.
I think the ARC-AGI tasks was manually drawn with an early version of fchollet's editor.
Recently Michael Hodel has reverse engineered 400 of the tasks, so more tasks can be generated. Interestingly it can generate python programs that solves the tasks too.
Replaying the interaction histories, and you can see people have different approaches. It's 100ms per interaction. IRL people doesn't solve task that fast.
https://www.youtube.com/watch?v=vQt7UZsYooQ
What is weird. The way that I implement a solver for a specific ARC task is much different than the way that I would manually solve the puzzle. Having to deal with all kinds of edge cases.
Huge thanks to the team behind the ARC Prize. Well done.
I had not imagined that it would become real so soon.