Early Evals for OpenAI O3(openai.com)
openai.com
Early Evals for OpenAI O3
https://openai.com/12-days/?day=12
4 comments
[deleted]
Wow, the demo where the user asks for untraceable payments shows some pretty sophisticated reasoning. The word "crafty" comes to mind.
New SOTA's on:
SWE-Bench - 71.7
Competition Code - 2727
ARC (Semi Private Eval) - 75.7 on low, 87.5% on high compute
Frontier Math (previous SOTA was 2%) - 25% on high compute
SWE-Bench - 71.7
Competition Code - 2727
ARC (Semi Private Eval) - 75.7 on low, 87.5% on high compute
Frontier Math (previous SOTA was 2%) - 25% on high compute
[deleted]