Anthropic researchers discover thinking longer sometimes makes models dumber(venturebeat.com)
venturebeat.com
Anthropic researchers discover thinking longer sometimes makes models dumber
https://venturebeat.com/ai/anthropic-researchers-discover-the-weird-ai-problem-why-thinking-longer-makes-models-dumber/
4 comments
Also see the paper "Inverse Scaling in Test-Time Compute": https://arxiv.org/abs/2507.14417
I wonder if this is similar to the idea that when your taking a test you should most often go with your initial answer, and not change your answer.
No more "ultrathink", got it :)
For me when I use an LLM for coding the value and accuracy comes from the conversation, I give context, I evaluate the response, I give more context and prompts, sometimes I correct it and get it back on task, and ... we get there (well not always but generally).
The idea being our interactions are what gets us towards the answer I was looking for / the "right" answer.
I imagine that without me being in the loop and the LLM given lots more time it's going to take those original words of mine, do it's "word math" (I like to think of it that way) as much as possible and ... maybe go down a rabbit hole that I wasn't headed to, possibly way down.
Is that rabbit hole kinda scenario what they're talking about?
Is it also possible that because of their training data, Q & A and Q and context and Q and A is just the better path than deep thoughts that produced some of that content?
I also wonder maybe just because I'm a simpleton, and for me the "right" answer really was the simplest first.