GPT-4 doesn't pay close attention to detail in some cases(github.com)
github.com
GPT-4 doesn't pay close attention to detail in some cases
https://github.com/openai/evals/pull/272
1 comments
The trick with the F is a good one. Yes, tokenization is very likely one of the contributing factors.
The point of the eval is an attempt to exercise: attention to detail, tokenization, ability to stop, ability to follow instruction and ignore irrelevant instructions, biases, laziness, computational limits, etc.
The point of the eval is an attempt to exercise: attention to detail, tokenization, ability to stop, ability to follow instruction and ignore irrelevant instructions, biases, laziness, computational limits, etc.
> Here is the sequence in which the actors act: MAMA.
I didn't even get this on the first reading.
It's like that brain teaser "count the number of Fs in 'Finished files are the result of years of scientific study combined with the experience of years'" (https://www.researchgate.net/figure/How-many-Fs-do-you-see-i...). It shows that GPT4 understands the instructions like a human does, and makes the same mistakes.