GPT-3 vs. 3M free-text trivia responses(watercoolertrivia.com)
watercoolertrivia.com
GPT-3 vs. 3M free-text trivia responses
https://www.watercoolertrivia.com/blog/gpt-3-vs-water-cooler-trivia-participants-a-human-vs-robot-showdown
12 comments
All text generated by current language models is defective. For reasons that are not fully understood, text language models emit is less interesting than text emitted by humans. Unlike humans, they can not score well on both coherence, as in grammatically correct sentences, and interest, as in "having something to say". They are however good at "coherent rambling".
Since most language models work by using its outputs as its inputs, this means that the emitted text becomes worse the more words that are emitted. Suppose you want your model to emit a novel. The first 30 words might come out ok. The next 30 are so-so. Eventually, after a few hundred words, the model will begin to ramble and will get stuck in loops: "then she went to visit her mother and she went to visit her mother and she vent to visit her mother and she ..."
All language models suffer from this problem; Markov models, RNNs, Transformers, GPT-2/3... In fact, one could argue that GPT-3 has "only" decreased the deterioration speed. Markov models deteriorate after a dozen words while GPT-3 can keep going for perhaps a hundred words before it begins to ramble. However, the statistical properties of the text GPT-3 generates are different from what a human emits (and what those "statistical properties" are is unknown - it appears that capturing what makes human prose human-like in a mathematical formula is very difficult) from the very first word so you can't use GPT-3-generated text to train GPT-4. The paper "The Curious Case of Neural Text Degeneration" describes the problem in more detail and with math.
A major breakthrough in language modeling may come if someone is able to invent a model that doesn't degenerate. One whose thousandth word is as human-like as its first word.
Since most language models work by using its outputs as its inputs, this means that the emitted text becomes worse the more words that are emitted. Suppose you want your model to emit a novel. The first 30 words might come out ok. The next 30 are so-so. Eventually, after a few hundred words, the model will begin to ramble and will get stuck in loops: "then she went to visit her mother and she went to visit her mother and she vent to visit her mother and she ..."
All language models suffer from this problem; Markov models, RNNs, Transformers, GPT-2/3... In fact, one could argue that GPT-3 has "only" decreased the deterioration speed. Markov models deteriorate after a dozen words while GPT-3 can keep going for perhaps a hundred words before it begins to ramble. However, the statistical properties of the text GPT-3 generates are different from what a human emits (and what those "statistical properties" are is unknown - it appears that capturing what makes human prose human-like in a mathematical formula is very difficult) from the very first word so you can't use GPT-3-generated text to train GPT-4. The paper "The Curious Case of Neural Text Degeneration" describes the problem in more detail and with math.
A major breakthrough in language modeling may come if someone is able to invent a model that doesn't degenerate. One whose thousandth word is as human-like as its first word.
That's a No True Scotsman way of looking at language models. GPT-3 or any Transformer model doesn't need to be innovative to be successful. But that's why it's important to run experiments and build test cases.
In some cases, degeneration can actually be a feature: https://twitter.com/dril_gpt2
In some cases, degeneration can actually be a feature: https://twitter.com/dril_gpt2
Great points. I suspect a non-degenerative model is orders of magnitude harder, in that good long prose requires much deeper complexity.
For example, I was just reading this excellent long piece from Rebecca Traister: https://nymag.com/intelligencer/article/andrew-cuomo-miscond...
Even if a language model had all the raw facts needed to write a long piece, there's a ton of structure here. She develops ideas, builds themes, assembles a thesis. She is not just going somewhere, but carefully taking us along with her. That requires modeling not just textual relationships, but taking conceptual relationships and expressing them in ways that require deep awareness of how the audience processes information.
So I'd be willing to bet a true non-degenerative model requires an increasingly complex and accurate theory of mind.
For example, I was just reading this excellent long piece from Rebecca Traister: https://nymag.com/intelligencer/article/andrew-cuomo-miscond...
Even if a language model had all the raw facts needed to write a long piece, there's a ton of structure here. She develops ideas, builds themes, assembles a thesis. She is not just going somewhere, but carefully taking us along with her. That requires modeling not just textual relationships, but taking conceptual relationships and expressing them in ways that require deep awareness of how the audience processes information.
So I'd be willing to bet a true non-degenerative model requires an increasingly complex and accurate theory of mind.
The solution proposed in that paper, nucleolus sampling, effectively solves this problem though...
It’s an odd hack, though.
Adding a bit of randomness results in a more interesting exploration of Borge’s Library of Babel, but it seems weird to assign meaning to the word choice?
Adding a bit of randomness results in a more interesting exploration of Borge’s Library of Babel, but it seems weird to assign meaning to the word choice?
Nucleus sampling can increase the rambling horizon for some datasets and models by some small percentage. A top-k sampled model might start rambling after 50 words and a nucleus sampled model perhaps can keep going until 60 words. But even with nucleus sampling, the text will eventually degenerate.
> For reasons that are not fully understood, text language models emit is less interesting than text emitted by humans.
What's not understood? As you describe, every standard language model is statistically predicting the next word given the previous words, ad infinitum. GPT3 has more parameters to build a more precise "thought" vector, but the goal is still only to produce the next word.
What's not understood? As you describe, every standard language model is statistically predicting the next word given the previous words, ad infinitum. GPT3 has more parameters to build a more precise "thought" vector, but the goal is still only to produce the next word.
Suppose you had an impossibly perfect conditional probability distribution over "text that a person would write". I realize there may be some issues with that being a well-defined concept. If you don't think it is well defined (and perhaps it isn't), please momentarily pretend that that is something that is/can-be well defined.
In that case, for any property of a text, such as "the text gets ramble-y and basically doesn't have any real point", the probability that a randomly selected text produced by a person has the property, and the probability that a text sampled from that distribution, would be equal.
Could text be sampled from such a distribution on a word by word (or token by token) basis? Yes. Let's see why:
For any event (or property), there are countably many texts which satisfy the property. So, it can be expressed as a countable disjoint union of sets which each have as their only element one of the texts which have the property. So, the probability of a text sampled from the distribution having the property, is equal to the sum of the probabilities of each of the texts which have the property. The probability of each text is equal to the probability of "the next token is [the last word of the text]" given [all the tokens before the last one], multiplied by the probability of the text [all of the tokens before the last one]. And then recurse on that, in order to break the probability of each text down into products of conditional probabilities of (next token | previous tokens) .
Hm, actually, I think there's an issue with what I said. It would work if all texts people wrote had the same length, or if all of them continued indefinitely, but that's not the case. Maybe that can be patched by including a conditional probability for an "end of text" token. Actually, yes, that seems likely. Say that the probability of the text is the probability of (that is all of the text | [the text] is some initial segment of the text), multiplied by the probability of ([the text] is some initial segment of the text). and then the probability of ([an initial segment] is some initial segment of the text) is the probability of ([the last token of the initial segment] is the next token in an initial segment of the text | [the tokens before the last one of the segment] is an initial segment of the text ) times the probability of ([the tokens before the last one of the segment] is an initial segment of the text).
Ok, yeah.
So, in theory, if you allow arbitrary conditional probabilities for the next token (or an end-of-text token) conditioned on all the previous tokens, you can produce any probability distribution over texts.
Therefore, "predicting the next word given the prior words" isn't any limitation on what probability distributions over text can be produced. (though perhaps it could drastically increase the cost of producing certain probability distributions, and perhaps in such a way that makes it practically infeasible to produce in such a way.)
In that case, for any property of a text, such as "the text gets ramble-y and basically doesn't have any real point", the probability that a randomly selected text produced by a person has the property, and the probability that a text sampled from that distribution, would be equal.
Could text be sampled from such a distribution on a word by word (or token by token) basis? Yes. Let's see why:
For any event (or property), there are countably many texts which satisfy the property. So, it can be expressed as a countable disjoint union of sets which each have as their only element one of the texts which have the property. So, the probability of a text sampled from the distribution having the property, is equal to the sum of the probabilities of each of the texts which have the property. The probability of each text is equal to the probability of "the next token is [the last word of the text]" given [all the tokens before the last one], multiplied by the probability of the text [all of the tokens before the last one]. And then recurse on that, in order to break the probability of each text down into products of conditional probabilities of (next token | previous tokens) .
Hm, actually, I think there's an issue with what I said. It would work if all texts people wrote had the same length, or if all of them continued indefinitely, but that's not the case. Maybe that can be patched by including a conditional probability for an "end of text" token. Actually, yes, that seems likely. Say that the probability of the text is the probability of (that is all of the text | [the text] is some initial segment of the text), multiplied by the probability of ([the text] is some initial segment of the text). and then the probability of ([an initial segment] is some initial segment of the text) is the probability of ([the last token of the initial segment] is the next token in an initial segment of the text | [the tokens before the last one of the segment] is an initial segment of the text ) times the probability of ([the tokens before the last one of the segment] is an initial segment of the text).
Ok, yeah.
So, in theory, if you allow arbitrary conditional probabilities for the next token (or an end-of-text token) conditioned on all the previous tokens, you can produce any probability distribution over texts.
Therefore, "predicting the next word given the prior words" isn't any limitation on what probability distributions over text can be produced. (though perhaps it could drastically increase the cost of producing certain probability distributions, and perhaps in such a way that makes it practically infeasible to produce in such a way.)
That paper includes this observation:
Why is human-written text not the most probable text? We conjecture that this is an intrinsic property of human language... Grice’s Maxims of Communication (Grice,1975) show that people optimize against stating the obvious.
I don't think that should be at all surprising, if one purpose of speech is to convey information: predictability is the converse of being informational.
Furthermore, to be informative, it is not sufficient for a passage to avoid predictability; it also needs to be pertinent.
Why is human-written text not the most probable text? We conjecture that this is an intrinsic property of human language... Grice’s Maxims of Communication (Grice,1975) show that people optimize against stating the obvious.
I don't think that should be at all surprising, if one purpose of speech is to convey information: predictability is the converse of being informational.
Furthermore, to be informative, it is not sufficient for a passage to avoid predictability; it also needs to be pertinent.
Your ascribing way too much intelligence to a very simplistic text prediction algorithm. Neither GPT-3 nor GPT-4 (assuming it will just be a bigger GPT-3) have any concept of identity that could influence their predictions in the way you seem to assume.
If GPT-4 would be trained on text produced by GPT-3 and on text that analyzes that, what GPT-4 would learn is how to sound like GPT-3 and like detractors or admirers of GPT-3.
While impressive achievements, GPT-2/3/4/X are just models that predict what the most likely next word is based on a huge corpus of text. There is no model of the world in there, no objects, no identities. Just probabilities of glyphs in certain orders, without any meaning attached.
If GPT-4 would be trained on text produced by GPT-3 and on text that analyzes that, what GPT-4 would learn is how to sound like GPT-3 and like detractors or admirers of GPT-3.
While impressive achievements, GPT-2/3/4/X are just models that predict what the most likely next word is based on a huge corpus of text. There is no model of the world in there, no objects, no identities. Just probabilities of glyphs in certain orders, without any meaning attached.
I mostly agree with you, but I wonder what's the distance between what you described and our actual intelligence.
I don't think there is any kind of similarity between a text prediction model and human (or animal) intelligence. At most we maybe have a subsystem that could work somewhat similarly.
It's very clear though that human intelligence has a model of the world being made up of agents which interact with objects. Words and sentences refer to entities in this model in our head, and our use of language essentially consists of describing instances of this model, with various purposes.
Text prediction works exclusively on text, no model of the world, and it has a single purpose: to produce the text that would be most likely to follow the prompt based on its training.
This isn't just a theoretical difference. GPT-3 couldn't be used to, for example, be presented with a situation and asked to respond to it in such a way that it achieves some goal.
It's very clear though that human intelligence has a model of the world being made up of agents which interact with objects. Words and sentences refer to entities in this model in our head, and our use of language essentially consists of describing instances of this model, with various purposes.
Text prediction works exclusively on text, no model of the world, and it has a single purpose: to produce the text that would be most likely to follow the prompt based on its training.
This isn't just a theoretical difference. GPT-3 couldn't be used to, for example, be presented with a situation and asked to respond to it in such a way that it achieves some goal.
The first part of your statement feels like it's anthropomorphising GPT-4. GPT-4s response to learning that GPT-3 is bad at word play will to store the concept of GPT-3 being bad at word play to recall if prompted about GPT-3 in future, not to swot up on word play. If you want GPT-4 to be better at word play you'll need a corpus of puns, rhymes and higher level concept association and a lot of training on replicating them and when to recognise a prompt might require them.
The whole point of a model is to take advantage of data with more information. If GPT-3 output sucks for predicting human output then it will be ignored by the model during descent. Otherwise the learning algorithm is defective.
Somewhere in it’s billions of parameters it will have a sense of self. A model of itself. That’s pretty mind boggling.
One intuition about the large GPT-models is that they are learning "facts" like they are learning grammar rules. It's just that their capacity is so immensely large as to learn facts as part of the language model.
> The more important takeaway: dozens of technologists at IBM spent more than three years and untold millions of dollars building the program specifically trained for Jeopardy! prowess. Less than 10 years later, a general-purpose open-sourced technology without the massive mainframe or cooling fans can compete on the same level.
This statement from the article confused me. GPT-3 is general-purpose, but not open source, nor does it run without massive hardware, nor was it developed without spending millions of dollars.
This statement from the article confused me. GPT-3 is general-purpose, but not open source, nor does it run without massive hardware, nor was it developed without spending millions of dollars.
Yep, it's not only open sourced, it's not even open access. (I get the reason, but somehow, maybe due to the fact that it's from an organization called OpenAI, a lot of people seem to think that everyone could use it if they wanted to.)
You could compare it to T5 instead. T5 does awesome at trivia, both the code and trained models are available, and it'll run on ordinary hardware.
(Not sure if one could argue 'the millions of dollars' part, but that GB R&D wasn't spent for the purpose of trivia playing like the Watson Jeopardy system was aimed at doing, so seems reasonably fair to me - the trivia playing is merely the byproduct of the target capabilities of translation, knowledge graph construction etc).
(Not sure if one could argue 'the millions of dollars' part, but that GB R&D wasn't spent for the purpose of trivia playing like the Watson Jeopardy system was aimed at doing, so seems reasonably fair to me - the trivia playing is merely the byproduct of the target capabilities of translation, knowledge graph construction etc).
How is GPT3 considered publicly available? I applied months ago and have received no response.
"The robot was best at Fine Arts and Current Events, worst at Word Play and Social Studies. ...This one’s not so surprising. We have a type of question called a “Two’fer Goofer” which asks for a pair of rhyming words that satisfy a given clue. It’s similar to the Rhyme Time category in Jeopardy or the old newspaper puzzle Wordy Gurdy. We had three of these questions in the showdown and GPT-3 missed all three of them. For Word Play questions that were more like vocabulary quizzes, GPT-3 performed admirably:"
More evidence, if it was needed, that the use of BPEs sabotages GPT-3 in a lot of subtle ways (https://www.gwern.net/GPT-3#bpes). GPT-3 can understand things like vocab definitions which do not depend on the internal spelling of a word or phonetics (which are erased by the BPE encoding of the data it was trained on), but as soon as you have to do things like puns... The BPEs are deadly. Ah well. Eventually OA or someone will train a proper character-level model, and then I'll be able to generate rhyming poetry without hacks like rhyming dictionaries.
More evidence, if it was needed, that the use of BPEs sabotages GPT-3 in a lot of subtle ways (https://www.gwern.net/GPT-3#bpes). GPT-3 can understand things like vocab definitions which do not depend on the internal spelling of a word or phonetics (which are erased by the BPE encoding of the data it was trained on), but as soon as you have to do things like puns... The BPEs are deadly. Ah well. Eventually OA or someone will train a proper character-level model, and then I'll be able to generate rhyming poetry without hacks like rhyming dictionaries.
> Less than 10 years later, a general-purpose open-sourced technology without the massive mainframe or cooling fans can compete on the same level.
First, are GPT-3 models generally available?
Secondly, GPT-3 is many orders of magnitude larger than any model existing 10 years ago. There are certainly cooling fans involved.
First, are GPT-3 models generally available?
Secondly, GPT-3 is many orders of magnitude larger than any model existing 10 years ago. There are certainly cooling fans involved.
> 2. Clues confuse GPT-3.
They should probably have been removed. This gives me the overall impression that the testers treat GPT-3 a bit too much as something like an artificial human, and not enough like an algorithm (which will work better with sanitized input). This is not a major criticism, the experiment is still interesting.
Could it be that the marketing from OpenAI it to blame? From the OpenAI front page:
> Discovering and enacting the path to safe artificial general intelligence.
> Our first-of-its-kind API can be applied to any language task, and currently serves millions of production requests each day.
Does that seem misleading?
They should probably have been removed. This gives me the overall impression that the testers treat GPT-3 a bit too much as something like an artificial human, and not enough like an algorithm (which will work better with sanitized input). This is not a major criticism, the experiment is still interesting.
Could it be that the marketing from OpenAI it to blame? From the OpenAI front page:
> Discovering and enacting the path to safe artificial general intelligence.
> Our first-of-its-kind API can be applied to any language task, and currently serves millions of production requests each day.
Does that seem misleading?
OpenAI's marketing has been fair, and it's not misleading (GPT-3 can be applied to any language tasks but your mileage will vary as this submission demonstrates).
However, OpenAI's endorsement of the demos amplifying and anthropomorphizing GPT-3 as a sentient mind don't help, and it's been disappointing that OpenAI doesn't really push back on that. (the hype is what prompted my rebuttal on GPT-3 expectations, which still holds up: https://news.ycombinator.com/item?id=23891226)
I do believe the future of AI text generation is more bespoke, algorithmic-input-friendly models which is what I've been working on as my side project.
However, OpenAI's endorsement of the demos amplifying and anthropomorphizing GPT-3 as a sentient mind don't help, and it's been disappointing that OpenAI doesn't really push back on that. (the hype is what prompted my rebuttal on GPT-3 expectations, which still holds up: https://news.ycombinator.com/item?id=23891226)
I do believe the future of AI text generation is more bespoke, algorithmic-input-friendly models which is what I've been working on as my side project.
Very good write-up. It seems too soon to think of genuine commercialization at this time.
I thought about removing those two-word clues but ultimately, they exist for users so I wanted to be closer to apples-to-apples!
Could GPT-3 lower barriers of entry for search engines? Assuming, it could update its "index" (weights) at regular intervals, it's not too far-fetched to see it competing with Google with high-quality relevant answers to queries.
I've tried asking GPT-3 factual questions and sometimes it does great. Other times, well... let's just say it doesn't quite understand. Bad answers are in italics.
Q: How tall is the Eiffel Tower? A: About twenty feet.
Q: What color are the clouds? A: They're a light blue.
Q: Which way does the sun rise in Paris? A: You'd have to be standing on top of a building to see it because it doesn't rise that high.
Q: Who invented the telephone? A: Alexander Graham Bell.
Q: Who invented Google? A: Larry Page and Sergey Brin.
Q: What's the capital of California? A: It's not located in California. It's Sacramento.
Q: How tall is the Eiffel Tower? A: About twenty feet.
Q: What color are the clouds? A: They're a light blue.
Q: Which way does the sun rise in Paris? A: You'd have to be standing on top of a building to see it because it doesn't rise that high.
Q: Who invented the telephone? A: Alexander Graham Bell.
Q: Who invented Google? A: Larry Page and Sergey Brin.
Q: What's the capital of California? A: It's not located in California. It's Sacramento.
There are some works resonate with your idea, here's the first one I can recall.
https://arxiv.org/abs/2012.14983
https://arxiv.org/abs/2012.14983
There was recently a thread[1] about an open-source alternative to GPT-3 here, called "GPT-Neo".
I hope it takes off. But the language on their website undermines some confidence: "GPT-Neo is the code name for a series of transformer-based language models [...] we plan to train and open source."
PS: The original URL that was submitted to HN is now 404; it moved to here[2].
[1] https://news.ycombinator.com/item?id=25819803
[2] https://www.eleuther.ai/projects/gpt-neo/
I hope it takes off. But the language on their website undermines some confidence: "GPT-Neo is the code name for a series of transformer-based language models [...] we plan to train and open source."
PS: The original URL that was submitted to HN is now 404; it moved to here[2].
[1] https://news.ycombinator.com/item?id=25819803
[2] https://www.eleuther.ai/projects/gpt-neo/
It would be helpful to see the prompt structure used for gpt-3: state of the art prompting methods do significantly better than naive ones, and randomness and other settings can make a big difference in quality as well.
There are also cyclical prompt methods which can help derandomize gpt3, for example, you could generate three to five answers per question then feed the set back to gpt-3 and ask it which is the most correct; the first round would use high randomness settings the last very low.
My own experience is that you sort of have to work with gpt3 to get on the same page sometimes, and when you do the results can be remarkable.
Anyway, fun idea!
There are also cyclical prompt methods which can help derandomize gpt3, for example, you could generate three to five answers per question then feed the set back to gpt-3 and ask it which is the most correct; the first round would use high randomness settings the last very low.
My own experience is that you sort of have to work with gpt3 to get on the same page sometimes, and when you do the results can be remarkable.
Anyway, fun idea!
I think than human communication is better than any ai model because human goes from general to concrete, adding more information to convert general context into insightful ideas. We use language to communicate what is important for us. Ai is about navigating in an ocean trying not to be captured by concrete things, because it knows nothing about being real.
As a mathematical methafor, each word is a window in the tangent space of an n-dimensional sphere and the normal points to the center of the sphere, as time goes on communication is about shrinking the radious of the sphere so that the center is like the kernel of what we wish to communicate. The sphere in an ai system has a very big radius and is not able to shrink because it lacks a kernel.
You could simulate a human conversation with an ai, for example to sell something following steps like description of a product, price advantages and so on, but in a real context those are ad hoc methods and there is no general rule to shrink that is to penetrate deep into the meaning of what you want to communicate.
As a mathematical methafor, each word is a window in the tangent space of an n-dimensional sphere and the normal points to the center of the sphere, as time goes on communication is about shrinking the radious of the sphere so that the center is like the kernel of what we wish to communicate. The sphere in an ai system has a very big radius and is not able to shrink because it lacks a kernel.
You could simulate a human conversation with an ai, for example to sell something following steps like description of a product, price advantages and so on, but in a real context those are ad hoc methods and there is no general rule to shrink that is to penetrate deep into the meaning of what you want to communicate.
"Two'fer Goofer", "Tough Training" - why would GPT-3 give the question back as its response in these cases?
GPT-3 has a habit of repeating its input. GPT-2 had the same problem, as did all language models before. LSTMs were really bad in this respect and would often get stuck in single word loops. GPT-3 just does it less often, and tends to repeat sentences or ideas, rather than single words.
A larger corpus and more training seemed to have helped reduce the issue, and I think they tweaked the temperature algorithms as well (I think top-p is an option now, whereas most language models used top-k historically).
In my experience GPT repeats itself more often when it doesn't understand the prompt. I mean, if you don't understand _anything_ about an input sequence, what is the most likely next word? One of the previously used words.
It's likely that the following occurs: Given the phrase "1+1=? ", the first layer assigns a baseline probability to "1", "+", "=", and "?", because those occur in the input and its a good fallback. But then the layers after recognize the input as a math equation and quickly assign a high probability to "2". Since the probability on "2" is so high the temperature algorithm always picks "2" as the output token.
But if we imagine GPT doesn't understand math, then all the layers after the first have no clue what the input is. They can't assign higher probability to anything. They might make a few weak guesses, say "duck". But it isn't confident in them, so really "duck" is as likely as "1". The temperature algorithm then has a fairly decent chance of selecting "1" as the output token. Now GPT is given "1+1=? 1" and it seems like "+" is the next logical token. "1+1=? 1+". And so forth until it GPT is stuck in self reinforcing a loop saying "1+1=? 1+1=? 1+1=? 1+1=?" etc.
Funny thing is, it's not like humans don't do this. When you get asked a hard question that you don't have an answer for, don't you find yourself repeating the question to yourself over and over again, hoping you'll better understand it? I'm not necessarily creating a parallel between that behavior and GPT's behavior. Just food for thought, I suppose.
A larger corpus and more training seemed to have helped reduce the issue, and I think they tweaked the temperature algorithms as well (I think top-p is an option now, whereas most language models used top-k historically).
In my experience GPT repeats itself more often when it doesn't understand the prompt. I mean, if you don't understand _anything_ about an input sequence, what is the most likely next word? One of the previously used words.
It's likely that the following occurs: Given the phrase "1+1=? ", the first layer assigns a baseline probability to "1", "+", "=", and "?", because those occur in the input and its a good fallback. But then the layers after recognize the input as a math equation and quickly assign a high probability to "2". Since the probability on "2" is so high the temperature algorithm always picks "2" as the output token.
But if we imagine GPT doesn't understand math, then all the layers after the first have no clue what the input is. They can't assign higher probability to anything. They might make a few weak guesses, say "duck". But it isn't confident in them, so really "duck" is as likely as "1". The temperature algorithm then has a fairly decent chance of selecting "1" as the output token. Now GPT is given "1+1=? 1" and it seems like "+" is the next logical token. "1+1=? 1+". And so forth until it GPT is stuck in self reinforcing a loop saying "1+1=? 1+1=? 1+1=? 1+1=?" etc.
Funny thing is, it's not like humans don't do this. When you get asked a hard question that you don't have an answer for, don't you find yourself repeating the question to yourself over and over again, hoping you'll better understand it? I'm not necessarily creating a parallel between that behavior and GPT's behavior. Just food for thought, I suppose.
We don’t have to imagine GPT3 doesn’t understand maths. It can reliably answer 1+1= because there are so many instances of it in the corpus, but if you give anything more complicated it will just throw out a random, incorrect, answer in most cases.
Given the parallel you draw at the end, perhaps future models will adopt some kind of common phrase to denote thinking, such as ‘Umm’, ‘er’, etc. Maybe there’s even a neurologically concise explanation for why humans do that!
Isn't it also that in human writing (and training) the most common or correct response/followup to a prompt structured as a question often is a complete "x is y" sentence? GPT-3 appears to consistently parse these question prompts as requiring it to include certain strongly associated words in its output, but not consistently as trivia questions implying terse, associated-word-only answers
It's the opposite of the 'know all' the blog describes, it's GPT-3 knowing where to find the answers without even understanding it's a quiz!
It's the opposite of the 'know all' the blog describes, it's GPT-3 knowing where to find the answers without even understanding it's a quiz!
> Credit where due: friend of WCT Dennis wrote the script to feed Water Cooler Trivia questions to GPT-3 and access the OpenAI API.
It should be noted it's against OpenAI's rules to share access to GPT-3, although they've been inconsistent about enforcing it.
It should be noted it's against OpenAI's rules to share access to GPT-3, although they've been inconsistent about enforcing it.
Articles like these that discuss GPT-3's output, or comments and articles directly generated by GPT-3, are going to be fed into GPT-4 as part of a theoretical WebText3 dataset. Will that help or hurt GPT-4's learning? What effect will it have?
Theoretically you can feed GPT-3's responses back into itself and ask it to introspect about them; whether it was right or wrong and to comment on why it gave the response it did. But I doubt GPT-3 is particularly good at self introspection. GPT-3 was trained before GPT-3 existed, obviously. So it was never trained on articles analyzing the output of an AI of its caliber.
But GPT-4 is going to be trained on a corpus filled with people analyzing GPT-3's outputs, like this article. We would expect GPT-4 to be able to write an article like this. So it should be theoretically possible to give GPT-4 its own output, and then ask it to provide introspection, and for that introspection to be insightful.
EDIT: Follow up thought. It's almost as if the internet is being filled with a training corpus on GPT-3's failings. Every fact that GPT-3 failed to learn from WebText2 is now going to be repeated, alongside the correct answer, in WebText3. Humans are globally working, unknowingly, to build a curated dataset by which GPT-4 can learn from GPT-3's mistakes.