On ChatGPT(acoup.blog)
acoup.blog
On ChatGPT
https://acoup.blog/2023/02/17/collections-on-chatgpt/
18 comments
> And surely they have no mind like us.
I agree with this, it's the old "are submarines swimming" kind of question, semantics and not substance, i.e. irrelevant.
However this is an article written by someone non-technical for other non-technical people, and this (emphasis is the author's) is perfectly on-point:
> The tricky part is that ChatGPT and chatbots like it are designed to make use of a very influential human cognitive bias that we all have: the tendency to view things which are not people as people or at least as being like people.
Which is absolutely a problem. Whether or not you ascribe a "mind" to it, laypeople absolutely need to understand that it's not a human they're talking to.
> Is it useful?
A better question is useful to whom and useful how. As a search engine it's pretty miserable for instance, as we found out. I'd like to leave myself some wiggle room because it's way too early to tell, but my hunch is that operating on natural language is a profound limitation that will make it far less useful than it is generally believed.
I agree with this, it's the old "are submarines swimming" kind of question, semantics and not substance, i.e. irrelevant.
However this is an article written by someone non-technical for other non-technical people, and this (emphasis is the author's) is perfectly on-point:
> The tricky part is that ChatGPT and chatbots like it are designed to make use of a very influential human cognitive bias that we all have: the tendency to view things which are not people as people or at least as being like people.
Which is absolutely a problem. Whether or not you ascribe a "mind" to it, laypeople absolutely need to understand that it's not a human they're talking to.
> Is it useful?
A better question is useful to whom and useful how. As a search engine it's pretty miserable for instance, as we found out. I'd like to leave myself some wiggle room because it's way too early to tell, but my hunch is that operating on natural language is a profound limitation that will make it far less useful than it is generally believed.
> laypeople absolutely need to understand that it's not a human they're talking to
That's hard when the bot either gets angry at you or declares its love if you spend an hour with it. But is anthropomorphising AIs useful? that's the important question. I think it is useful. If you tell GPT3 it is a respected scientist, it will have higher accuracy solving tasks.
That's hard when the bot either gets angry at you or declares its love if you spend an hour with it. But is anthropomorphising AIs useful? that's the important question. I think it is useful. If you tell GPT3 it is a respected scientist, it will have higher accuracy solving tasks.
> But is anthropomorphising AIs useful?
No it isn't. It's disastrous. The bot declares your love for you if you spend an hour with it because it's trained on incel shit from reddit, thinking anything else is 100% a user problem.
> If you tell GPT3 it is a respected scientist, it will have higher accuracy solving tasks.
Even taking that statement at face value, what I think about AI has no bearing on what it does. You are giving input tokens to a machine and you are fooling yourself if you think of it as anything but.
This is exactly the problem about natural language I'm pointing out. I don't want a friend, a coworker or an assistant, I want a mechanical slave to instruct in very specific ways, and natural language is a pretty lousy way to give orders.
No it isn't. It's disastrous. The bot declares your love for you if you spend an hour with it because it's trained on incel shit from reddit, thinking anything else is 100% a user problem.
> If you tell GPT3 it is a respected scientist, it will have higher accuracy solving tasks.
Even taking that statement at face value, what I think about AI has no bearing on what it does. You are giving input tokens to a machine and you are fooling yourself if you think of it as anything but.
This is exactly the problem about natural language I'm pointing out. I don't want a friend, a coworker or an assistant, I want a mechanical slave to instruct in very specific ways, and natural language is a pretty lousy way to give orders.
I have consistently been helped by ChatGPT every time Google has failed me. As a search engine it's really good. Only failing is re new data
Buddhist and zen folks would tell you we have no mind ourselves. I'm seeing a convergence.
Thomas Metzinger would agree with the assessment that we are a hallucination of a self and that we are Being No One.
I'm not sure hallucination is the right word, if we're operating in the theater of large language models. The correct way to describe human consciousness in those terms would be a descriptive model that outputs a lossy reconstructed linear viewpoint, combined with a prompt instructing it to believe the delusion that it's definitely for real in control and making choices instead of telling stories.
The article is quite specific about ChatGPT not being useful for writing essays. It also carefully speculates about some other uses. It even insists on it being a very fancy form of autocompletion, so it shouldn't be surprising that it was useful in your example. And it also points out that it's bad teaching that should be concerned about it, which is also seems the case for your daughter's teacher (for perhaps a very mild example of bad teaching).
When you drill down these discussion points, it all comes down to the fact we don’t exactly know how the human mind works so we can’t properly answer what is the difference between pretending to understand, and understanding.
We know that some humans can use information to solve harder problems, while others can not do it. We say the first kind built understanding about the information.
You can easily test for this by just throwing hard problems at them that they haven't seen before, Google does that in their interviews for example (but many who do similar just takes problems people definitely have seen).
So doing similar kinds of testing on ChatGPT we see that it doesn't understand the information. It can repeat a lot of stuff more or less verbatim, but it cannot apply most of the knowledge it can repeat, so it doesn't understand any of it. The only thing ChatGPT really understands are relationships between words that has been repeated in texts it has seen, that is some level of understanding but that has no relationship to what ChatGPT says about the words.
When you ask ChatGPT to describe something it repeats a description it has seen of the word, when you ask it to do something with a word then it fully ignores that description and instead look at the relationships it has seen the word have to other words. So its understanding and its knowledge are completely separate things, so you can say that it doesn't understand anything it knows.
You can easily test for this by just throwing hard problems at them that they haven't seen before, Google does that in their interviews for example (but many who do similar just takes problems people definitely have seen).
So doing similar kinds of testing on ChatGPT we see that it doesn't understand the information. It can repeat a lot of stuff more or less verbatim, but it cannot apply most of the knowledge it can repeat, so it doesn't understand any of it. The only thing ChatGPT really understands are relationships between words that has been repeated in texts it has seen, that is some level of understanding but that has no relationship to what ChatGPT says about the words.
When you ask ChatGPT to describe something it repeats a description it has seen of the word, when you ask it to do something with a word then it fully ignores that description and instead look at the relationships it has seen the word have to other words. So its understanding and its knowledge are completely separate things, so you can say that it doesn't understand anything it knows.
This is a great idea, when I ask for help with something, so often the first thing is, code style, or variable names.
This is the first hell yea use case I have seen for chat gtp! Thank you and your daughter!
This is the first hell yea use case I have seen for chat gtp! Thank you and your daughter!
> And surely they have no mind like us.
I've often wondered about this, because this answers the question "are we alone on this planet?" with a clear, resounding "no".
For example, every species has a mind in it's DNA. It communicates very slowly and it thinks and communicates by killing large amounts of life. Or perhaps it can be better put as it communicates by enhancing or diminishing specific forms of life, not always outright killing them. You can talk to this, like we've done accidentally with vaccines. DNA's answer would then be antibiotic resistance in dangerous diseases. It's a very different "mind".
I've often wondered about this, because this answers the question "are we alone on this planet?" with a clear, resounding "no".
For example, every species has a mind in it's DNA. It communicates very slowly and it thinks and communicates by killing large amounts of life. Or perhaps it can be better put as it communicates by enhancing or diminishing specific forms of life, not always outright killing them. You can talk to this, like we've done accidentally with vaccines. DNA's answer would then be antibiotic resistance in dangerous diseases. It's a very different "mind".
The analogy with alchemy was very on point; this stuff is an Eliza de nos jours, a clever parlour trick and for sure not without its uses, but it is not intelligence in any meaningful definition of that term.
A lot of folks are dismissive of these criticisms by claiming that we have no evidence that human cognition is fundamentally different, but it must be - our referents are based in the real world, whereas the LLMs have no such referents, and it is this tie-back to reality that underpins our ability to act intelligently. And the argument that an equivalent understanding can somehow be an emergent property is undermined by the observation that the LLMs have no concept of truth, and no ability to tell whether their output is pure BS.
A lot of folks are dismissive of these criticisms by claiming that we have no evidence that human cognition is fundamentally different, but it must be - our referents are based in the real world, whereas the LLMs have no such referents, and it is this tie-back to reality that underpins our ability to act intelligently. And the argument that an equivalent understanding can somehow be an emergent property is undermined by the observation that the LLMs have no concept of truth, and no ability to tell whether their output is pure BS.
I like the essay a lot and I find it intellectually amusing that many negative comments here manifest exactly the flaws that could be prevented by learning the essay writing as described in the blog. The article clearly states assumptions, definitions, context and limitations of it's arguments and in that specific area it's very hard to refute them because they are sound. On the other side the gist of many comments here is that article is negative towards GPT and should be positive, i.e. the same generalist aproach that GPT generated text would take and which is hard to argue about.
I'm not going to analyse particular flaws of separate comments because the article itself is really comprehensive and answers for many objections are better in original form there than any reproduction I can make here. But the high number of rebuttals that obviously have problems either understanding the essay or formulating a solid counterargument is curious. An obvious explanation is that some people couldn't resist the pleasure of letting AI dismiss the statement about insufficient AI capabilities, but that's a lazy one.
Comments saying current AI is comparable to actual human inteligence may be right even though I can clearly see that AI is not performing actions I consider necessary for thinking. It's because I consider my own mind as a model of human inteligence, but as I learned many times before, the thinking process may be hugely different for different people. I have no idea if other people also do need to create an explicit model of something in their head to be able to "think" about it.
It would explain A LOT for me if thinking of some other people really works in the same way as in GPT. That situation should however still be viewed not as a technology enhancement so big it reaches human level but reconsideration of what we consider a human level.
This happens to me a lot. I want to make a joke, but it gets so long an complex that it becomes an analysis.
I'm not going to analyse particular flaws of separate comments because the article itself is really comprehensive and answers for many objections are better in original form there than any reproduction I can make here. But the high number of rebuttals that obviously have problems either understanding the essay or formulating a solid counterargument is curious. An obvious explanation is that some people couldn't resist the pleasure of letting AI dismiss the statement about insufficient AI capabilities, but that's a lazy one.
Comments saying current AI is comparable to actual human inteligence may be right even though I can clearly see that AI is not performing actions I consider necessary for thinking. It's because I consider my own mind as a model of human inteligence, but as I learned many times before, the thinking process may be hugely different for different people. I have no idea if other people also do need to create an explicit model of something in their head to be able to "think" about it.
It would explain A LOT for me if thinking of some other people really works in the same way as in GPT. That situation should however still be viewed not as a technology enhancement so big it reaches human level but reconsideration of what we consider a human level.
This happens to me a lot. I want to make a joke, but it gets so long an complex that it becomes an analysis.
Education has always done a terrible job of keeping up with technology. Kids in school now will graduate into a world in which all information will be instantly available and will be presented in whatever format is most suitable. They'll have computers correcting their grammar, improving their ideas, completing their sentences and paragraphs. Instead of learning those technologies themselves and instructing students how to best utilize them, teachers tell kids, "that's cheating". When you graduate and get your first job, your boss isn't going to take away your books, phone, and laptop, and ask you to write a report about a well understood subject. I understand why that's a common practice in education. Maybe it's time to change.
Being able to give a well-reasoned opinion about a matter in your particular area of expertise to a manager who isn't deep into it during a face-to-face meeting without pulling out your phone or laptop is essential to career progress. Essays are a good way to develop this, especially the kind you have to be able to write in-class. Can you absorb enough information and context about a topic to make a decent argument on demand?
Being able to distinguish between useful, valid information and whatever YouTube video the search engine happened to turned up is going to be an even more important skill for those kids in school now than it was for those of us who were in college when Google was a hot new startup.
For those of you who didn't read the whole post (and it was long, so I kind of understand), Mr. Devereaux made an aside about his belief in the continuing value of initially learning how to do arithmetic without a calculator despite their easy availability over the past few decades, an opinion I've always shared.
Before reading this post, I still believed the same about the value of learning how to write essays ("delivery boxes for thoughts" was his expression, I think) and will make sure that my kid can write one with just a pencil and paper, even though he'll also be able to use whatever technical assistance is available in 10-15 years. He's learning to draw and make letters with crayons and pens before I'll let him spend a lot of time with my iPad; he's sussed out how that worked just by watching me, so I'm not concerned about a technology gap with his future classmates.
This post gives me something to forward to my non-technical but curious friends when they ask about ChatGPT and similar.
Being able to distinguish between useful, valid information and whatever YouTube video the search engine happened to turned up is going to be an even more important skill for those kids in school now than it was for those of us who were in college when Google was a hot new startup.
For those of you who didn't read the whole post (and it was long, so I kind of understand), Mr. Devereaux made an aside about his belief in the continuing value of initially learning how to do arithmetic without a calculator despite their easy availability over the past few decades, an opinion I've always shared.
Before reading this post, I still believed the same about the value of learning how to write essays ("delivery boxes for thoughts" was his expression, I think) and will make sure that my kid can write one with just a pencil and paper, even though he'll also be able to use whatever technical assistance is available in 10-15 years. He's learning to draw and make letters with crayons and pens before I'll let him spend a lot of time with my iPad; he's sussed out how that worked just by watching me, so I'm not concerned about a technology gap with his future classmates.
This post gives me something to forward to my non-technical but curious friends when they ask about ChatGPT and similar.
> I understand why that's a common practice in education. Maybe it's time to change.
not that i disagree, but what would you have it change to?
The fact that there's no need to memorize anything any more, means that any tests which allows the use of tech (like the internet, or chatGPT) to retrieve data, means that students no longer need to commit things to long term memory.
However, having the capability, and be able to recall facts when needed, and at high speed, is something that is foundational to higher creative thoughts.
This higher, creative thought, is not really easy to test, so the memorization is the proxy.
I do not know what form education (and the testing of it) would take on, if tools like chatGPT is allowed to be used.
not that i disagree, but what would you have it change to?
The fact that there's no need to memorize anything any more, means that any tests which allows the use of tech (like the internet, or chatGPT) to retrieve data, means that students no longer need to commit things to long term memory.
However, having the capability, and be able to recall facts when needed, and at high speed, is something that is foundational to higher creative thoughts.
This higher, creative thought, is not really easy to test, so the memorization is the proxy.
I do not know what form education (and the testing of it) would take on, if tools like chatGPT is allowed to be used.
In my experience (from exams allowing full access to computers and the Internet (for information retrieval, not communication with other people)) you need to memorise the most important concepts and patterns anyway, or you will be too slow during exams. YMMV.
Memorizing things is not useless. Learning concepts and principles is more important, but you also need some amount of facts memorized to make use of them.
For example, I had to memorize the structural formulas for all amino acids in university. This does seem a bit useless at first, but is actually very important the moment you work with protein sequences or structures. You might not need the exact structure, but if you read about a specific important residue in a protein or a mutation in one you need to understand the properties of the involved amino acids to make sense to this. And if you had to look that up every time you'd never get through a paper.
For example, I had to memorize the structural formulas for all amino acids in university. This does seem a bit useless at first, but is actually very important the moment you work with protein sequences or structures. You might not need the exact structure, but if you read about a specific important residue in a protein or a mutation in one you need to understand the properties of the involved amino acids to make sense to this. And if you had to look that up every time you'd never get through a paper.
It’s about the method.
Teachers aren’t interested in a recount of the battle of so-and-so but in training you to gather knowledge, structure your thoughts and express them clearly.
You can only learn that by doing. A chatbot bypasses the learning process, so you will have neither gained subject knowledge nor methodical one.
Teachers aren’t interested in a recount of the battle of so-and-so but in training you to gather knowledge, structure your thoughts and express them clearly.
You can only learn that by doing. A chatbot bypasses the learning process, so you will have neither gained subject knowledge nor methodical one.
> The calculator was meant to make computation more convenient for people who already knew about numbers. Now, it threatens to crash the intellectual order, assuming the role of an end, when it is only a means.
https://www.nytimes.com/1991/09/29/opinion/the-calculator-cr...
I'm pretty sure that a "sufficiently smart" chatbot (or maybe even an extra dumb one) is a useful tool "in training you to gather knowledge, structure your thoughts, and express them clearly". I've found it remarkably useful for clarifying my thoughts, considering alternative arguments, and general tomfoolery that can spark creativity.
https://www.nytimes.com/1991/09/29/opinion/the-calculator-cr...
I'm pretty sure that a "sufficiently smart" chatbot (or maybe even an extra dumb one) is a useful tool "in training you to gather knowledge, structure your thoughts, and express them clearly". I've found it remarkably useful for clarifying my thoughts, considering alternative arguments, and general tomfoolery that can spark creativity.
The problem is that computing things is something most people don't do that frequently, while "structuring your thoughs and expressing them clearly" is a prerequisite to have any sort of meaningful conversation or even opinion.
I am mostly worried about young people who will grow up relying too much on ChatGPT, what will they do when they do not have a bot hand-holding them through some complicated idea? And if this kind of bots become so ubiquitous, what is the place for humans?
When calculators became wide-spread, we calculated a lot more. When LLM become wide-spread, we will.. Think more? I seriously don't know.
I am mostly worried about young people who will grow up relying too much on ChatGPT, what will they do when they do not have a bot hand-holding them through some complicated idea? And if this kind of bots become so ubiquitous, what is the place for humans?
When calculators became wide-spread, we calculated a lot more. When LLM become wide-spread, we will.. Think more? I seriously don't know.
I'm very, very sure that a machine which requires you to structure your thoughts and express them clearly will not lower the capacity for that in the general public. LLMs are extremely prone to garbage in, garbage out - if you can't be precise in structuring and expressing your thoughts, your results will be likewise questionable.
I certainly benefit greatly already from using LLMs to accomplish a number of tasks. I think the answer on where the responsibility lies depends greatly on your view of the same sorts of questions around auteur theory - is the director responsible for the quality of the film? Or is it the writer of the screenplay? What about the cast, or the producers? Is Microsoft the author if you write a novel in Word, without scribing the lines onto the page yourself? I think it's going to be very interesting to see how all of this plays out, and where the lines are drawn. I suspect that what is causing concern now will, in ten years perhaps, be normal, obvious and not even discussed.
I certainly benefit greatly already from using LLMs to accomplish a number of tasks. I think the answer on where the responsibility lies depends greatly on your view of the same sorts of questions around auteur theory - is the director responsible for the quality of the film? Or is it the writer of the screenplay? What about the cast, or the producers? Is Microsoft the author if you write a novel in Word, without scribing the lines onto the page yourself? I think it's going to be very interesting to see how all of this plays out, and where the lines are drawn. I suspect that what is causing concern now will, in ten years perhaps, be normal, obvious and not even discussed.
> LLMs are extremely prone to garbage in, garbage out - if you can't be precise in structuring and expressing your thoughts, your results will be likewise questionable.
I agree, and that is the issue. People like us can use LLMs effectively because we are already capable of expressing our thoughts in a decent manner and we can recognize when the output does not make sense. But to know whether the results can be trusted or not, you already need to be one level above that. If one is not capable of producing a coherent argument on their own, how can they evaluate whether an argument they hear is itself coherent? And if one, say because of lazyness, relies on LLMs from their childhood to fill in all the difficult steps, how will they learn how to do it on their own? Practising has always been the best way to learn things.
> I think it's going to be very interesting to see how all of this plays out, and where the lines are drawn. I suspect that what is causing concern now will, in ten years perhaps, be normal, obvious and not even discussed.
Well said.
I agree, and that is the issue. People like us can use LLMs effectively because we are already capable of expressing our thoughts in a decent manner and we can recognize when the output does not make sense. But to know whether the results can be trusted or not, you already need to be one level above that. If one is not capable of producing a coherent argument on their own, how can they evaluate whether an argument they hear is itself coherent? And if one, say because of lazyness, relies on LLMs from their childhood to fill in all the difficult steps, how will they learn how to do it on their own? Practising has always been the best way to learn things.
> I think it's going to be very interesting to see how all of this plays out, and where the lines are drawn. I suspect that what is causing concern now will, in ten years perhaps, be normal, obvious and not even discussed.
Well said.
That would be all fine and good if it were true. But what will happen in practice is that they will think that they have access to all information, that the corrections may be subtly wrong, that their ideas will no longer really be their ideas, and that sentences and paragraphs will be completed with meaningless or faulty junk.
You didn't make it till the end of the article did you? Based on what you're attacking you made it to the... third paragraph? Fourth?
The goal of late modern and even postmodern school has been to raise compliant soldiers, factory workers, maybe low level clerks, and also later to serve as a day nursery for kids / teenagers. This way of doing things indeed might be reaching its limits for the last half of a century.
But this is somewhat off topic because the article is talking about essays in the context of a university, not school, education.
(Not to mention that parents are still at least partially responsible for their children's education.)
But this is somewhat off topic because the article is talking about essays in the context of a university, not school, education.
(Not to mention that parents are still at least partially responsible for their children's education.)
I guess we might as well stop teaching math too, because everyone has calculators?
I don’t buy this “it is just statistics” dismissal. Practically speaking, any multiple choice question is ultimately a “predict the next character, A, B, or C” challenge. Should we dismiss all multiple choice questions because they are just “predict the next token” tasks?
There is a broader question about whether it is possible to learn from reading. Can a blind person ever really understand “blue”? If not, what can be learnt by reading? Why do we rely on reading and writing so heavily for learning?
Edit: just want to note that this is a great site, really like the author’s approach and style. And maybe if I’ve not learnt from his writing, at the very least it is a good read :-)
There is a broader question about whether it is possible to learn from reading. Can a blind person ever really understand “blue”? If not, what can be learnt by reading? Why do we rely on reading and writing so heavily for learning?
Edit: just want to note that this is a great site, really like the author’s approach and style. And maybe if I’ve not learnt from his writing, at the very least it is a good read :-)
I don’t think I solve multiple choice questions like that.
I consider the letters to correspond to facts about the world, consider the facts symbolically, and then map back to the letters.
It has the same output, but it’s a different process.
I consider the letters to correspond to facts about the world, consider the facts symbolically, and then map back to the letters.
It has the same output, but it’s a different process.
Yes ? We already dismiss them, because they are too easy to hack, and don't give enough insight to the teacher as to the student's learning process ?
In fact now that I think of it, almost the only fields that seem to be using them would be... foreign language teaching ! (Where they are probably appropriate, since so much of it is just pattern memorisation...)
In fact now that I think of it, almost the only fields that seem to be using them would be... foreign language teaching ! (Where they are probably appropriate, since so much of it is just pattern memorisation...)
It's funny because large parts of the world outside the US already do not routinely use multiple choice questions and the motivation behind their use in the US seems to be primarily an easy grading process.
Yes, grading free form writing tasks is a lot more work, but they are also an obviously vastly superior form of exam question.
Yes, grading free form writing tasks is a lot more work, but they are also an obviously vastly superior form of exam question.
I agree. Quantum mechanics is "just statistics", and yet it still causes chemistry to happen.
I think that the argument for qualia is pretty weak - if we can have a conversation about "blueness" with a blind person, we have to admit the possibility of "understanding" existing in some form in LLMs.
Even if it's just a statistical emulation, at what point does a statistical emulation become reality, if it's a good enough emulation down to the metaphorical planck distance.
I think that the argument for qualia is pretty weak - if we can have a conversation about "blueness" with a blind person, we have to admit the possibility of "understanding" existing in some form in LLMs.
Even if it's just a statistical emulation, at what point does a statistical emulation become reality, if it's a good enough emulation down to the metaphorical planck distance.
Anybody who has ever done arithmetic knows that human logic is more than just statistical pattern prediction. Humans can learn and execute algorithms in precise, deterministic ways, which current LLM's fundamentally cannot do.
That's funny.
So being intelligent is executing algorithms deterministically and being a machine is making mistakes and being non deterministic?
How the tables have turned
How the tables have turned
I’m not sure that’s true. Human make mistakes performing math all the time. Human’s ability to accurately perform multiplication is very much a statistical phenomenon. Now you probably dismiss those mistakes as different than the sort of thing that a llm does when following a rigid set of instructions, but to me they seem different in degree but not kind
Why cant LLM's fundamentally execute in a deterministic way? Its a computer running computation on some fixed data. Without randomization parameters for e.g. temperature it would be pretty deterministic?
My understanding is "tech enthousiast" level, so happy to learn.
My understanding is "tech enthousiast" level, so happy to learn.
So for LLMs like ChatGPT, one issue with doing arithmetic is that the input is tokenised, so it doesn't "see" the individual digits in numbers. That will make it harder for it to learn addition, multiplication etc. You can see what the inputs to the model might look like here: https://platform.openai.com/tokenizer
So for example, the text "123456789" is tokenised as "123", "45", "67", "89", and the actual input to the model would be the token IDs: [10163, 2231, 3134, 4531]. Whereas the text "1234" is tokenised as "12", "34" with IDs [1065, 2682]. So learning how these relate in terms of individual digits is pretty hard, as it never gets to see the individual digits.
So for example, the text "123456789" is tokenised as "123", "45", "67", "89", and the actual input to the model would be the token IDs: [10163, 2231, 3134, 4531]. Whereas the text "1234" is tokenised as "12", "34" with IDs [1065, 2682]. So learning how these relate in terms of individual digits is pretty hard, as it never gets to see the individual digits.
I think to extend on the question, though, the fundamental answer is "There is nothing stopping the LLM from containing the embedding of all basic math", with the proviso that tokenization makes it vanishingly unlikely (perhaps in the current generation, or within reasonable resource limits).
I see it analogous to asking a human why they don't just "learn all the answers to simple arithmetic involving integers below 10,000" - you possibly could, it would just be a huge waste of time when you can instead learn the algorithm directly. Of course, LLMs are inherently a layer on top of an existing system which solves those problems quite well already, so it'd be somewhat silly there too.
I see it analogous to asking a human why they don't just "learn all the answers to simple arithmetic involving integers below 10,000" - you possibly could, it would just be a huge waste of time when you can instead learn the algorithm directly. Of course, LLMs are inherently a layer on top of an existing system which solves those problems quite well already, so it'd be somewhat silly there too.
A congenitally blind person ? For a quite weak form of "understanding" then... (and they might still have some form of built-in but unused/repurposed specialized neural circuitry for sight !)
Our consciousness is biochemical statistics. It's a weak argument to dismiss AI as "not like us".
https://clementneo.com/posts/2023/02/11/we-found-an-neuron
https://clementneo.com/posts/2023/02/11/we-found-an-neuron
It's a little frustrating when the author calls out "technofetishistic" faith in large language models[1] and then goes on to misunderstand their capabilities in the opposite direction. I'm no "true believer", they're just useful tools for modeling text generation, but to underestimate them is a shame.
> Ideally, the essay writer has first observed their subject, then drawn some sort of analytical conclusion about that subject, then organized their evidence in a way that expresses the logical connections between various pieces of evidence, before finally communicating that to a reader in a way that is clear and persuasive.
Of the elements discussed about "writing an essay" above, none are clearly impossible for a large language model to do at the present, despite the author's insistence that some of them are impossible. If the subject is something in writing (and you can pass it in the prompt), or is written about in the training set, then the model has indeed "observed" it, and to get it to draw analytical conclusions all you have to do is ask nicely and correctly, maybe using those words. Again, with organizing evidence, you merely must ask for it by name, rather than hoping that the model understands what you mean when you say "write an essay" - ask for "organize the evidence in a way to support/contradict the conclusion X", and finally of course communicating cogently to the reader is something the writer correctly understands is possible even with a trivial prompt.
Sure, it moves the challenge from "write an essay" to "design a prompt to get the language model to write an essay", but that's "not writing an essay", or is it?
It is immensely frustrating that people feel like a large language model is "cheating", but also don't consider using word processing software with grammar and spelling correction "cheating", even though they use fundamentally the same processes behind the scene and differ only in the user interface, really.
[1] I don't like the term AI, I prefer large language model because it is unambiguous about the properties of the system - AI implies thought processes, which is a distraction at best from the useful properties of a complex statistical model.
> Ideally, the essay writer has first observed their subject, then drawn some sort of analytical conclusion about that subject, then organized their evidence in a way that expresses the logical connections between various pieces of evidence, before finally communicating that to a reader in a way that is clear and persuasive.
Of the elements discussed about "writing an essay" above, none are clearly impossible for a large language model to do at the present, despite the author's insistence that some of them are impossible. If the subject is something in writing (and you can pass it in the prompt), or is written about in the training set, then the model has indeed "observed" it, and to get it to draw analytical conclusions all you have to do is ask nicely and correctly, maybe using those words. Again, with organizing evidence, you merely must ask for it by name, rather than hoping that the model understands what you mean when you say "write an essay" - ask for "organize the evidence in a way to support/contradict the conclusion X", and finally of course communicating cogently to the reader is something the writer correctly understands is possible even with a trivial prompt.
Sure, it moves the challenge from "write an essay" to "design a prompt to get the language model to write an essay", but that's "not writing an essay", or is it?
It is immensely frustrating that people feel like a large language model is "cheating", but also don't consider using word processing software with grammar and spelling correction "cheating", even though they use fundamentally the same processes behind the scene and differ only in the user interface, really.
[1] I don't like the term AI, I prefer large language model because it is unambiguous about the properties of the system - AI implies thought processes, which is a distraction at best from the useful properties of a complex statistical model.
> also don't consider using word processing software with grammar and spelling correction "cheating", even though they use fundamentally the same processes behind the scene and differ only in the user interface, really.
i think it's a stretch to claim that spelling/grammar correction and large language models are similar or fundamentally the same.
that's like saying the horse-drawn carriage is fundamentally the same as a plane. Sure, they both move people around, but the differences are much larger than the similarities.
i think it's a stretch to claim that spelling/grammar correction and large language models are similar or fundamentally the same.
that's like saying the horse-drawn carriage is fundamentally the same as a plane. Sure, they both move people around, but the differences are much larger than the similarities.
It's a little bit more like saying an F1 car is fundamentally the same as a Model T.
Spellcheck was one of the first language models (albeit miniscule) in widespread use. You can draw a direct line from bloom filters for spellcheck in resource constrained environments to predictive text and thence on to LLMs.
Spellcheck was one of the first language models (albeit miniscule) in widespread use. You can draw a direct line from bloom filters for spellcheck in resource constrained environments to predictive text and thence on to LLMs.
I think this is more semantics than useful discussion at this point.
Spellcheckers for the msot part act on what you made. ChatGPT in this case is used to generate stuff, so you don't have to make it.
That's a huge difference and the first analogy is more fitting than the Model T one.
Spellcheckers are more like wheelbarrows, they don't move anything by themselves. ChatGPT is like an airplane with a broken autopilot.
Spellcheckers for the msot part act on what you made. ChatGPT in this case is used to generate stuff, so you don't have to make it.
That's a huge difference and the first analogy is more fitting than the Model T one.
Spellcheckers are more like wheelbarrows, they don't move anything by themselves. ChatGPT is like an airplane with a broken autopilot.
So then don't torture students by making them write long essays and waste their valuable time. Make it the task expressing the argument on a back of a napkin. More thought, less time wasted on writing text-amount. Also less of the teacher's time wasted, do they really read 100x student 3 pages, weekly? Don't they have anything better to do?
Word counts in essays in my experience are maximums not minimums.
For an advanced student writing about sufficiently complex topic, the challenge is to develop an argument in less than 5000 words, rather than to reach 5000 words.
And there is a big difference between writing an outline (which is valuable and important!) and actually expressing those ideas fully. The latter forces you to clarify your ideas much more precisely - which is a really valuable experience in exploring and understanding the details of a topic.
For an advanced student writing about sufficiently complex topic, the challenge is to develop an argument in less than 5000 words, rather than to reach 5000 words.
And there is a big difference between writing an outline (which is valuable and important!) and actually expressing those ideas fully. The latter forces you to clarify your ideas much more precisely - which is a really valuable experience in exploring and understanding the details of a topic.
This assumes that the student places the class and knowledge (and writing effort) high in their list of priorities. I'd imagine that the people who are enticed to use an AI solution for writing an essay would have otherwise found writing it way down on their list of desires and would therefore be using many of the tried-and-true tricks to pad out their work.
Well sure, if a student doesn't engage meaningfully with the essay and just writes a load of waffle to fill the word count, they won't learn as much and they'll get a bad mark.
In that case sure, the whole exercise is kind of a waste of time, but you could say that about basically any educational method surely? You can take a horse to water and all that...
In that case sure, the whole exercise is kind of a waste of time, but you could say that about basically any educational method surely? You can take a horse to water and all that...
It's not fair to assume that's what this teacher asks of their students. Word minimums may be useful for children as a crude deterrent against laziness, but any serious student or teacher would recognize a good essay when they see one, no matter the length (though fitting a good and interesting essay on the back of a napkin would be almost unthinkably prolific).
"I have made this letter longer than usual, only because I have not had the time to make it shorter." - Blaise Pascal
"I have made this letter longer than usual, only because I have not had the time to make it shorter." - Blaise Pascal
There's nothing to assume because he describes what he asks of his students towards the last part of the essay.
But people who think ChatGPT is great usually aren't capable of reading that far :P
But people who think ChatGPT is great usually aren't capable of reading that far :P
A word count is also a signal to the student of what level of depth the teacher is hoping for.
For most interesting questions you could write anything between a few sentences and a full book, depending on how much you develop and defend your arguments, evaluate other possible answers, etc.
The word count is a signal on where on that spectrum you should be aiming.
For most interesting questions you could write anything between a few sentences and a full book, depending on how much you develop and defend your arguments, evaluate other possible answers, etc.
The word count is a signal on where on that spectrum you should be aiming.
If we see Pascal's quote as a pretext, it could be used to justify writing longer texts, even when brevity is possible and appropriate. However, by interpreting the quote as a genuine apology for the length of his letter, we can recognize the importance of clarity and conciseness in communication, and strive to make our own messages as clear and concise as possible. This highlights the importance of interpreting someone's words in the right context, and not using them to justify behaviors or attitudes that are not aligned with the speaker's true intentions. By being mindful of how we interpret others' words, we can avoid misunderstandings and communicate more effectively.
Sorry for the potential insult, but your paragraph feels very ChatGPT-like to me. Did I get that right?
Yes, it is a gray wall, a building of formal structure of literature.
In the end, it is a tool - education system, academia, not chatGPT- , what we see is representation.
For current -common- attention span, it is way too long.
For current perception of speed of time, it is very expensive.
Can we complain ?
Tool that we shape.
We may just need to change the representation.
Funny thing, in 2023 it takes a 8000 word essay (45Kb) to convey why chatGPT's essay is not really an essay. A mere 5 years ago this would have been sci-fi territory.
> But it is only pretend, there is no mind there and that is the key to understanding what ChatGPT is (and thus what it is capable of).
This is where an educator thinks they know AI better than people who work on it. A-priori decision it can't work. What they miss when they do that is the special quality of the training set. Language has special properties that lead to the emergence of abilities previously only possible for humans.
Humans for example like to read fiction. It's all pretend, but enjoyable, maybe also instructive. We can simulate other people and their actions, we can infer their emotional states and motives. But it's still all pretend. What if language models can do the same - infer our emotional states, and respond in kind? It would be like generating a novel, something it has seen plenty of in the training data.
The magic dust is not in the model and the next token prediction task, but in what data the model was trained on. Our language, our mind stream, taken from real experiences.
Language has the ability to create things like chatGPT, language can create modern capable humans from just babies. The author missed language and only saw flaws on the model.
Already, by coupling search with language model like bingChat we see even more extreme cases. Now it knows you said things about it on Twitter. These are already going beyond pure LLM. They are going to have a whole host of apps: calculator, calendar, search engine, python and JS execution engines, simulators, recursive calls, language chains, games, memory modules, episodic memory, knowledge bases, etc. I call this paradigm "language model with toys". And they will have this discussion and all the other discussions about it in the next training corpus, thus gaining a way to create a Self. It will have a self because we talk about it like a human.
> But it is only pretend, there is no mind there and that is the key to understanding what ChatGPT is (and thus what it is capable of).
This is where an educator thinks they know AI better than people who work on it. A-priori decision it can't work. What they miss when they do that is the special quality of the training set. Language has special properties that lead to the emergence of abilities previously only possible for humans.
Humans for example like to read fiction. It's all pretend, but enjoyable, maybe also instructive. We can simulate other people and their actions, we can infer their emotional states and motives. But it's still all pretend. What if language models can do the same - infer our emotional states, and respond in kind? It would be like generating a novel, something it has seen plenty of in the training data.
The magic dust is not in the model and the next token prediction task, but in what data the model was trained on. Our language, our mind stream, taken from real experiences.
Language has the ability to create things like chatGPT, language can create modern capable humans from just babies. The author missed language and only saw flaws on the model.
Already, by coupling search with language model like bingChat we see even more extreme cases. Now it knows you said things about it on Twitter. These are already going beyond pure LLM. They are going to have a whole host of apps: calculator, calendar, search engine, python and JS execution engines, simulators, recursive calls, language chains, games, memory modules, episodic memory, knowledge bases, etc. I call this paradigm "language model with toys". And they will have this discussion and all the other discussions about it in the next training corpus, thus gaining a way to create a Self. It will have a self because we talk about it like a human.
> Language has special properties that lead to the emergence of abilities previously only possible for humans.
Citation needed.
> Humans for example like to read fiction. It's all pretend, but enjoyable, maybe also instructive. We can simulate other people and their actions, we can infer their emotional states and motives.
Humans like to read fiction that is extremely rooted in our actual experience of the world. We don't read scifi about 5-dimensional universes with no protagonist that mostly behaves like people. Even on this very blog you'll find excellent articles on why Rings of Powers is bad because it's not believable, and why Lord of the Rings is great because it is (and rooted in much historical knowledge).
> Language has the ability to create things like chatGPT, language can create modern capable humans from just babies.
No? Babies grow into adults after years of experiencing the real world at every instant, as well as the social world. Language starts as a tool to express basic concepts that are very much grounded in the real world, and abstraction comes years later. It's not language that makes babies into adults!
Citation needed.
> Humans for example like to read fiction. It's all pretend, but enjoyable, maybe also instructive. We can simulate other people and their actions, we can infer their emotional states and motives.
Humans like to read fiction that is extremely rooted in our actual experience of the world. We don't read scifi about 5-dimensional universes with no protagonist that mostly behaves like people. Even on this very blog you'll find excellent articles on why Rings of Powers is bad because it's not believable, and why Lord of the Rings is great because it is (and rooted in much historical knowledge).
> Language has the ability to create things like chatGPT, language can create modern capable humans from just babies.
No? Babies grow into adults after years of experiencing the real world at every instant, as well as the social world. Language starts as a tool to express basic concepts that are very much grounded in the real world, and abstraction comes years later. It's not language that makes babies into adults!
I guess that his point is that language is partially genetically transmitted (the universal language bits of it), and is also the bedrock of our social world ?
He's already a few weeks behind. Bing's new AI can query Bing's search engine. So it hasn't "burned the library". It can reach out for new information.
That's going to be really important. Large language models can already summarize documents. How long before you can tell a system "Read this book. Then I will have some questions about it"? Once you have that, you can apply the base model to domain-specific problems.
That's going to be really important. Large language models can already summarize documents. How long before you can tell a system "Read this book. Then I will have some questions about it"? Once you have that, you can apply the base model to domain-specific problems.
I think you missed the argument about the two Roman Empire books. ChatGPT was clearly trained on them, or at least metadata about them, yet it couldn't produce an intuitive response about their relationship. And it couldn't even return the top search result that explained it.
Sure, these issues will be fixed eventually, but the point of the article is that it can't replace the thinking of a student to write a quality essay. It might be able to regurgitate search results which will need to be fact checked, so at best it's useful as a research tool.
BTW, Bing's AI is a really poor counterexample, as it can't even return search results reliably.
Sure, these issues will be fixed eventually, but the point of the article is that it can't replace the thinking of a student to write a quality essay. It might be able to regurgitate search results which will need to be fact checked, so at best it's useful as a research tool.
BTW, Bing's AI is a really poor counterexample, as it can't even return search results reliably.
> there is no mind there
Yesterday I read Wolfram's piece and he wrote:
> I think, is that language is at a fundamental level somehow simpler than it seems.
If Wolfram is true we could argument that GPT-3 was successful in capturing an at least an important aspect of language. And if we also presume that language is part of the mind, then we should refute the statement that there is [absolutely] no mind in GPT-3.
However, GPT-3's mind, if at all, is not human. And this is difficult for us to keep in mind. After all we anthropomorphize everything and their grandsomething.
Even I catch myself having the urge to be polite to ChatGPT and saying thank you. Or, the last time I even told ChatGPT not to worry about something. I know exactly that this is just weird and incongruous, but it did make me feel better, so I just did that. YOLO.
Yesterday I read Wolfram's piece and he wrote:
> I think, is that language is at a fundamental level somehow simpler than it seems.
If Wolfram is true we could argument that GPT-3 was successful in capturing an at least an important aspect of language. And if we also presume that language is part of the mind, then we should refute the statement that there is [absolutely] no mind in GPT-3.
However, GPT-3's mind, if at all, is not human. And this is difficult for us to keep in mind. After all we anthropomorphize everything and their grandsomething.
Even I catch myself having the urge to be polite to ChatGPT and saying thank you. Or, the last time I even told ChatGPT not to worry about something. I know exactly that this is just weird and incongruous, but it did make me feel better, so I just did that. YOLO.
> In that sense, ChatGPT’s greatest limitation is that it doesn’t know anything about anything; it isn’t storing definitions of words or a sense of their meanings or connections to real world objects or facts to reference about them.
> ChatGPT is, in fact, incapable of knowing anything at all.
All my knowledge about modern high-energy physics comes from reading about them. There are a great many things that I know of only second-hand, and not direct observation. Theoretically, any AI's knowledge of these things cannot be worse than mine, then.
> ChatGPT is, in fact, incapable of knowing anything at all.
All my knowledge about modern high-energy physics comes from reading about them. There are a great many things that I know of only second-hand, and not direct observation. Theoretically, any AI's knowledge of these things cannot be worse than mine, then.
Your reading comprehension of these texts is probably more than just the statistical words correlations that LLMs only care about.
I asked chatgpt to generate the different orders of 5 words, it failed at it. Then I ask it to generalize the problem and then it refers to permutations (giving the formula). Then I ask it to use that information to generate the differents orders and then the chatgpt answer the original problem correctly. So it can use some of the information that it can't understand.
> ChatGPT is, in fact, incapable of knowing anything at all.
this is foundamentally wrong. I've been instructing chat gpt about wargame rules, and it's understanding those rules well enough to run rounds of combat. that is far beyond what this article pretends chatgpt to be (a massive pretrained markov chain) as all that learning is happening after training
heck, you can make chatgpt pretend to be something that isn't, exhibiting creativity that is well beyond previous llm.
> All it knows, all it knows are the statistical relationships of how words appear together
again, you can ask nonsense, and it will not spew nonsense back. it has a cursory understanding of what nonsense is, and will not fall for easy traps (i.e. what year apollo 7 landed on the moon) that would trick straighforward probabilistic models
all in all, this article toned down my respect for the blog down two pegs, if he's so wrong about this topic, how much was he wrong on other topics where I had no understanding and took his word for good?
this is foundamentally wrong. I've been instructing chat gpt about wargame rules, and it's understanding those rules well enough to run rounds of combat. that is far beyond what this article pretends chatgpt to be (a massive pretrained markov chain) as all that learning is happening after training
heck, you can make chatgpt pretend to be something that isn't, exhibiting creativity that is well beyond previous llm.
> All it knows, all it knows are the statistical relationships of how words appear together
again, you can ask nonsense, and it will not spew nonsense back. it has a cursory understanding of what nonsense is, and will not fall for easy traps (i.e. what year apollo 7 landed on the moon) that would trick straighforward probabilistic models
all in all, this article toned down my respect for the blog down two pegs, if he's so wrong about this topic, how much was he wrong on other topics where I had no understanding and took his word for good?
So, would you say that the author here is displaying known AI pitfalls : extrapolating nonsense from a too limited corpus of information, overconfidence while being very good at sounding confident ? ;p
More seriously, I have so far no reason to believe that he's wrong about neither him nor his students having managed to make ChatGPT produce an even passable essay, so if you want to prove him wrong...
More seriously, I have so far no reason to believe that he's wrong about neither him nor his students having managed to make ChatGPT produce an even passable essay, so if you want to prove him wrong...
Knowing is putting in relationship with, IMHO. You know what a table is, by were it usually is placed, differences with a chair, differences with a cat, materials it can have, usage given by people. In summary, how it exists in the context of something else.
If we imagine a universe with only one thing (difficult because we would be there too, but bear with me), what could we know about that one thing? It would be the whole universe, all and nothing at the same time.
I would like to propose an alternative take on the same: we also are incapable of knowing. We are just putting in relationship one thing with others, in a similar way as ChatGPT, but on a much bigger scale. And that is fine.
We are still special: we have sensors (senses), we can move, feel pain, but from the point of view of knowledge we are still comparing things and putting them in relationship to one another.
If we imagine a universe with only one thing (difficult because we would be there too, but bear with me), what could we know about that one thing? It would be the whole universe, all and nothing at the same time.
I would like to propose an alternative take on the same: we also are incapable of knowing. We are just putting in relationship one thing with others, in a similar way as ChatGPT, but on a much bigger scale. And that is fine.
We are still special: we have sensors (senses), we can move, feel pain, but from the point of view of knowledge we are still comparing things and putting them in relationship to one another.
> if he's so wrong about this topic
This sounds very arrogant. What is your experience grading college essays? (In other words, you are talking about completely differet use-cases)
This sounds very arrogant. What is your experience grading college essays? (In other words, you are talking about completely differet use-cases)
his cursory understanding of chatgpt is easily falsiable
> if he's so wrong about this topic, how much was he wrong on other topics where I had no understanding and took his word for good?
No value judgement on this specific case, but this is commonly known as Gell-Mann amnesia: http://www.wikibin.org/articles/gell-mann-amnesia-effect.htm...
No value judgement on this specific case, but this is commonly known as Gell-Mann amnesia: http://www.wikibin.org/articles/gell-mann-amnesia-effect.htm...
Eh. Not really ? Because while grading history essays is his domain of expertise, ChatGPT and LLMs are not.
Agree. The guy is in deep cope territory.
So then don't torture students by making them write long essays and waste their valuable time. Make it the task expressing the argument on a back of a napkin. More thought, less time wasted on writing stuff no one ever read properly.
A lot of time writing is a personal endeavor. A way to communicate with yourself rather than others.
It allows you to lay out your chaotic thoughts in a clear way. It's the foundation for critical thinking.
It'll be a very sad world where kids don't atleast practice this skill.
It may be possible to resist something predicted, expected, feared.
I couldn't quite get what the article wanted to reveal, except for the already spoken and known things?
it just makes me sad and angry that people discount their pets experience because they are afraid of over-anthropomorphizing. we are animals which evolved just as our nearby mammalian friends. yes their experience is very different but not in any way fundamentally alien. sure they aren't jealous but they could experience anxiety and many other forms of suffering or excitement and pleasure just like we can. why negate that?
if we can't understand that, I have a deep horror for the atrocities we may commit if/when we do create machine subjectivity.
if we can't understand that, I have a deep horror for the atrocities we may commit if/when we do create machine subjectivity.
One of my cats gets jealous all the time. Whenever I pet her sister she comes running to get pets too. If I just call for her first she doesn't care. Looks like jealousy to me x)
My friend's cat never wanted me to pet her and wouldn't sit near me or anyone much. Now that my friend and his wife have a baby who I assume gets all the attention, that cat says what's up to me every time and wants me to pet her nonstop. She straight up got jealous.
Re: ChatGPT, what can it offer in entertainment for house pets?
Re: ChatGPT, what can it offer in entertainment for house pets?
yeah sorry I don't mean to single out a particular emotion. I give probably a lot more emotional legitimacy to other animals than many people.
It‘s still not clear to me why any subjective experience would arise in a computing process. We know it happens for humans, we can presume that it happens for mammals due to the biological similarity and common history. Unfortunately we don‘t understand it so far, but until we do, the burden of proof is on those who claim without evidence that there is consciousness in machines.
There's no "consciousness" there. There's consciousness in what it is emulating, though, and if you give it enough of that, it will be indistinguishable from an actual consciousness, and so it may very well be that we must treat it as conscious.
There's a philosophical concept, "zimboes", which is useful here - philosophical zombies which believe that they are conscious, and believe that they experience subjective understanding. I don't follow all the arguments, but from what I understand, they're used to argue for "if it quacks like a duck" presentations of consciousness, rather than some more abstract criteria.
There's a philosophical concept, "zimboes", which is useful here - philosophical zombies which believe that they are conscious, and believe that they experience subjective understanding. I don't follow all the arguments, but from what I understand, they're used to argue for "if it quacks like a duck" presentations of consciousness, rather than some more abstract criteria.
> There's consciousness in what it is emulating
This begs the question (if you made it to the footnotes of the article discussed here, you will know what I mean).
More precisely, you start your argument by begging me to take for granted that it can emulate conscious beings - no, you will have to proof that, and not by making it into an axiom.
This begs the question (if you made it to the footnotes of the article discussed here, you will know what I mean).
More precisely, you start your argument by begging me to take for granted that it can emulate conscious beings - no, you will have to proof that, and not by making it into an axiom.
My argument is that there's no way of distinguishing "behaving like conscious beings" from "being conscious". It does not matter whether it is "genuinely emulating conscious beings" or not.
It's a simple extension from "it mimics human text" to "what if it got so much better we can't tell the difference". If we can't tell the difference, definitionally it may as well be conscious. You can disagree with the axiom that it does mimic human text, but generating text with it looks like a pretty good mimicry to me.
You may not agree that it will ever get that good - I would also agree with that. I don't think the proposed scenario is at all likely, certainly not in my lifetime, but if it was, I'd be there lobbying for the large language models to have rights.
It's a simple extension from "it mimics human text" to "what if it got so much better we can't tell the difference". If we can't tell the difference, definitionally it may as well be conscious. You can disagree with the axiom that it does mimic human text, but generating text with it looks like a pretty good mimicry to me.
You may not agree that it will ever get that good - I would also agree with that. I don't think the proposed scenario is at all likely, certainly not in my lifetime, but if it was, I'd be there lobbying for the large language models to have rights.
[deleted]
ChatGPT, please write a bland uninformed critique of LLMs as a boomer overinvested in education saying there is no way a chatbot can write something as complex as a college essay
You demonstrate a pretty superficial understanding of the article. The main point is that writing an essay is just the final step of a process that involves researching a topic, pondering about it, and organizing your thought logically and coherently. The danger of using ChatGPT is that students avoid all these steps, which means they will not learn about the topic, will not think about it, and will not learn to think logically.
> Learn. Don’t cheat.
Teach, then?
I don't know this professor, but many college professors themselves could be replaces with ChatGPT.
Teach, then?
I don't know this professor, but many college professors themselves could be replaces with ChatGPT.
Wow. That is all you got from the article?
Students will find ways to make their lives "easier" by sabotaging their own education with assistants like AI, regardless of the quality of professors teaching them.
Tools like ChatGPT can only regurgitate what was already written, often with confident but wrong takes. They can't offer new perspectives, or inspire students with a different way of thinking about the world. These are not teacher, let alone college professor, replacements.
Students will find ways to make their lives "easier" by sabotaging their own education with assistants like AI, regardless of the quality of professors teaching them.
Tools like ChatGPT can only regurgitate what was already written, often with confident but wrong takes. They can't offer new perspectives, or inspire students with a different way of thinking about the world. These are not teacher, let alone college professor, replacements.
>Tools like ChatGPT can only regurgitate what was already written, often with confident but wrong takes. They can't offer new perspectives, or inspire students with a different way of thinking about the world.
That is not how ChatGPT works. It doesn't just regurgitate data it has seen. It its capable of offering new perspectives. Just ask it.
That is not how ChatGPT works. It doesn't just regurgitate data it has seen. It its capable of offering new perspectives. Just ask it.
> Tools like ChatGPT can only regurgitate what was already written, often with confident but wrong takes.
Yeah, just like many professors
Yeah, just like many professors
> They can't offer new perspectives
i think this remains to be seen.
At least, if you asked chatGPT for recipes, they can produce recipes that aren't pre-existing. So for at least some small domain, it can offer new perspective. Whether those recipes tastes good is a different story...
i think this remains to be seen.
At least, if you asked chatGPT for recipes, they can produce recipes that aren't pre-existing. So for at least some small domain, it can offer new perspective. Whether those recipes tastes good is a different story...
Just as it can offer "new perspectives" on actual facts.
I don’t know why you’re getting downvoted- most of my professors were lazy, ignorant, and had a strong left political bias that clouded their thinking. Sounds exactly like chatgpt - replace lazy with low uptime, ignorant with fake facts, and the left political bias is well documented.
Most teachers have a higher uptime than ChatGTP fortunately.
they pretend to be like minds – like human minds. But it is only pretend, there is no mind there and that is the key to understanding what ChatGPT is
And surely they have no mind like us. But this obsession with whether or not it is like us, seems to miss the point, doesn't it. Is it useful? Despite the writers insistence that it is not, my children provide my with many examples that it is.
For example, my daughter is learning to program and got negative feedback on her commenting style. (which admittedly where mostly absent) So, she asked how to do it better. The teacher gave only non-committal responses, like being helpful to the reader, etc. It is not the kind of instruction that is helpful for her. So she gave the whole program the ChatGTP and asked it to insert comments without changing the code. The next day, she took the output to the teacher and asked if this is what he meant. It was perfect. But now, she knows what is expected of her, and now she can do it without ChatGTP.