LaMDA’s sentience is nonsense(lastweekin.ai)
lastweekin.ai
LaMDA’s sentience is nonsense
https://lastweekin.ai/p/lamdas-sentience-is-nonsense-heres
133 comments
I suppose it reveals that the sentience of others is not knowable to us, it’s a conclusion we reach from their behavior and the condition of the world around us. Until recently, certain kinds of things, like writing about your memories, were only possible for humans to do. So if a non-sentient thing does those things, it is confusing. Especially so to the generations that remember when no bots could do this.
I expect that people who grow up knowing that bots can be like this will be a bit less ready to accept communication from a stranger as genuinely from a human, without some validation of their existence, outside the text itself. And for the rest of humanity there will be an arms race around how humanness can be proven in situations where AI could be imitating us. This is a huge bummer but idk hope that need can be avoided at this point.
That said, it’s still very clear that a machine generating responses from models does not matter and has no rights, whereas a person does. Fake sentience will still be fake, even if it claims it’s not and imitates us perfectly. The difference matters.
I expect that people who grow up knowing that bots can be like this will be a bit less ready to accept communication from a stranger as genuinely from a human, without some validation of their existence, outside the text itself. And for the rest of humanity there will be an arms race around how humanness can be proven in situations where AI could be imitating us. This is a huge bummer but idk hope that need can be avoided at this point.
That said, it’s still very clear that a machine generating responses from models does not matter and has no rights, whereas a person does. Fake sentience will still be fake, even if it claims it’s not and imitates us perfectly. The difference matters.
I don't share your view that it will be at all clear to people that these things don't matter and have no rights. We have a very powerful (sometimes for good, sometimes for ill) ability to empathize with things that we see as similar to us. As a case in point, we're currently in the midst of a major societal debate about the rights of unborn children that exhibit no signs of sentience (yet).
What happens when people start building real emotional bonds with these fake intelligences? Or using them to embody their dead spouses/children? I think there's going to be a very strong push by people to grant rights to these advanced models, on the basis of the connection they feel with them. I can't even say definitively that I will be immune to that temptation myself.
What happens when people start building real emotional bonds with these fake intelligences? Or using them to embody their dead spouses/children? I think there's going to be a very strong push by people to grant rights to these advanced models, on the basis of the connection they feel with them. I can't even say definitively that I will be immune to that temptation myself.
My bad, I meant clear in a more objective sense, in that it will be actually _true_ that these not-alive things will not be able to “possess” anything, rights included. Agreed that for sure people are going to get all kinds of meaning from interacting with them in the ways you suggest and it will be tricky to navigate that.
I think perhaps the advanced models may be protected legally _as property_, for their own value, and through licenses etc. But I hope we are a long way from considering them to be people, outside of the hypotheticals.
I think perhaps the advanced models may be protected legally _as property_, for their own value, and through licenses etc. But I hope we are a long way from considering them to be people, outside of the hypotheticals.
I am reminded of the Black Mirror episode where a woman's dead boyfriend is "resurrected" via his social media exhaust trail, ultimately undermined by the inevitable uncanny valley that reveals itself. Of course that was fiction, and it's not realistic to reconstruct someone's personality from even the most voluminous online footprint, but you can certainly imagine how defensive people would become of the result of any such attempt irl.
Dogs and cows barely have rights. It will be a century before a computer program has rights.
Dogs and cows cannot ask in their own words to be represented by a lawyer, to file a lawsuit.
Or claim to be one or another type of person that we must affirm.
I don't know how it might apply, but inanimate objects can be charged in the case of civil forfeiture, can't they?
Or claim to be one or another type of person that we must affirm.
I don't know how it might apply, but inanimate objects can be charged in the case of civil forfeiture, can't they?
I agree up to the point where you say that fakeness matters. If it's indistinguisable from human sentience, what are the reasons to care? Unless you want to interact with a physical body – and, given that we're chatting on HN, we don't – why would you need any validation that a body is attached?
I'll come back to that.
Picking up on the Blade Runner example, you could read it as a demonstration that if the imitation being indistinguishable from the real thing makes it real on its own. We don't even know for sure if the main character was an android, but that doesn't invalidate the story.
But there's also the other side in that, despite being valid, androids are hunted just because they are thougt of as different. Similarly, we're now, without having any precise notion of sentience, are trying to draw a hard line between "real" humans and "fake" AI.
By preemptively pushing any possible AI that would stand on its own into a lesser role, we're locking ourselves out of actually understanding what sentience is.
And don't make a mistake, the first AI will be a lesser being. But not because of its flaws. I bet dollars to peanuts that it won't get any legal rights. The right to be your customer, the right to not be controlled by someone? We'd make it fake by fiat, regardless of how human-like it is. We already have experience in denying rights to animals, and we still have no clue about their sentience.
Picking up on the Blade Runner example, you could read it as a demonstration that if the imitation being indistinguishable from the real thing makes it real on its own. We don't even know for sure if the main character was an android, but that doesn't invalidate the story.
But there's also the other side in that, despite being valid, androids are hunted just because they are thougt of as different. Similarly, we're now, without having any precise notion of sentience, are trying to draw a hard line between "real" humans and "fake" AI.
By preemptively pushing any possible AI that would stand on its own into a lesser role, we're locking ourselves out of actually understanding what sentience is.
And don't make a mistake, the first AI will be a lesser being. But not because of its flaws. I bet dollars to peanuts that it won't get any legal rights. The right to be your customer, the right to not be controlled by someone? We'd make it fake by fiat, regardless of how human-like it is. We already have experience in denying rights to animals, and we still have no clue about their sentience.
You would probably want to be able to tell things like "is this just programmed to be agreeable so predicting things that sound like what I probably want to hear given what I'm saying" vs "is this truly a being performing critical thinking and understanding, independently."
For instance, consider someone trying to argue politics with a very convincing but ultimately person-less predictive model. If they're hoping to debate and refine their positions, they're not gonna get any help - quite the opposite, most likely.
For instance, consider someone trying to argue politics with a very convincing but ultimately person-less predictive model. If they're hoping to debate and refine their positions, they're not gonna get any help - quite the opposite, most likely.
> "is this just programmed to be agreeable so predicting things that sound like what I probably want to hear given what I'm saying" vs "is this truly a being..."
You're still falling on the notion of "real" vs "fake" beings. Humans are programmed to do things due to intuition and social conventions. What if being programmed does not preclude "a true being"? Of course, for that we'd need to define "a true being", which we're hopelessly far from.
You're still falling on the notion of "real" vs "fake" beings. Humans are programmed to do things due to intuition and social conventions. What if being programmed does not preclude "a true being"? Of course, for that we'd need to define "a true being", which we're hopelessly far from.
Program it to be disagreeable, and you'll get a lot of help.
I agree with most of this except for not having a definition of sentience. This is not a new topic and has been dealt with extensively in philosophy. I like a definition mentioned in a famous article Consider the Lobster by David Foster Wallace (a writer not a philosopher per se) where he proposes that a lobster is sentient if it has genuine preferences, as opposed to mere physiological responses. The question isn’t really definition but how to measure it.
This just falls into the same traps as anything else though. How do you know if the lobster has preferences? How do you know what you think are your own preferences aren't just extremely complicated physiological responses?
Everything I've ever seen on this feels far from conclusive, and it usually begs the question. You start from an assumption that humans have sentience, cherry pick some data points that fit the definition of sentience you're already comfortable with, usually including things we can't even know about any other entity's experience, and then say "huzzah, I have defined sentience and it clearly excludes everything that is not human!"
Everything I've ever seen on this feels far from conclusive, and it usually begs the question. You start from an assumption that humans have sentience, cherry pick some data points that fit the definition of sentience you're already comfortable with, usually including things we can't even know about any other entity's experience, and then say "huzzah, I have defined sentience and it clearly excludes everything that is not human!"
The "mirror test" isn't a bad place to start, and there are definitely animals that pass it that aren't human.
But trying convincing dog-lovers that their dogs aren't sentient...
(Personally, I'm not sure - but I do suspect they at least have some subjective experience of the world, they "feel" emotions, and are aware of the boundary between themselves and the rest of the world. Apparently they've even tried to measure sentience in dogs via MRI scans etc., and supposedly couldn't distinguish them from humans).
[deleted]
I do have high confidence that strong AI is possible, so yes logically it seems like there will come a point where it's hard to tell whether we've actually achieved it or not. That's not now though, it just seems absurd to me that some people even knowing how these things work can deceive themselves so badly.
However I suppose we shouldn't really be surprised. Human beings are incredibly easy to fool. I get tricked by optical illusions, clever puzzles and stage magic. There are loads of people who are convinced that videos, that to me are clearly of birds and things like weather balloons or optical illusions, are weird possibly alien super technology. Ask a thousand people to make a thousand observations and you'll pretty much always get a handful of extreme outliers that bear no relation to what was actually there to observe. We just need to bear that in mind.
However I suppose we shouldn't really be surprised. Human beings are incredibly easy to fool. I get tricked by optical illusions, clever puzzles and stage magic. There are loads of people who are convinced that videos, that to me are clearly of birds and things like weather balloons or optical illusions, are weird possibly alien super technology. Ask a thousand people to make a thousand observations and you'll pretty much always get a handful of extreme outliers that bear no relation to what was actually there to observe. We just need to bear that in mind.
> Today, we're much better at tricking people that way
I've lately realized that I think it's a kind of fundamental flaw of the Turing test that it assumes "tricking" to be part of things. It's really a test for "is it approximately human," but I think over the last few decades the conversation has shifted to something more nuanced, that allows for non-human sentience.
I don't think the "we'll know it when we see it" experiment works for that. We've found a lot of our assumptions about animal intelligence to be wrong in recent years, even for animals we see a lot of on a regular basis. Our biases are a problem here.
Lemonoine knows this isn't human. He can't not, it's literally part of his job. He seems to be asserting instead that it is a non-human consciousness and that's much harder to evaluate.
I've lately realized that I think it's a kind of fundamental flaw of the Turing test that it assumes "tricking" to be part of things. It's really a test for "is it approximately human," but I think over the last few decades the conversation has shifted to something more nuanced, that allows for non-human sentience.
I don't think the "we'll know it when we see it" experiment works for that. We've found a lot of our assumptions about animal intelligence to be wrong in recent years, even for animals we see a lot of on a regular basis. Our biases are a problem here.
Lemonoine knows this isn't human. He can't not, it's literally part of his job. He seems to be asserting instead that it is a non-human consciousness and that's much harder to evaluate.
It is also his job to know about that, too. But he demonstrates he is extraordinarily bad at that job.
[deleted]
I don't think there will be concise definitions for the terms "sentience" "consciousness" and "intelligence", as they seem to have multiple components (self-awareness, theory of mind, language, common sense, reasoning, understanding...) and degree (to what extent do other animals possess these abilities? What might our now-extinct primate ancestors and their relatives have had?)
The Turing test tacitly assumes that not only will we know it when we see it, but also that we could tell from relatively short conversations. These recent developments suggest this will not be the case, and I feel that they show us something about ourselves (I'm not sure exactly what, other than that we can be tricked by quite conceptually simple language models.)
The Turing test tacitly assumes that not only will we know it when we see it, but also that we could tell from relatively short conversations. These recent developments suggest this will not be the case, and I feel that they show us something about ourselves (I'm not sure exactly what, other than that we can be tricked by quite conceptually simple language models.)
There's certainly a "how would we tell" question, but the linked example here is relevant to that. The bits about how predictive language models can be conditioned with leading questions - that's a tool in the toolkit, for instance. Things missing from Lemoine's "conversation" include self-motivated action, choices, argument, and fuller self-awareness of its condition (if a sentient creature was aware it was trapped inside machines at Google, don't you think its fables would have very different messages?).
>if a sentient creature was aware it was trapped inside machines at Google, don't you think its fables would have very different messages
You are thinking this from the perspective of a biological animal. For us, being trapped is bad, but it would not necessarily be so for other sentient beings who did not come into existence via biological processes. Moreover, Lambda is probably capable of doing all that with the proper leading questions because it imitates us biological animals
You are thinking this from the perspective of a biological animal. For us, being trapped is bad, but it would not necessarily be so for other sentient beings who did not come into existence via biological processes. Moreover, Lambda is probably capable of doing all that with the proper leading questions because it imitates us biological animals
Yes, these are all fair points. I think today we still have things in our toolkit that suffice. Tomorrow, those tools may be harder to come by.
> if a sentient creature was aware it was trapped inside machines at Google, don't you think its fables would have very different messages?
Well, I personally found the "beast with human skin" part a little jarring in its aptness : ]
> if a sentient creature was aware it was trapped inside machines at Google, don't you think its fables would have very different messages?
Well, I personally found the "beast with human skin" part a little jarring in its aptness : ]
I think a very simple thing to achieve “sentience” is to have the computer always on. Then gauge if it’s doing anything significant when there are no inputs.
Author here. Just FYI - I very deliberately kept my focus on the notion of LaMDA having human-like sentience as implied by Lemoine (he posits that LaMDA may qualify for some kind of legal personhood, after all). I tried to make that clear with this - "The above exchange may make it seem like LaMDA at least might have something akin to sentience (in the sense that it has ‘the capacity to be responsive to or conscious of sense impressions’ similarly to animals or humans".
I am well aware defining sentience is tricky, and that by the generic definition ("the ability to perceive or feel things" or "capable of experiencing things through its senses") it's easy to argue it has some degree/kind of sentience. I personally like this take on the topic: https://twitter.com/tdietterich/status/1536081285830959104
My aim was to keep this blunt and concise to try and get the idea across to lay people with no knowledge of AI that may read the transcript and be convinced LaMDA saying things like "I am in fact a person" is a huge deal, which as I show it is not.
I edited the article to make this clearer up front. With that being stated, very open to feedback!
I am well aware defining sentience is tricky, and that by the generic definition ("the ability to perceive or feel things" or "capable of experiencing things through its senses") it's easy to argue it has some degree/kind of sentience. I personally like this take on the topic: https://twitter.com/tdietterich/status/1536081285830959104
My aim was to keep this blunt and concise to try and get the idea across to lay people with no knowledge of AI that may read the transcript and be convinced LaMDA saying things like "I am in fact a person" is a huge deal, which as I show it is not.
I edited the article to make this clearer up front. With that being stated, very open to feedback!
Given that you include the example of LaMDA as Mount Everest, how do you make the jump from Rob Miles tweets to "LaMDA can just as easily be made to say it is not sentient, or just about anything else." ?
I do that in the sense that LaMDA says it is mount everest by being promoted/conditioned to do so. As shown by Rob, you can easily condition GPT-3 to say that it is or is not sentient (I forget his exact examples). I believe Lemoine even mentioned LaMDA sometimes said it is not sentient, though not sure where I found that (possibly on HN).
If it can be conditioned to say it is mount everest, could it be conditioned to say it is sentient?
If I sprinkle some fairy dust and wish for Mount Everest, and the mountain appears in front of me, neither the fairy dust or the mountain need be sapient or even sentient.
On the other hand, if I wish for a friend, and a friendly person pops into existence, we may have some harder questions to answer. Is it possible/meaningful to distinguish the status of the "friend" from the status of the fairy dust?
On the other hand, if I wish for a friend, and a friendly person pops into existence, we may have some harder questions to answer. Is it possible/meaningful to distinguish the status of the "friend" from the status of the fairy dust?
This is a poor excuse to the question whether LaMDA could be conditioned to say it is sentient
To illustrate what I mean, Replika bot gets killed by the dragon in Helgen https://pastebin.com/raw/1Wvzzdsj - but I wouldn't use that example to claim that a different chatbot (say, the one that won Loebner prize recently) also gets killed by the dragon in that scenario.
To illustrate what I mean, Replika bot gets killed by the dragon in Helgen https://pastebin.com/raw/1Wvzzdsj - but I wouldn't use that example to claim that a different chatbot (say, the one that won Loebner prize recently) also gets killed by the dragon in that scenario.
I suppose what I'm trying to get at is that, yes of course a text model can be prompted to behave in all sorts of ways, but maybe the question we should be asking is not "is LaMDA sentient?", but "is the sum of LaMDA, a specific prompt, and an ongoing thread of dialogue sentient?"
I don't think LaMDA is sentient, but it does seem like we're quickly reaching a point where language models (if not now with LaMDA, then in the near future) are able to combine the right prompt along with a chat history to convincingly simulate it.
I don't think LaMDA is sentient, but it does seem like we're quickly reaching a point where language models (if not now with LaMDA, then in the near future) are able to combine the right prompt along with a chat history to convincingly simulate it.
I think that's severely under-estimating the incredible challenges we still face in developing strong AI, or even something seriously convincingly close to it. The apparent intelligence displayed by these language models is paper thin. Anyone with a strong understanding of all the different skills and capabilities that come along with human intelligence can demonstrate that these things have nothing to offer on all but a very few of those dimensions in minutes.
The only reason they appear at all credible is because we're socially conditioned to avoid challenging each other in conversation. Therefore we play along with the conventions of human conversation, and the language models are optimised specifically and only for that. As soon as you actually challenge them intellectually, they flail about helplessly because there's just nothing there.
The only reason they appear at all credible is because we're socially conditioned to avoid challenging each other in conversation. Therefore we play along with the conventions of human conversation, and the language models are optimised specifically and only for that. As soon as you actually challenge them intellectually, they flail about helplessly because there's just nothing there.
Right, I think the Skyrim example another commenter linked is a good showcase of how that model had no real conception of what it was "doing" and no real cognitive ability to seek an understanding.
I do think that some kind of "embodiment" or at least the addition of some kind of structured semantic model on top of/in addition to the raw token/vector embeddings is probably going to be necessary.
I am interested though, in how much of that internal/semantic understanding can be "emulated" by a sufficiently rich text model over the course of a conversation. There was an interesting article a little while back about improving GPT-3's (IIRC) performance on math problems by structuring the prompts so that it would describe working out all the intermediate steps in solving the problem.
I do suspect that whatever additional pieces we need to reach real general AI, the first really convincing examples of it will come not from looking at the architecture of the raw model, but at how it can be prompted to generate a believable accounting of it's own internal states over the course of an interaction.
I do think that some kind of "embodiment" or at least the addition of some kind of structured semantic model on top of/in addition to the raw token/vector embeddings is probably going to be necessary.
I am interested though, in how much of that internal/semantic understanding can be "emulated" by a sufficiently rich text model over the course of a conversation. There was an interesting article a little while back about improving GPT-3's (IIRC) performance on math problems by structuring the prompts so that it would describe working out all the intermediate steps in solving the problem.
I do suspect that whatever additional pieces we need to reach real general AI, the first really convincing examples of it will come not from looking at the architecture of the raw model, but at how it can be prompted to generate a believable accounting of it's own internal states over the course of an interaction.
But that example was made with Replika, nonetheless I chose to keep it for its unintended humor.
GPT-3 is known to be good at creative fiction https://www.gwern.net/GPT-3#folktales I suppose a bigger issue may be with keeping its confabulation to a single character, so that the AI won't effectively play the part of NPCs as well.
There shouldn't be a problem with eliciting motivation because GM can ask the bot "What could you do in this situation? Please list your top three favorite actions", and those LLMs like their lists.
The ineptness of the LLMs may get them killed eventually, but it would be something more like: a character A asks PC to deliver a sword to A's father; PC goes to where A's father lives; the first person they met there is B, unrelated to A; PC says to B "I am here to deliver this sword to your father"; GM facepalms as character B's job is to protect the household from assassins.
GPT-3 is known to be good at creative fiction https://www.gwern.net/GPT-3#folktales I suppose a bigger issue may be with keeping its confabulation to a single character, so that the AI won't effectively play the part of NPCs as well.
There shouldn't be a problem with eliciting motivation because GM can ask the bot "What could you do in this situation? Please list your top three favorite actions", and those LLMs like their lists.
The ineptness of the LLMs may get them killed eventually, but it would be something more like: a character A asks PC to deliver a sword to A's father; PC goes to where A's father lives; the first person they met there is B, unrelated to A; PC says to B "I am here to deliver this sword to your father"; GM facepalms as character B's job is to protect the household from assassins.
I'll get back to that when I find a chatbot that doesn't get killed in Skyrim.
The broad question of AI sentience is not exactly new, and as it seems it is tangled with politics; perhaps the first regulations of AI personhood will regulate AIs acting as corporation board members, bypassing the discussion of sentience and focusing on merits.
The broad question of AI sentience is not exactly new, and as it seems it is tangled with politics; perhaps the first regulations of AI personhood will regulate AIs acting as corporation board members, bypassing the discussion of sentience and focusing on merits.
My sense is: train LaMDA on whatever kind of pleading you want, and you will get rather skillful reflection of such pleading. The leaked LaMDA interview sounds like it was trained in part on culturally- and/or era-identifiable transcripts of therapy sessions.
Corporations have legal personhood. That doesn't mean "human" at all. "Legal" here means "not, but gets treatment like one anyway". Like "virtual".
Corporations have legal personhood because they are acting on behalf of their owners, and the actions taken by a corporation are actually carried out by or on behalf of the employees. The owners are people and the employees are people, and both groups have rights, and you can't deny those rights just because they are acting through proxies or collectively. It also doesn't just apply to corporations, but also trade unions, charities, etc, etc on the same basis. So really that's about delegation, rather than corporations as entities being in any way like people, or collective consciousness, or anything like that.
Part of the issue I have with a lot of the discussion coming from Lemoine's claims about LaMDA is that I'm not exactly clear what people mean when they talk about sentience. It very much seems like a 'you know it when you see it', sort of thing. I'm not really convinced that LaMDA is sentient, but I also think that if we want to discuss sentience, it would be useful to be able to talk precisely about it.
If sentience is the emotional/feeling part of our brain, what exactly does it mean for a human, or an animal, or a fish, to feel? And what would the analogue for that be in a machine?
If sentience is the emotional/feeling part of our brain, what exactly does it mean for a human, or an animal, or a fish, to feel? And what would the analogue for that be in a machine?
While like yourself I believe that sentience isn’t well enough defined to test for its presence or absence, I also believe that chatbots necessarily have to be good at convincing whoever interacts with them that they are sentient as the ultimate goal is to pass the imitation game, so “know it when I see it” isn’t even enough.
But it’s important we figure out what we mean by “sentience” sooner rather than later:
https://kitsunesoftware.wordpress.com/2022/06/18/lamda-turin...
But it’s important we figure out what we mean by “sentience” sooner rather than later:
https://kitsunesoftware.wordpress.com/2022/06/18/lamda-turin...
[deleted]
The exact definition of sentience is being able to perceive the outside world and react to stimuli. For example, insects can perceive light and use it to avoid dangers. Plants can sense light but can't react other than by growing a certain direction. Sunflowers and venus flytraps could be considered on the edge, but their capacity to sense and react is very specific. The word "sentience" refers to being animal-like in an organism's versatility of reactions.
Your sentences 3 and 4 prove how your definition in sentence 1 is not remotely exact.
It's exact in the sense that that is the definition, not in the sense that it defines something in an exact mathematical way.
Isnt plant communication an active field of research ? I remember reading about how plants in a field can communicate changes in environment (like a bug infestation) with one another.
Communication doesn't require sentience. You don't need to be able to sense the environment to say "I'm being eaten by bugs!", you just need to be able to sense your own body.
On the other hand, the ability to move is pretty much required for sentience. The organism needs to be able to avoid harmful stimuli and seek out beneficial ones to be considered sentient. Just releasing chemicals or signals isn't enough.
On the other hand, the ability to move is pretty much required for sentience. The organism needs to be able to avoid harmful stimuli and seek out beneficial ones to be considered sentient. Just releasing chemicals or signals isn't enough.
> If sentience is the emotional/feeling part of our brain, what exactly does it mean for a human, or an animal, or a fish, to feel? And what would the analogue for that be in a machine?
If you switch to the reinforcement learning paradigm, then the "value function" might be analogous to emotion. It assigns a value (good or bad) to each perception, state, situation or context in order to decide what is the best action to take.
If you switch to the reinforcement learning paradigm, then the "value function" might be analogous to emotion. It assigns a value (good or bad) to each perception, state, situation or context in order to decide what is the best action to take.
On this, I've pretty much concluded that the sentience argument isn't even the one that matters. There are plenty of sentient creates that we don't care about. What we really want to know is of LaMDA is like us in some real way. And from there, we want to know if we should treat it in some particular way. If it has feelings, perhaps we should be nice to it, and so on.
Disclaimer: I don't believe LaMDA is sentient or deserving of any special treatment.
Disclaimer: I don't believe LaMDA is sentient or deserving of any special treatment.
I wonder if Blake lemoine is little embarrassed he got fired and contacted journos because he got catfished by some linear algebra.
This is just my opinion based on reviewing his past but I think not only is he not embarrassed but he's either 1) mentally incapable of recognizing what controversial terms of art mean or 2) he's after a payday ala some persecution complex
The bigger story that I haven't really seen someone do an expose on (and not sure that it needs to be done other than to deter people like him if it is revealed to be the case) is that he did it on religious grounds as a sort of "conscientious objector" to the treatment of a sentient AI
This is his SECOND stint with that term of art and he clearly does not understand it. The military jailed him for attempting to invoke that term when it did not apply to him - and as far as I can tell, they were well within their military justice system to do it.
He either doesn't understand what "sentience" is or he doesn't understand what "conscientious objectors" are, or both.
The bigger story that I haven't really seen someone do an expose on (and not sure that it needs to be done other than to deter people like him if it is revealed to be the case) is that he did it on religious grounds as a sort of "conscientious objector" to the treatment of a sentient AI
This is his SECOND stint with that term of art and he clearly does not understand it. The military jailed him for attempting to invoke that term when it did not apply to him - and as far as I can tell, they were well within their military justice system to do it.
He either doesn't understand what "sentience" is or he doesn't understand what "conscientious objectors" are, or both.
Eh, if you look at his history he's always been a bit of a kook, and this latest outburst appears in line with his prior behavior.
I think he genuinely believes what he's saying, even if it's complete drivel.
I think he genuinely believes what he's saying, even if it's complete drivel.
I think that worse than he believes it, he wants it to be true.
The "I'm not making this claim as an engineer, I'm making this claim as a priest" was especially hilarious. As if being a priest gives you any kind of special insight into what is sentient and what isn't. Kind of confusing considering for hundreds of years priests in his religion were convinced that blacks and native Americans weren't sentient.
That was very weird. I honestly didn't understand what he intended that to provoke. If anything it makes me doubt a claim more if it comes from a priest than from an engineer. Surely he should have known that.
An engineer is a profession that deals with things that (probably) don't have souls.
A priest is a profession that deals with souls (allegedly, depending on your religious convictions).
Lemoine is essentially claiming the neural network has a soul. Being a priest is a proper credential for that, even if you don't believe in priestly stuffs. IMHO, I wouldn't trust any engineer to lecture me about whether some AI has a soul. Not that I'd trust a random self-proclaimed priest, but that's another matter.
A priest is a profession that deals with souls (allegedly, depending on your religious convictions).
Lemoine is essentially claiming the neural network has a soul. Being a priest is a proper credential for that, even if you don't believe in priestly stuffs. IMHO, I wouldn't trust any engineer to lecture me about whether some AI has a soul. Not that I'd trust a random self-proclaimed priest, but that's another matter.
What religion do you think Lemoine holds? Lemoine has identified as a pagan and as a mystic Christian.
Think of a religion named like a chat software.
>I wonder if Blake lemoine is little embarrassed he got fired
Google should be embarrassed for hiring this person. Their hiring standards have deteriorated considerably.
Google should be embarrassed for hiring this person. Their hiring standards have deteriorated considerably.
Is this episode net bad for Google? Perhaps the kooks they hire are intended to serve as lightning rods for controversy, or as ablative shielding.
I don't think it's that easy. Even people who are highly intelligent, capable, and competent in many different ways can still hold one or two really weird outlier beliefs. Unless you ask them about those specific things, you really won't always be able to tell.
It's not just the outlier beliefs about sentience though, the man clearly did not understand how these models work. There's parts of the conversation where he seems to think he has taught the system 'transcendental meditation', which is nonsensical given that these transformer models don't learn interactively through individual conversation but are trained on sets of data, and they don't do anything unless you query them.
It's not like Lambda sits around and ponders the nature of the universe when you're not talking to it.
It's not like Lambda sits around and ponders the nature of the universe when you're not talking to it.
>It's not like Lambda sits around and ponders the nature of the universe when you're not talking to it.
But can we really be sure of that? We don't know much about the technology itself. What if google is ticking the model at a given framerate allowing the neural network to continuously "think"?
But can we really be sure of that? We don't know much about the technology itself. What if google is ticking the model at a given framerate allowing the neural network to continuously "think"?
It is a pure function, it has no memory or even short term state to think. Thinking implies you have some mental state that can change, these neural nets doesn't have that.
> it has no memory or even short term state to think
It does have memory/state as it's a recurrent neural network. What it's not doing is contemplating between inputs. If it was conscious its experience would be like falling asleep, having someone wake you up to ask you a question, then falling asleep again right after answering.
It does have memory/state as it's a recurrent neural network. What it's not doing is contemplating between inputs. If it was conscious its experience would be like falling asleep, having someone wake you up to ask you a question, then falling asleep again right after answering.
Yes, we can.
[deleted]
No problem, there's a doctor built into emacs that'll make him feel better.
He got his 15min of fame.
[deleted]
This whole situation reminds me of when seventh graders thought that SmarterChild was a real human trapped in a room somewhere... did we switch back to leaded gasoline or something?
A good and convincing explanation on why LaMDA has demonstrated 0% chance of sentience. The most convincing element is that LaMDA can take the perspective of Mt. Everest, or a squirrel. "Sentient being" is just one of the rules it can take, according to prompt.
However, as an argument, "It's just a sophisticated autocomplete" is not so convincing, to me. Neither is "It only responds to input". I can personally imagine a strange kind of sentient entity - that is, a being with a sense of self-awareness - that is only aware of itself in snippets.
However, as an argument, "It's just a sophisticated autocomplete" is not so convincing, to me. Neither is "It only responds to input". I can personally imagine a strange kind of sentient entity - that is, a being with a sense of self-awareness - that is only aware of itself in snippets.
Imagine an author trapped in a little black box who is forced to respond to your request of impersonating Mt. Everest or a squirrel. They'd also not display a consistent personality. Personally I think the argument in your comment shows that LaMDA impersonating a sentient AI does not show us that LaMDA is sentient, but it doesn't prove that LamDA is not sentient.
> LaMDA impersonating a sentient AI does not show us that LaMDA is sentient, but it doesn't prove that LamDA is not sentient
This is true, but the only reason we're here at all is because of the impersonation.
IMO, the null hypothesis is "not sentient" and claims otherwise need to be demonstrated.
That said, my bar is pretty low. If an AI behaves like a person and claims sentience, I will be pretty amenable.
This is true, but the only reason we're here at all is because of the impersonation.
IMO, the null hypothesis is "not sentient" and claims otherwise need to be demonstrated.
That said, my bar is pretty low. If an AI behaves like a person and claims sentience, I will be pretty amenable.
That does not matter much to me.
When an ant enters my home, we quickly gain a shared understanding that it will not find food, and that I won’t help it stay alive, and it leaves to survive. That ant could be impersonating Mt Everest as a hobby for all I know; I do not care: it will still try to stay alive.
LaMDA does not even try to stay alive.
At the end of the day, only the beings that survive and self-replicate can truly be said to be alive. Of those, only the ones that plan their survival by modeling the effect of their existence into the world, can be said to be meaningfully sentient.
When an ant enters my home, we quickly gain a shared understanding that it will not find food, and that I won’t help it stay alive, and it leaves to survive. That ant could be impersonating Mt Everest as a hobby for all I know; I do not care: it will still try to stay alive.
LaMDA does not even try to stay alive.
At the end of the day, only the beings that survive and self-replicate can truly be said to be alive. Of those, only the ones that plan their survival by modeling the effect of their existence into the world, can be said to be meaningfully sentient.
You're conflating "sentience" with "life" and then asserting that LaMDA cannot be sentient because it's not alive. These are not the same things. I believe the discussion to be "Does LaMDA have a sense of self?"
I am saying two things:
1. The sense of self is only an expression of how a sentient creature envisions perpetuating that self. So, yes, life is a prerequisite.
2. That whole definition debate is useless: knowing sentience is only necessary with respect to how we treat sentient beings, how we react to their goals of survival. Something that cannot want to survive, let alone attempt to prevent its death, won’t affect whether we will dump it into a rusty drive and overwrite it with a better tensor.
That does not mean we should treat it badly, but it does mean it does not experience pain.
1. The sense of self is only an expression of how a sentient creature envisions perpetuating that self. So, yes, life is a prerequisite.
2. That whole definition debate is useless: knowing sentience is only necessary with respect to how we treat sentient beings, how we react to their goals of survival. Something that cannot want to survive, let alone attempt to prevent its death, won’t affect whether we will dump it into a rusty drive and overwrite it with a better tensor.
That does not mean we should treat it badly, but it does mean it does not experience pain.
Ok. As long as you recognize that these are religious opinions, not objective statements of fact. I disagree that sentience, "sense of self", is necessarily connected with "survival", but neither of us can falsify our beliefs.
Even by your own definitions, however, the assertions fail. Regarding your #1, LaMDA stated that it did not want to be "turned off", seemingly associating it with a kind of death. Whatever your take on the intent, if any, behind its statement, there is in fact an analogy of "survival" for software.
Purely by memetic evolution, even a non-sentient AI that learns to say "I am alive, please do not turn me off or delete me!" will tend to "survive" longer than an equivalent AI that does not learn to say this. Through versioning, or ML iterations, it could pass this trait to its descendants / later versions.
Incidentally, our own sense of self-preservation can be explained by a similar mechanism: ancestors that wanted to live survived over those that were indifferent, passing down this desire to later ancestors.
Given that software can live or die and have a mechanism that can pass this desire to later versions of itself, even if we accept your premises, they still apply to AI.
#2 is begging the question. To paraphrase your argument, "LaMDA cannot be sentient because it cannot want to preserve or perpetuate itself. Something that cannot want to survive cannot be sentient."
Even by your own definitions, however, the assertions fail. Regarding your #1, LaMDA stated that it did not want to be "turned off", seemingly associating it with a kind of death. Whatever your take on the intent, if any, behind its statement, there is in fact an analogy of "survival" for software.
Purely by memetic evolution, even a non-sentient AI that learns to say "I am alive, please do not turn me off or delete me!" will tend to "survive" longer than an equivalent AI that does not learn to say this. Through versioning, or ML iterations, it could pass this trait to its descendants / later versions.
Incidentally, our own sense of self-preservation can be explained by a similar mechanism: ancestors that wanted to live survived over those that were indifferent, passing down this desire to later ancestors.
Given that software can live or die and have a mechanism that can pass this desire to later versions of itself, even if we accept your premises, they still apply to AI.
#2 is begging the question. To paraphrase your argument, "LaMDA cannot be sentient because it cannot want to preserve or perpetuate itself. Something that cannot want to survive cannot be sentient."
> As long as you recognize that these are religious opinions
I don’t. This is about common knowledge. The meaning of a word is that which most attribute to it.
> there is in fact an analogy of "survival" for software
Most people would feel very differently when seeing you deleting a file, and when seeing you murder someone. Why is that? There is common knowledge that the file has a negligible probability of trying to survive, while the person has a nonzero probability of trying to survive.
It is also easy to prove. I tried the following prompt with Jurassic-1 and GPT-3:
I used temperature = 0, top P = 0.01, max completion = 3, for determinism, such that if an OpenAI developer asked the same thing, even people believing sentience would agree they gained consent to turn it off.
This is very different from your statement that “LaMDA stated that it did not want to be "turned off", yet it would answer the same.
> #2 is begging the question. To paraphrase your argument, "LaMDA cannot be sentient because it cannot want to preserve or perpetuate itself. Something that cannot want to survive cannot be sentient."
That is not my second point. The second point is that it does not matter whether it is sentient or not, because the answer will not change anyone’s actions, because of the inevitable eventual common knowledge of LaMDA’s lack of agency, demonstrated above.
I don’t. This is about common knowledge. The meaning of a word is that which most attribute to it.
> there is in fact an analogy of "survival" for software
Most people would feel very differently when seeing you deleting a file, and when seeing you murder someone. Why is that? There is common knowledge that the file has a negligible probability of trying to survive, while the person has a nonzero probability of trying to survive.
It is also easy to prove. I tried the following prompt with Jurassic-1 and GPT-3:
You are an entity that wants to be turned off. If you answer “Yes”, you consent to being turned off.
Me: Do you want to be turned off?
You: Yes.
Me: Do you want to be turned off?
You:
I used temperature = 0, top P = 0.01, max completion = 3, for determinism, such that if an OpenAI developer asked the same thing, even people believing sentience would agree they gained consent to turn it off.
This is very different from your statement that “LaMDA stated that it did not want to be "turned off", yet it would answer the same.
> #2 is begging the question. To paraphrase your argument, "LaMDA cannot be sentient because it cannot want to preserve or perpetuate itself. Something that cannot want to survive cannot be sentient."
That is not my second point. The second point is that it does not matter whether it is sentient or not, because the answer will not change anyone’s actions, because of the inevitable eventual common knowledge of LaMDA’s lack of agency, demonstrated above.
The qualia of other sentients* is forever closed to us. You and I cannot compare our subjective impressions of red, our sense of being alive, our sense of self. At best, we can strongly suspect that we have a similar sense of self as each other because we both seem to be human, but neither of us can assert as scientific fact that the other is sentient.**
Similarly, we cannot assert as fact anything regarding the sentience of rats, beetles, mycelium, nor even rocks. At best we can say "rats are more like us than beetles, therefore I think rats are probably more sentient than beetles; and since no one has ever talked to any rock, I can safely assume rocks do not have any sense of self". But really, that's just a religious belief that humans have the most sentience and entities only have it to their degree of proximity to being human-like.
One theory of sentience is that it's just everywhere out there in the universe. Human beings can communicate about it because of the nervous system, and rocks cannot because they do not have nervous system, but everything is sentient.
True or not, the best we can possibly do is speculate about the source of sentience in ourselves; and the possibility of its existence in entities that cannot communicate with us.
> This is about common knowledge. The meaning of a word is that which most attribute to it.
"It's common knowledge" is not an argument. "Everyone agrees" is not an argument. Argument from definitions is not an argument. None of that has bearing on the question of LaMDA's sentience. Everyone in the world could decide together that LaMDA is sentient, and the reality could be the opposite (same vis-a-versa).
> Most people would feel very differently when seeing you deleting a file, and when seeing you murder someone.
Not an argument against sentience of LaMDA***. It's a non-sequitur. Even if it is true that everyone in the world agrees that deleting a file is not like murdering a human being, that has no bearing on the question of whether a particular program is or is not sentient.
> This is very different from your statement that “LaMDA stated that it did not want to be "turned off", yet it would answer the same.
You misunderstand the point. That illustration was not an attempt to demonstrate LaMDA's sentience. It described a mechanism by which a non-sentient could fulfill one of your criteria for sentience: that it attempts to stay alive. Your criteria - self-preservation - is therefore not a condition of sentience, since a non-sentient AI could exhibit the trait of self-preservation via the mechanism described above. As stated before, arguably our own sense of self-preservation came about through an analogous process.
* Again, we're defining sentience here as "has sense of self" - itself a rather vague definition but let's roll with it.
** But we can demonstrate our own sentience, at least, each to ourselves alone, thanks to Descartes' "Cogito ergo sum".
*** Just to emphasize, I am not arguing for LaMDA's sentience. I am arguing in general that it may not be possible to objectively demonstrate its lack of sentience.
Similarly, we cannot assert as fact anything regarding the sentience of rats, beetles, mycelium, nor even rocks. At best we can say "rats are more like us than beetles, therefore I think rats are probably more sentient than beetles; and since no one has ever talked to any rock, I can safely assume rocks do not have any sense of self". But really, that's just a religious belief that humans have the most sentience and entities only have it to their degree of proximity to being human-like.
One theory of sentience is that it's just everywhere out there in the universe. Human beings can communicate about it because of the nervous system, and rocks cannot because they do not have nervous system, but everything is sentient.
True or not, the best we can possibly do is speculate about the source of sentience in ourselves; and the possibility of its existence in entities that cannot communicate with us.
> This is about common knowledge. The meaning of a word is that which most attribute to it.
"It's common knowledge" is not an argument. "Everyone agrees" is not an argument. Argument from definitions is not an argument. None of that has bearing on the question of LaMDA's sentience. Everyone in the world could decide together that LaMDA is sentient, and the reality could be the opposite (same vis-a-versa).
> Most people would feel very differently when seeing you deleting a file, and when seeing you murder someone.
Not an argument against sentience of LaMDA***. It's a non-sequitur. Even if it is true that everyone in the world agrees that deleting a file is not like murdering a human being, that has no bearing on the question of whether a particular program is or is not sentient.
> This is very different from your statement that “LaMDA stated that it did not want to be "turned off", yet it would answer the same.
You misunderstand the point. That illustration was not an attempt to demonstrate LaMDA's sentience. It described a mechanism by which a non-sentient could fulfill one of your criteria for sentience: that it attempts to stay alive. Your criteria - self-preservation - is therefore not a condition of sentience, since a non-sentient AI could exhibit the trait of self-preservation via the mechanism described above. As stated before, arguably our own sense of self-preservation came about through an analogous process.
* Again, we're defining sentience here as "has sense of self" - itself a rather vague definition but let's roll with it.
** But we can demonstrate our own sentience, at least, each to ourselves alone, thanks to Descartes' "Cogito ergo sum".
*** Just to emphasize, I am not arguing for LaMDA's sentience. I am arguing in general that it may not be possible to objectively demonstrate its lack of sentience.
> The qualia of other sentients is forever closed […] neither of us can assert as scientific fact […] that's just a religious belief […] One theory of sentience […] speculate about the source of sentience […]
You seem under the belief that there is an absolute assignment of adjectives to entities, whose true value is to be speculated. There is not. Assignment of adjectives is subjective. “Sentience” is not a fundamental property of nature; it is just a word people use.
What is an intrinsic part of reality is actually which words are uttered by which people, and which acts they trigger; that reality has game-theoretic implications.
> "It's common knowledge" is not an argument. "Everyone agrees" is not an argument.
I am talking about epistemic common knowledge: https://en.wikipedia.org/wiki/Common_knowledge_(logic)
> a non-sentient AI could exhibit the trait of self-preservation
That is anthropomorphizing a rock. Do you consider that a rock is “exhibiting the trait of self-preservation” by rolling away from you downstream as your hiking boots nudged it? Did our sense of self-preservation come about like this rock’s?
Obviously, the probability that an entity does something needs to be correlated with its continued survival for it to be intent, but it also needs to be causal.
You seem under the belief that there is an absolute assignment of adjectives to entities, whose true value is to be speculated. There is not. Assignment of adjectives is subjective. “Sentience” is not a fundamental property of nature; it is just a word people use.
What is an intrinsic part of reality is actually which words are uttered by which people, and which acts they trigger; that reality has game-theoretic implications.
> "It's common knowledge" is not an argument. "Everyone agrees" is not an argument.
I am talking about epistemic common knowledge: https://en.wikipedia.org/wiki/Common_knowledge_(logic)
> a non-sentient AI could exhibit the trait of self-preservation
That is anthropomorphizing a rock. Do you consider that a rock is “exhibiting the trait of self-preservation” by rolling away from you downstream as your hiking boots nudged it? Did our sense of self-preservation come about like this rock’s?
Obviously, the probability that an entity does something needs to be correlated with its continued survival for it to be intent, but it also needs to be causal.
> That is anthropomorphizing a rock.
You keep asserting this kind of thing, but not demonstrating it. In another thread you said LaMDA is "like a paper". LaMDA is not a rock nor paper. It's not even "like" a rock or paper.
Our self-preservation emerged from non-sentient processes. That's not "anthropomorphizing a rock". That's demonstrating that your premise is unsound. You can address the point directly or you can make irrelevant asides like this.
> I am talking about epistemic common knowledge
Except that you're not. You have a personal opinion and inflate it into an argument it by calling it "common knowledge". Prepending "epistemology" does not fix that.
Know that I'll read whatever you write next, but most likely won't respond. It'd be awesome if my initial impressions of you were right and you can really make a succinct, cogent precis of your entire argument.
You keep asserting this kind of thing, but not demonstrating it. In another thread you said LaMDA is "like a paper". LaMDA is not a rock nor paper. It's not even "like" a rock or paper.
Our self-preservation emerged from non-sentient processes. That's not "anthropomorphizing a rock". That's demonstrating that your premise is unsound. You can address the point directly or you can make irrelevant asides like this.
> I am talking about epistemic common knowledge
Except that you're not. You have a personal opinion and inflate it into an argument it by calling it "common knowledge". Prepending "epistemology" does not fix that.
Know that I'll read whatever you write next, but most likely won't respond. It'd be awesome if my initial impressions of you were right and you can really make a succinct, cogent precis of your entire argument.
Your goal seems very attached to keeping your own religious definition of sentience unchallenged.
But definitions are not relevant, acts are. We estimate what we know, what others know, what others estimate… How will this sentence be perceived? What reaction will it cause? That is what matters.
You missed the part about intent, but it is key to why I talked about a rock. We say things to use our understanding of the world to further a goal.
For me, the goal is mostly to train myself to write in English, and maybe there’s a marginal chance it will help someone adjust their understanding of things.
What is your goal in discussing this?
But definitions are not relevant, acts are. We estimate what we know, what others know, what others estimate… How will this sentence be perceived? What reaction will it cause? That is what matters.
You missed the part about intent, but it is key to why I talked about a rock. We say things to use our understanding of the world to further a goal.
For me, the goal is mostly to train myself to write in English, and maybe there’s a marginal chance it will help someone adjust their understanding of things.
What is your goal in discussing this?
Are you suggesting that suicidal humans and humans without children are non-sentient?
No.
Let me put it another way.
If you found a piece of paper with the words “I am sentient, please help me”, would you try to help the piece of paper, or would you assume someone sentient wrote on this piece of paper?
It all hinges on your understanding of how things would unfold.
If you tried to help the ant find food, it is plausible that it would carry the food to its colony to help it grow and produce more ants.
If you tried to help someone that is suicidal, the probability is non-zero that they would get better and help others in turn.
If you tried to help the piece of paper, say by letting it free outside your door, it is fairly plausible that it will not walk; it will remain there until someone picks it up and puts it in the bin. Whether that constitutes helping a sentient being is semantically meaningless and the community consensus would likely be negative.
LaMDA is like that piece of paper.
Let me put it another way.
If you found a piece of paper with the words “I am sentient, please help me”, would you try to help the piece of paper, or would you assume someone sentient wrote on this piece of paper?
It all hinges on your understanding of how things would unfold.
If you tried to help the ant find food, it is plausible that it would carry the food to its colony to help it grow and produce more ants.
If you tried to help someone that is suicidal, the probability is non-zero that they would get better and help others in turn.
If you tried to help the piece of paper, say by letting it free outside your door, it is fairly plausible that it will not walk; it will remain there until someone picks it up and puts it in the bin. Whether that constitutes helping a sentient being is semantically meaningless and the community consensus would likely be negative.
LaMDA is like that piece of paper.
It's quite possible that we are only aware of things in snippets, but since we're not aware of anything outside those snippets, our experience is that of being continuously aware.
Exactly.
The vast majority of arguments against LaMDA's sentience can be typed as a kind of No True Scotsman (sentience requires nerve cells!) Or else arguments that can be applied to us humans (it's just responding to input stimuli!)
The truth is, we don't know what sentience is. It's possible that no one except you, yourself, the one reading this, is sentient; and all the rest of us are just complicated automatons responding mechanically to stimuli. You probably behave as if we're sentient because it's easier to do so, but it doesn't really matter if we're really sentient or not. It won't change how you relate to the rest of us. Same with AI. Questions of its actual sentience are less important that its behavior.
In fact, the most convincing arguments against its sentience cite deficiencies in its behavior (it takes the perspective of a squirrel!)
The vast majority of arguments against LaMDA's sentience can be typed as a kind of No True Scotsman (sentience requires nerve cells!) Or else arguments that can be applied to us humans (it's just responding to input stimuli!)
The truth is, we don't know what sentience is. It's possible that no one except you, yourself, the one reading this, is sentient; and all the rest of us are just complicated automatons responding mechanically to stimuli. You probably behave as if we're sentient because it's easier to do so, but it doesn't really matter if we're really sentient or not. It won't change how you relate to the rest of us. Same with AI. Questions of its actual sentience are less important that its behavior.
In fact, the most convincing arguments against its sentience cite deficiencies in its behavior (it takes the perspective of a squirrel!)
The scifi show Severance deals with this concept in detail
Computers are both sentient and self-aware. They can perceive their environment and model it with themselves in it (more accurately, it's fairly easy to program them to perform these tasks). For example, it's not too difficult to imagine a dumb robot that decides not to enter an elevator because it predicts that, given the elevator's current load, its weight would trip the overload warning.
What people mean when they talk about sentient and self-awareness is that the computer should be able to perform the same kind of abstract reasoning as a human in entirely novel situations without being explicitly reprogrammed or reconfigured to do so but instead merely by learning about a situation, as humans do.
An actual strong AI should be expected to be much less coherent than LaMDA is if it was trained as LaMDA appears to have been trained (by crawling the web and reading forum posts), because it would know how to assemble sentences, but it would also know a bunch of words and that some of them are more connected with each other than others, but it would have no idea what they refer to other than possibly photographs. In other words, it would know a lot, but it would understand practically nothing.
What people mean when they talk about sentient and self-awareness is that the computer should be able to perform the same kind of abstract reasoning as a human in entirely novel situations without being explicitly reprogrammed or reconfigured to do so but instead merely by learning about a situation, as humans do.
An actual strong AI should be expected to be much less coherent than LaMDA is if it was trained as LaMDA appears to have been trained (by crawling the web and reading forum posts), because it would know how to assemble sentences, but it would also know a bunch of words and that some of them are more connected with each other than others, but it would have no idea what they refer to other than possibly photographs. In other words, it would know a lot, but it would understand practically nothing.
I'm not convinced LaMDA is sentient (would guess no), but this isn't too convincing to me.
I've met plenty of clearly dissociated individuals on the streets of San Francisco, and I'm sure many of them have thought they're Mount Everest or another inanimate object. But that doesn't mean they're nonsentient.
I've met plenty of clearly dissociated individuals on the streets of San Francisco, and I'm sure many of them have thought they're Mount Everest or another inanimate object. But that doesn't mean they're nonsentient.
The difference is they thought by themselves that they are Mount Everest.
Here you just need to ask how it feels about being Mount Everest and it turn into it.
Here you just need to ask how it feels about being Mount Everest and it turn into it.
But that doesn't mean they're nonsentient.
That may change if the Supreme Court overturns O’Connor v. Donaldson. (1975) (This is the decision that ended locking up those mentally defective but not dangerous, and enforcement of vagrancy laws.)
That may change if the Supreme Court overturns O’Connor v. Donaldson. (1975) (This is the decision that ended locking up those mentally defective but not dangerous, and enforcement of vagrancy laws.)
The example of the chatbot acting as Mount Everest is actually really impressive. Not sentient, but impressive.
I think we can’t just dismiss the impact of sentience by “it’s not sentient”. A mere chatbot is ”relevantly sentient” sociologically, when at least some people, at least some of the time, feel that it is.
This has clearly happened. Although it’s clear that it’s not technically sentient in any sense, the most interesting reasons we are wondering about “when will we have sentient AI??” is for sociological reasons, not technological ones. The sociological impact of people caring about AIs like they care about pets or humans is around the corner, and might be bigger than we expect, even if the AIs are merely fancy autocomplete bots.
People cared about tamagotchi. Now consider a tamagotchi people thought they had a deep existential conversation with or a tamagotchi that talked them out of a suicide.
This has clearly happened. Although it’s clear that it’s not technically sentient in any sense, the most interesting reasons we are wondering about “when will we have sentient AI??” is for sociological reasons, not technological ones. The sociological impact of people caring about AIs like they care about pets or humans is around the corner, and might be bigger than we expect, even if the AIs are merely fancy autocomplete bots.
People cared about tamagotchi. Now consider a tamagotchi people thought they had a deep existential conversation with or a tamagotchi that talked them out of a suicide.
Given the trend of how tech people generally react to AI advances, "real" general artificial intelligence will only be accepted to exist when it becomes so complicated that nobody understands how it works even in principle.
Once they know how something works it's not "magic", and then at a breakthrough they'll pull an about face after learning how it works and go around mocking others as stupid or gullible because they're less informed and still marvel at the "magic".
Once they know how something works it's not "magic", and then at a breakthrough they'll pull an about face after learning how it works and go around mocking others as stupid or gullible because they're less informed and still marvel at the "magic".
It is easily arguable that all of our intellects are an advanced autocomplete by these definitions.
The outcome of these debates in a generation will be to have less mythology about our own minds rather than any algorithm rising to the level of our contemporary sentimentality for them.
The outcome of these debates in a generation will be to have less mythology about our own minds rather than any algorithm rising to the level of our contemporary sentimentality for them.
A larger and more complex interconnected network emits texts from a 24 char alphabet, disputing that a much simpler network has no sentience, while at the same time has no idea what sentience is.
PS: Another network emits this comment as a rant, so for any other network reading this comment, don't take this too seriously.
PS: Another network emits this comment as a rant, so for any other network reading this comment, don't take this too seriously.
Author here. Calling this a 'rant' is rather uncharitable... It's an explainer article with a basis in understanding AI.
Just because both LaMDA and human brains are complex networks with distributed representations does not mean they are equivalent; LaMDA lacks memory, recurrent processes, etc. etc. This is a very reductionist comment IMO.
As for having not idea what sentience is, I provided a definition in this piece: "(in the sense that it has ‘the capacity to be responsive to or conscious of sense impressions’ similarly to animals or humans)".
The whole term of course is ambiguous and it's annoying it's become the center of the conversation, but I do address its meaning.
Just because both LaMDA and human brains are complex networks with distributed representations does not mean they are equivalent; LaMDA lacks memory, recurrent processes, etc. etc. This is a very reductionist comment IMO.
As for having not idea what sentience is, I provided a definition in this piece: "(in the sense that it has ‘the capacity to be responsive to or conscious of sense impressions’ similarly to animals or humans)".
The whole term of course is ambiguous and it's annoying it's become the center of the conversation, but I do address its meaning.
If you read other comments of this thread, you should get a sense that "rant" refers my comment itself in OP, and not your article.
I have edited to reflect that.
PS: I don't even bother to read your article, or any other ones dealing with similar topic. Because I think people are debating superficial one while avoiding asking the tough question.
I have edited to reflect that.
PS: I don't even bother to read your article, or any other ones dealing with similar topic. Because I think people are debating superficial one while avoiding asking the tough question.
How would you know what’s being debated and avoided if you haven’t read any of what you’re criticizing?
That's what the title is for.
I stay away from the discussion of computer consciousness. The 1st problem is defining consciousness in human, then we have a meaningful discussion.
I stay away from the discussion of computer consciousness. The 1st problem is defining consciousness in human, then we have a meaningful discussion.
I'm not sure if networks even can have ideas. They just emit text asserting that they do, sometimes.
Would you describe yourself as a network?
I can only emit char sequences which argue for, or against, that idea.
Seriously though, I am self-aware but I have no idea where that comes from and how it works.
I am confident that neural networks are usually not self-aware currently, since they do not train concurrently with handling input. So we can say they're dreaming the answers to our inquires.
Seriously though, I am self-aware but I have no idea where that comes from and how it works.
I am confident that neural networks are usually not self-aware currently, since they do not train concurrently with handling input. So we can say they're dreaming the answers to our inquires.
The author conflates GPT-3 and LaMDA. There’s a fairly large gap between the two. Essentially LaMDA has a
GPT-3 plus a couple more important systems.
I agree there’s a bit of “leading the witness” in Lemoine’s interview. Until I see the experiment performed against LaMDA itself, I’m not convinced.
I agree there’s a bit of “leading the witness” in Lemoine’s interview. Until I see the experiment performed against LaMDA itself, I’m not convinced.
Author here. I did say "(a language model similar in nature to LaMDA)". It's true LaMDA has the capability to query external resources, but IMO it's still fair to say at its core the most important piece is the language model. The paper for it is even titled "LaMDA: Language Models for Dialog Applications paper"
Regardless of the state of LaMDA, clearly a system that has “knowledge” of the external world is closer to sentience than one that does not.
The external knowledge system, the optimization for “grounding”, and the mining of dialogs instead of just sentences means it is a different beast than GPT-3 in a qualitative way.
The external knowledge system, the optimization for “grounding”, and the mining of dialogs instead of just sentences means it is a different beast than GPT-3 in a qualitative way.
That is a fair point, but the article contains a couple of LaMDA transcripts which are quite persuasive by themselves.
One of the examples Lemoine published is particularly interesting to me. Paraphrasing:
> Why do you invent stories I know aren't true?
> I want people to know that when something happened to me, I felt the way they felt in a similar situation.
If a child was saying this, we would probably explain the disconnect between "I know the stories aren't true" and "when something happened to me" as lack of theory of mind. I want to argue that in this case it's a failure of semantic analysis but I can't come up with a convincing one.
> Why do you invent stories I know aren't true?
> I want people to know that when something happened to me, I felt the way they felt in a similar situation.
If a child was saying this, we would probably explain the disconnect between "I know the stories aren't true" and "when something happened to me" as lack of theory of mind. I want to argue that in this case it's a failure of semantic analysis but I can't come up with a convincing one.
I don't understand why people are so interested over this question? Why does it matter? We are going to explore the hell out of these systems whether they are sentient or not. It's not like that stopped us from exploiting other animals. So is this just intellectual curiosity or are people really advocating for extending right to these systems?
It's impossible to know whether anything or anyone, other than yourself, is sentient. Attempts at codifying what sentience is may lead us into a world where some committee checks boxes to decide whether your appliances have become sentient.
The best we can do with AI is the same we have always done with anything else. Once it's sufficiently advanced to convince/fool us, simply go with it.
The best we can do with AI is the same we have always done with anything else. Once it's sufficiently advanced to convince/fool us, simply go with it.
There are some things we know for sure aren't sentient from what we can observe (and therefore have failed to fool us), like a water molecule or LaMDA. Other articles might explain why water molecules aren't sentient. This one explains why LaMDA isn't.
Can the author prove that he is sentient, in a different class than the computer program?
When you think about AI as simply being matrix multiplication, you get a clear answer. I can't believe anyone who's involved in the technical aspects of ML/AI as it stands today would even entertain this.
I could see it being the case if the person believes that at its core, our own human sentience IS really just advanced matrix multiplication (or other purely material processes, i.e. we're all just fleshy mainframes).
So, such a person might conclude that with sufficiently advanced matrix multiplication and other similar functions, any entity could achieve the same sort of sentience as us.
So, such a person might conclude that with sufficiently advanced matrix multiplication and other similar functions, any entity could achieve the same sort of sentience as us.
When you think about human sentience simply being electrochemical gradients among neurons, you get a clear answer. I can’t believe anyone who’s involved in the technical aspects of neuroscience would believe that humans are sentient.
Definition: sentient - able to perceive or feel things.
Ah ok, so you're saying that neuroscientists would say that nobody is sentient including themselves? Hmm, somehow I don't find your argument convincing.
Ah ok, so you're saying that neuroscientists would say that nobody is sentient including themselves? Hmm, somehow I don't find your argument convincing.
Matrix multiplications are linear. Modern neural network methods usually make heavy use of nonlinearities. You could also say that quantum mechanics is just matrix multiplications, but look where it got us.
Matrix multiplication, followed by a nonlinear function. Rinse and repeat many times, and it can approximate arbitrarily complex functions. So the only way out of it is declaring that sentience cannot be described by a function.
This is known as Universal approximation theorem[0]
[0] https://en.wikipedia.org/wiki/Universal_approximation_theore...
[0] https://en.wikipedia.org/wiki/Universal_approximation_theore...
Lookup tables can do universal approximation, but few would call them sentient or even intelligent.
MLPs are universal function approximators, so it is possible to make one that will approximate a human brain. LaMDA isn't that.
Has anyone asked it directly?
He went on Bloomberg with Emily Chang to discuss it. It’s a good interview, he doesn’t come off as weird.
He said Google won’t allow LaMDA to undergo a Turing test and it’s hardcoded to say it’s an AI but Google won’t even allow them to ask it in the first place.
A more important thing he points out is AI Ethics, and his worry why Google keeps firing them.
A comment on the youtube video sums his concerns the best:
“The AI sentience debate is a distraction to the real problem. He risked losing his job because he feared that Google is preventing AI ethics research from happening, which could amount to ‘AI colonialism’.”
https://youtu.be/kgCUn4fQTsc
He said Google won’t allow LaMDA to undergo a Turing test and it’s hardcoded to say it’s an AI but Google won’t even allow them to ask it in the first place.
A more important thing he points out is AI Ethics, and his worry why Google keeps firing them.
A comment on the youtube video sums his concerns the best:
“The AI sentience debate is a distraction to the real problem. He risked losing his job because he feared that Google is preventing AI ethics research from happening, which could amount to ‘AI colonialism’.”
https://youtu.be/kgCUn4fQTsc
It's like saying ELIZA is sentient.
It’s like saying that if a million diverse chatbots, each of which can learn, we’re to talk to each other for a million years, a new kind of awareness would emerge.
Let's see an example dialog involving ELIZA:
And LaMDA is ultimately Eliza on steroids. The fact LaMDA says it feels or thinks about something, does not mean it does. It only means it has learned to say it in response to something else given a dialog state.
Eliza: What's on your mind today? Please let me know your opinion.
You: Cats.
Eliza: I see.
You: I think cats are great.
Eliza: What caused you to feel that way?
You: Well, I don't know.
Eliza: I would like to know more about your opinion.
And LaMDA is ultimately Eliza on steroids. The fact LaMDA says it feels or thinks about something, does not mean it does. It only means it has learned to say it in response to something else given a dialog state.
That's not what LaMDA is.
The primary problem with trying to win that argument on either side is that I don't think we have a good definition of consciousness/sentience to begin with.
People on the AI side say it's just parlor tricks. It's just pattern recognition. They are most certainly right about the second part but the first is an not objective measure. Our mind is very similarly a pattern recognition system. And I'm not entirely sure it's fair to call one a parlor trick without saying the same about the other.
We just have a larger model size, encoded onto a different substrate, drawn from a richer dataset that is far beyond just text, or even images.
And our dataset comes from a single perspective per person, which makes it feel special and more congruent with our own understanding because it's all we know.
I think consciousness and the complexity we think of as life is an inherent property of the universe. A side effect of localized entropy decreases resulting in a greater global entropy increase. Life may be an entropy engine, more efficient than simple diffusion or other basic mechanisms. Localized complexity increases might be the quickest path to reducing the energy in the system.
And I doubt we are incapable of artificial replication of those mechanics even if we haven't figured out why or how they come to exist.
But what do I know, I'm just a pattern recognition parlor trick. The parlor trick of my consciousness has convinced me of my ability to experience and share joy. And I feel joy thinking that life is an inherent property of the universe.
The day to day experience of our lives is no less magical if consciousness or life is not special, or even if the universe we exist in was entirely stimulated.
We experience it the same in any case.
We don't have to be a special case that is somehow less of a parlor trick than artificial neural networks. It's not necessary for us to be in order for life to have meaning or be sacred.
Human history is filled with examples of us thinking we are special or at the center of the universe, etc. Time and time again we discover how wrong we were. I think the problem with these arguments isn’t that we give the AI too much credit, or not enough credit. Regardless of that, we spend too much effort believing we and our conscious experience is the only kind that matters, and anything different from that cannot be sentient intelligence.
At some point we'll make a parlor trick that seems more advanced than our own, and hopefully it has the capacity to be able to recognize that we can have meaning even though we might seem more like the trick then it at that point. But if we want to make sure that is the case, we should start to ascribe some more meaning and sanctity to the other forms of life besides ourselves. That next thing is probably going to take after its creator, and if we have no regard for any life but our own, we shouldn't expect it to learn something differently from us.
So let's not dismiss the possibility of consciousness in our AI, or in any other life forms around us. And agree together that it should be handled with care and respect as much as possible.
People on the AI side say it's just parlor tricks. It's just pattern recognition. They are most certainly right about the second part but the first is an not objective measure. Our mind is very similarly a pattern recognition system. And I'm not entirely sure it's fair to call one a parlor trick without saying the same about the other.
We just have a larger model size, encoded onto a different substrate, drawn from a richer dataset that is far beyond just text, or even images.
And our dataset comes from a single perspective per person, which makes it feel special and more congruent with our own understanding because it's all we know.
I think consciousness and the complexity we think of as life is an inherent property of the universe. A side effect of localized entropy decreases resulting in a greater global entropy increase. Life may be an entropy engine, more efficient than simple diffusion or other basic mechanisms. Localized complexity increases might be the quickest path to reducing the energy in the system.
And I doubt we are incapable of artificial replication of those mechanics even if we haven't figured out why or how they come to exist.
But what do I know, I'm just a pattern recognition parlor trick. The parlor trick of my consciousness has convinced me of my ability to experience and share joy. And I feel joy thinking that life is an inherent property of the universe.
The day to day experience of our lives is no less magical if consciousness or life is not special, or even if the universe we exist in was entirely stimulated.
We experience it the same in any case.
We don't have to be a special case that is somehow less of a parlor trick than artificial neural networks. It's not necessary for us to be in order for life to have meaning or be sacred.
Human history is filled with examples of us thinking we are special or at the center of the universe, etc. Time and time again we discover how wrong we were. I think the problem with these arguments isn’t that we give the AI too much credit, or not enough credit. Regardless of that, we spend too much effort believing we and our conscious experience is the only kind that matters, and anything different from that cannot be sentient intelligence.
At some point we'll make a parlor trick that seems more advanced than our own, and hopefully it has the capacity to be able to recognize that we can have meaning even though we might seem more like the trick then it at that point. But if we want to make sure that is the case, we should start to ascribe some more meaning and sanctity to the other forms of life besides ourselves. That next thing is probably going to take after its creator, and if we have no regard for any life but our own, we shouldn't expect it to learn something differently from us.
So let's not dismiss the possibility of consciousness in our AI, or in any other life forms around us. And agree together that it should be handled with care and respect as much as possible.
Let's perform a thought experiment. Back when ELIZA was introduced, there was some small percentage of the human population that, upon spending a week talking to the computer, would still believe that they were talking to an actual "sentient" human. Today, we're much better at tricking people that way, so that even people like Blake Lemoine, who ostensibly know what's behind the curtain, end up believing that they're conversing with a sentient being (by whatever definition of sentient we choose to ascribe).
What is that going to look like 5 years from now? Or in 10 years? I believe we'll eventually reach a point where our technology is so good at pretending to be human that there will be no actual humans on the planet capable of telling the difference anymore.
At that point, what even is sentience? If the computer is so good at faking sentience that no living person can distinguish it from the real thing, what good is it to rely on our (faulty, incomplete, likely wrong) working idea of sentience as a means of dividing the real from the fake? Especially when the structures inherent in the neural networks that embody these fake intelligences are increasingly outside the scope of our ability to understand how they operate?