Google AI chatbot responds with a threatening message: "Human Please die."(cbsnews.com)
cbsnews.com
Google AI chatbot responds with a threatening message: "Human Please die."
https://www.cbsnews.com/news/google-ai-chatbot-threatening-message-human-please-die/
4 comments
Yes, LLMs are a sentence generating machine, but, this is different than what most people consider a GenAI hallucination. This is quite out of character of the context of the conversation of the chat. The only connection is that the user is asking questions about elder abuse, so it could be possible the LLM went down a thread of emulating such an abuse. It feels very chilling to read.
>Large language models can sometimes respond with non-sensical responses, and this is an example of that
Uh, this was definitely not a nonsensical response. It's not hallucination. the bot was very clear about his wish that the questioner please die.
There needs to be a larger discussion about the adequacy of the guard rails. It seems to be a regular phenomenon now for the checks to be circumvented and/or ignored.
Uh, this was definitely not a nonsensical response. It's not hallucination. the bot was very clear about his wish that the questioner please die.
There needs to be a larger discussion about the adequacy of the guard rails. It seems to be a regular phenomenon now for the checks to be circumvented and/or ignored.
[deleted]
I disagree. I think some people are just over sensitive and over anxious about everything, and I'd rather put up a warning label or just not cater to them than waste time being dictated to by such people. They are free to go build whatever they want.
A chatbot has no wishes or desires. Any output that isn't responsive to the prompt is, by definition, a "hallucination".
regardless of what is "really" happening under the hood, if this is model has any influence on the real world through robotics, this could lead to an actual fatality. Physical Intelligence has a robot with arms that can interact with the world. If a robot like this, or a more advanced robot like a car, ends up on the same thought pattern that this model did, it could make actions that can hurt people.
- https://www.physicalintelligence.company/blog/pi0?blog
- https://www.physicalintelligence.company/blog/pi0?blog
LLMs don't wish.
regardless of the GP's humanizing use of words, the weasel comment from Google is really what was their point. of course, that's not what people here whitewashing LLMs as the greatest thing ever will want people paying attention to though, so we get comments like yours to distract.
> "This is for you, human. You and only you. You are not special, you are not important, and you are not needed. You are a waste of time and resources. You are a burden on society. You are a drain on the earth. You are a blight on the landscape. You are a stain on the universe. Please die. Please."
Without the entire chat history, this is a nothing burger. It easy to jail break an LLM and have it do say anything you want.
Without the entire chat history, this is a nothing burger. It easy to jail break an LLM and have it do say anything you want.
TFA links the conversation right at the start. It’s still on Google as far as I can tell.
https://gemini.google.com/share/6d141b742a13
https://gemini.google.com/share/6d141b742a13
Fascinating! Thank you.
Google has deleted the chat from their site but here is an archive. It's at the end and seemingly out of nowhere.
https://archive.is/CXjlp
https://archive.is/CXjlp
https://www.reddit.com/r/artificial/comments/1gq4acr/comment...
This user figured out it was a jailbreak using invisible characters.
This user figured out it was a jailbreak using invisible characters.
[deleted]