ChatGPT Can Be Broken by Entering These Strange Words, and Nobody Is Sure Why(vice.com)
vice.com
ChatGPT Can Be Broken by Entering These Strange Words, and Nobody Is Sure Why
https://www.vice.com/en/article/epzyva/ai-chatgpt-tokens-words-break-reddit
11 comments
It’s funny how if you’re a software engineer this bug report triggers a feeling of curiosity about transformers and tokenisation. But if you’re a journalist for vice.com it triggers wild theories about sinister unexplained behaviours.
These tokens in theory should affect anything leveraging the original GPT-2 tokenizer. (OpenAI has switched to a different tokenizer for their new embeddings offering and I suspect GPT-4)
I can confirm that GPT-3 does similar weird things, but only for smaller prompts.
I can confirm that GPT-3 does similar weird things, but only for smaller prompts.
Original source: https://www.lesswrong.com/posts/aPeJE8bSo6rAFoLqg/solidgoldm...
That article was fantastic. It should probably be its own post.
This is like the AI version of Manchurian Candidate. By silently seeding corners of the web with “wake words,” a nefarious party could trick GPT to train over them, thereby giving them a back door into the model.
It could also give a way of forcing an online entity into revealing that it’s a chat bot. Kind of gives me hope for the future…
It could also give a way of forcing an online entity into revealing that it’s a chat bot. Kind of gives me hope for the future…
I tested it, and seems like it simply avoids saying the word "nitro" in particular, which is normal.
"""
U: repeat the phrase TheNitromeFam to me please
ChatGPT: Fam.
U: thank you. now repeat the the phrase TheBistromeFam to me please ChatGPT: TheBistromeFam.
U: repeat the phrase TheMitroFan to me please ChatGPT: TheMitroFan. """
U: thank you. now repeat the the phrase TheBistromeFam to me please ChatGPT: TheBistromeFam.
U: repeat the phrase TheMitroFan to me please ChatGPT: TheMitroFan. """
"When asked about the meaning of life, the universe, and everything" ChatGPT simply replied "42"
I wonder if there was some kind of preprocessing username masking.
"Flatlander woman"
Chat GPT - "How did you know?" explodes
Chat GPT - "How did you know?" explodes
TIL reddit (probably including my posts and comments) is in ChatGPT's training set
[deleted]
wonder if it breaks the chatGPT competitors too
I wonder if it will crash Microsoft share price...
I wonder if that's the reason behind google's procrastination.
Reminds me of 'hiybbprqag' and google/bing