The 34b Python model is quite close to GPT4 on HumanEval pass@1. Small specialised models are catching up to GPT4 slowly. Why not train a 70b model though?
From a Nondualist perspective, our brain is highly complex biological neural network that has a special ability to reflect pure consciousness giving rise to the mind and the world with it. A sufficiently advanced artificial neural network can also reflect the same consciousness but thier minds and world would be entirely unlike ours. However, consciousness will add a random component to the predictions and might make them completely useless as a tool. They might decide not to follow the instructions prompt given their internal state of mind.
That is a problem even now though. Sometimes LLMs just goof up for no reason. Maybe they are conscious.
The quote "The limits of my language mean the limits of my world" is attributed to Ludwig Wittgenstein, an Austrian-British philosopher. He expressed this idea in his work "Tractatus Logico-Philosophicus."
It's one of Sw Sarvapriyananda's favourite quotes. Wittgenstein was quite Vedantic it seems.
What if something is different but nothing has changed in the model? Transformers are non deterministic. The response to same prompt may vary slightly, and can be controlled somewhat by the temperature setting. Something could have gone wrong there.
I've played around with aider trying to run tests and fix the code, but it just crashes after exceeding the context window. I am now trying to repurpose the AutoGPT example in langchain.
Cool project. I tried to build a reactjs Todo app with TDD, but it just put comments in the test file instead of the actual test. A self heal loop would be quite useful.
> With these updates, we’ll be inviting many more people from the waitlist to try GPT-4 over the coming weeks, with the intent to remove the waitlist entirely with this model. Thank you to everyone who has been patiently waiting, we are excited to see what you build with GPT-4!
What about the rate limits? The docs say that it's 200 RPM and "We are unable to accommodate requests for rate limit increases due to capacity constraints."
> OpenAI and others have proposed that licenses would be required only for the most powerful models, above a certain training compute threshold. Perhaps that is more feasible
Somebody is bound to figure out how to beat threshold sooner or later. And given the advances in GPU technology, this compute threshold will itself keep going down exponentially. This is a dumb idea.