Anecdotal: I used to beg my boss to get some help. I wanted 5 engineers and would have been upset if anyone had been hired that wasn't an engineer until I at least got 1-2. That was up until about 3 months ago.
Now, I don't want any additional engineers. Not only because I don't need them anymore, but because the prospect of having junior engineers using AI is absolutely terrifying to me. I can't eyeball their code to get a sense of how good of an engineer they are anymore, and I can't possibly review all of their code because of how much code AI can output now. So they're going to be outputting a ton of mostly high-quality code that could be making horrible mistakes that are much harder for me to catch now.
In 65 years Ozempic will have been in use for 65+ years to help people lose weight, but that won’t make it untrue that going for a walk helps you lose weight.
> They want you to rent their compute instead of running it on your own hardware
I don't think this is the main thing. I think the real risk open weights AI poses to them is that it allows us to get inference from providers that only care about making profits on inference. What OpenAI/Anthropic want is to be able to jack up inference prices once everyone is dependent. Open weights keeps them honest.
What's interesting is that the way in which they're childish is actually extremely human. In fact, one of the ways that you're often taught to draw more realistically is to stop thinking of the concepts as icons you're drawing the outlines of, and instead sort of blur your eyes and see things as they are: hues and values. In other words, become a camera or a printer that has no idea what it's capturing or printing other than a grid of values. That is how you achieve realism.
The fact that it has clearly iconified these concepts in its mind and is tracing the outlines of the things it thinks/expects to go where is very human.
It's funny that they triggered the infamous "Code Red" moment in OpenAI when the 3-3.1 models came out. I switched to using them for a lot of single-shot LLM calls because they were fast and cheap. Their only area that they were lacking in was agentic/tool calling.
Needless to say 3.5 was a disappointment. Curious to see 3.6.
If you want an LLM to have knowledge about something, the knowledge has to either exist in its weights or be provided to it in its context. Because context is expensive and limited, and models tend to get dumber the more their context is filled, there is usually more that you'd need to put into context than can reasonably fit in it in order for the model to answer questions about your data. So your options are basically
1.) stuff it into context
2.) figure out a way to determine what to put into context based off of what is being asked of the model (RAG)
3.) change the weights of the model to have knowledge of your data baked into it (fine tuning)
This is like a single prompt AI. The term labels are illegible against the white background. The connections don't make any sense. "Training vs Inference" connects to Large Language Models and GPU. It doesn't connect to RLHF. Machine Learning connects to fewer things than Large Language Models. The whole thing is insane.
MoE is literally exactly what they're describing. The classifier being described is baked into the model and is the thing that makes it MoE.
Imagine I had [a model that was good at math], [model that was good at code], [model that was good at writing], [model that was good at general knowledge]. If I then had [a model that was good at determining whether the user query would be best served by one of those models and sent it to it, leaving the rest of the models inactive], that is the platonic version of what MoE is. In practice, it works a bit differently. It instead basically restricts the number of pathways that can be utilized in solving problems during training, which allows for "expert neuron groupings" to form and "classifier layers" to form earlier on in the structure, but the effect is the same (better, even, since it allows some overlap between structures of experts). It also allows "routing to an expert" to happen token-by-token rather than at the prompt level.
I didn't know people could just have GPT running on their own hardware. How does one...do that? Do you have a special relationship with OpenAI and they lock down your servers or something?
I have built a few voice based integrations into my applications that use these live agents (gpt and gemini), but they are always too expensive to be viable. I have to end up hacking up context and turning on and off in ways that are very fragile. It'll end up being $2-5 for about the 30ish minute sessions I typically end up with, and it throws the price of the product I'm making completely out of whack.