I don't understand the RMS table, shouldn't it be non commutative? Eg "example 0 vs 1"'s RMS != "example 1 vs 0"'s RMS? Which doesn't seem the case for the checkpoints I checked.
But what if your rubber duck is actually steering your thought process (since you may not have a consolidated one)? In this way I think the AI as editor is far better than a rubber duck AI. While in the former, it might point out your mistakes and give useful advices (which is similar to what you describe), it might not steer your thought (unless your mistakes are far too severe!), and actually help in your reasoning. But AI as a brainstorming rubber duck (or thinking tool) could be harmful to your thought process.
mine too, but none was such a dick. also, anything related to school (particularly at a young age), is not viewed as something to boast of (at least in my experience in italy, serbia and portugal).
> But the extent and the way in which Zig specifically puts it to use -- which includes, but is not limited to, how it is used to replace other features that can then be avoided (and all without macros) -- is unprecedented.
That MrWhite wanted to knkw an example of Zig's comptime that is not merely a "macro", rather the usage as a replacement of other features (I guess more complex..)
PS just interested in zig, I'd like some pointer to these cool feature :)
so they are basically using a similar idea to that of a stirling engine in thermoelectric generator or they use a different mechanism to produce energy?
I don't have much experience with ROCm for large trainings, but NVIDIA is still shit with driver+cuda version+other things. The only simplification is due to ubuntu and other distros that already do the heavy lift by installing all required components, without much configuration.
why don't you ask the model about the shrinked system prompt and the original system prompt? in this way you can infer whether the same relevant informations are "stored" in the hidden state of the model.
Or better yet, check directly the hidden state difference between a model feed with the original prompt and one with the shrinked prompt.
This should avoid remove the randomness of the results.
I dunno, these reasoning models seems kinda "dumb" because they try to bootstrap itself via reasoning, even though a simple direct answer might not exist (for example key information are missing for a proper answer).
Ask something like: "Ravioli: x = y: France, what could be x and y?" (it thought for 500s and the answers were "weird")
Or "Order from left to right these items ..." and give partial information on their relative position, eg Laptop is on the left of the cup and the cup is between the phone and the notebook. (Didn't have enough patience nor time to wait the thinking procedure for this)
I think the issue with RL is that, in order for a model to perform well in a task, you have to make it stubborn. In the same way a student that thinks outside the scope of the task might not perform well in a graded exam, but that does not mean he/she is a bad reasoner. With RL and all training procedure you are creating a very focused and very fit to the task thinker, which might not be useful in all applications (consider an open problem, it might need an out of the box kind of thought).
meta is large enough (and diversified enough), when OpenAI valuation goes to pennies (because LLM won't solve any real problem, nor they'll remain relevant in the long term and open source alternative are more than enough for most chatbot usecases), they can acquire all the relevant technologies from OpenAI, Anthropic and such. The real question is why google didn't release their models.