You literally can do a kind of model PCA, using the Hessian (matrix of second derivatives of the loss function w/r/t the parameters, aka the local curvature of the loss landscape), and diagonalizing. These eigenvectors and eigenvalues (the spectrum of the Hessian) tend to be power-law distributed in just about every deep NN you can think of [1].
That is, there are a few "really important" (highly curved) dimensions in parameter space (the top eigenvectors) which control the model's performance (the loss function). Conversely, there are very many "unimportant"/low curvature dimensions in the model. There was a recent interesting paper that showed that "deleting" these low-curvature dimensions appeared to correspond to removing "memorized" information in LLMs, such that their reasoning performance was left unchanged while their ability to answer questions which require some memorized knowledge was reduced [2].
It appears that sometimes models undergo dramatic transitions from memorization to perfect generalization, which corresponds to the models becoming much more compressible [3].
I'm hopeful that we'll find a way to distill the models down to the most useful core cognitive/reasoning capabilities, and that that core will be far simpler than the current scale of LLMs. But they might need to look stuff up like we do without all that memorized world knowledge!
I don't recall Andrej making "next year!" claims, it was always Elon. I found Andrej's talks from that time to be circumspect and precise in describing their ideas and approach, and not engaging in timeline speculation.
They really should have just marketed the software "as-is" to whatever extent that is allowed by law. I guess they didn't because deployed automobile software is probably not allowed to be considered experimental.
Still, comms that framed it like: "This software purchase upgrades your car with state-of-the-art autonomy capabilities from our AI team, as we approach full self-driving" would have been more honest, still exciting to consumers, and avoided over-promising.
I'm a researcher at Oxford, and I've both taught and studied here and in the US.
The undergraduate teaching here is phenomenal. It's incredibly labor intensive for the staff, but the depth and breadth students are exposed to in their subject is astonishing. It's difficult to imagine how it can be improved.
My favorite study of university rankings comes from faculty hiring markets, which compute implicit rankings by measuring which institutions tend to hire (PhD->faculty) from others. [1] It's not perfect, but at the very least it's a parameter free way to get a sense of how different universities view each other. The parameters in most university rankings are rather arbitrary and game-able.
Some have pointed to things like contextual admissions [2], and more broadly some identity politics capture of the administration for declining standards. While this might be true, in my view Oxford is still far more meritocratic than US institutions on the whole. There are no legacy admissions, and many subjects have difficult tests which better distinguish between applicants who have all done extremely well on national standardised tests (British A Levels are far more difficult than the SAT/ACT/AP exams.)
Lastly, admissions at Oxford are devolved to the individual colleges, of which there are ~40. The faculty at each college directly interview and select the applicants which they will take as students. This devolved system and the friction it creates is surprisingly robust and makes complete ideological capture more difficult.
The most pressing issue for Oxford's long-term viability as a leading institution, in my view, is the funding situation. For one the British economy is in a long, slow decline. Secondly, even though Oxford has money, there are lots of regulations/soft power influence from the British govt to standardise pay across the country, which makes top institutions like Oxford less competitive on the international market for PhD students, postdocs, and faculty in terms of pay.
I sat the All Souls exam, taking the philosophy specialist papers, though I'm a math/physics/ML guy. It was a lot of fun, I really appreciate that there's somewhere in the world where these kinds of questions are asked in a formal setting. My questions/answers are written up in brief here [1]
Very cool! I've done research on reinforcement/imitation learning in world models. A great intro to these ideas is here: https://worldmodels.github.io/
I'm most excited for when these methods will make a meaningful difference in robotics. RL is still not quite there for long-horizon, sparse reward tasks in non-zero-sum environments, even with a perfect simulator; e.g. an assistant which books travel for you. Pay attention to when virtual agents start to really work well as a leading signal for this. Virtual agents are strictly easier than physical ones.
Compounding on that, mismatches between the simulated dynamics and real dynamics make the problem harder (sim2real problem). Although with domain randomization and online corrections (control loop, search) this is less of an issue these days.
Multi-scale effects are also tricky: the characteristic temporal length scale for many actions in robotics can be quite different from the temporal scale of the task (e.g. manipulating ingredients to cook a meal). Locomotion was solved first because it's periodic imo.
Check out PufferAI if you're scale-pilled for RL: just do RL bigger, better, get the basics right. Check out Physical Intelligence for the same in robotics, with a more imitation/offline RL feel.
The entropy of some data is well-defined with respect to a model, but the model choice is free. I.e. different models will assign different entropy to the same data.
And how do we choose a model...? Well, formally by minimizing the information needed to describe both the model and data (the sum of model complexity and data entropy under the model) [1]
You might argue that's all too information-theoretic and in physics there simply is an objective count of the state-space, a maximum entropy, and so on. Alas, there is not even general consensus on whether there is a locally finite number of degrees of freedom.
Maybe a correction is needed. Academia has become so gamified. It's supposed to be about ideas, truth, beauty. Too many are in it for the prestige, which has ironically made it less prestigious.
Hmm. At least it's still fine to define limits of the complexity for infinite strings. That should be unique, e.g.:
lim n->\infty K(X|n)/n
Possible solutions that come tom mind:
1) UTMs are actually too powerful, and we should use a finitary abstraction to have a more sensible measure of complexity for finite strings.
2) We might need to define a kind of "relativity of complexity". This is my preferred approach and something I've thought about to some degree. That is, that we want a way of describing the complexity of something relative to our computational resources.
Interesting. I guess then we would only be interested in the normalized complexity of infinite strings, e.g. lim n-> \infty K(X|n)/n where X is an infinite set of numbers (e.g. the decimal expansion of some real number), and K(X|n) is the complexity of the first n of them. This quantity should still be unique w/o reference to the choice of UTM, no?
This could also be a hardware startup. If only there were some entrepreneur types around...
Presumably there's a market for this in other niches, e.g. weather monitoring, defense/border monitoring, etc... The question is whether the juice is worth the squeeze. Where's the really valuable data?
The issue with the invariance theorem you point out always bugged me.
Let s be an algorithmically random string relative to UTM A. Is it the case that there exists some pathological UTM S, such that K(s|S) (the Kolmogorov complexity of s relative to S) is arbitrarily small? I.e. the blank print statement of S produces s. And there always exists such an S for any s?
Is there some way of defining a meta-complexity measure, the complexity of some UTM without a reference UTM? It seems intuitive that although some pathological UTM might exist that can "compress" whichever string you have, its construction appears very unnatural. Is there some way of formalizing this "naturalness"?
I agree with you in spirit. I just thought you might be interested in some of the technical details regarding the relationship between compression and generalization!
I'll have a paper out next week which makes your point precise, using the language of algorithmic rate--distortion theory (lossy compression applied to algorithmic information theory + neural nets).
I think another way of understanding this is through the "Value Equivalence Principle", which points out that if we are learning a model of our environment, we don't want to model everything in full detail, we only want to model things which affect our value function, which determines how we will act. The value function, in this sense, implies a distortion function that we can define lossy compression relative to.
Great point. This points to the related issue: what do we want to compress? Do we want to compress "the answer", here the arithmetic expression's solution, or do we want to compress the image?
You can formalize this with rate--distortion theory, by defining a distortion function that says what your objective is. That implies a well-defined complexity relative to that distortion function.
Okay to be clear, I've written a paper on exactly this topic which will be announced in a week or so. So you won't find anything on the subject yet, haha. But I use almost exactly this example.