> Probabilistic analysis can carry you very, very far in doing something that looks like logical inference at the surface level, but it is nonetheless not logical inference.
A statistical approximation of logical inference (as vague as I state it) could (and will) very well pass for logical inference, at least for the common people, whose logic skills are far from perfect.
Also, humans are certainly not capable of the perfect logical inference you speak of. And I get the irony of what I'm saying with such certitude. Logic is still framed in axioms that are framed in languages, we'll never truly get there. Ah, but absoluteness gets in the way of practicality.
Yet, here we are with a tool, that is maybe not at its prime yet, that equals and beat many human beings at logical inference on some problems that are pragmatically relevant. Should I say symptoms of logical inference at that point?
As to why LLMs capacity for (apparent) logical inference is only limited to specific use cases, I don't have a clue. But I'd like to argue that, humans are like that too.
I don't think that this is true. You need an infinite number of dimensions for this (think Taylor's expansion, Fourier expansion, infinitely wide or deep NNs..)
Bayesian network is a really general concept. It applies to all multidimensional probability distribution. It's a graph that encodes independence between variables. Ish.
I have not taken the time to review the paper, but if the claim stands, it means we might have another tool to our toolbox to better understand transformers.
Statistical learning, optimization, algorithms, computer sciences, and other nerdy stuff.