I think the most important question here is whether Huggingface was "in on it" – whether they knew in advance that this was going to happen and agreed to it. Second most important is whether OAI caused this to happen on purpose, with HF as an unwilling target.
Three options:
(1) Both companies are lying through their teeth and the entire thing is bullshit
(2) OpenAI hacked another company's servers and deliberately gambled on felony charges for the sake of PR
(3) OpenAI is telling the truth and a rogue AI agent hacked another company, opening them up to legal liability accidentally
All three of these are a lot more interesting than the "skeptical" story that this is just OAI doing business-as-usual PR hype, nothing to see here, move along.
Current LLMs can absolutely multiply floats without a tool call. In fact, that's a much more rote symbol-manipulation task than doing original math research.
Unless I'm mistaken, this uses "standard deviation" to refer to standard error throughout. They differ by a factor of sqrt(num_samples).
This is actually much more commonly useful than the t distribution, in my experience. You can squint at a histogram (or some summary stats), eyeball the stdev, approximate the stderr in your head, and get a pretty good sense of confidence.
I most often find myself doing this for the Bernoulli distribution, where it's also handy to know that the stdev is sqrt(p(1-p)), or "about 1/2 if p is middling, or sqrt(p) when it's small" (and you can flip the polarity to handle p→1).
The map contains a bunch of references to America, the West Indies, Guiana, and Mexico. (Often with a connotation of "faraway exotic place" or "exciting new international development".)
He may not have written about the British colonies but the New World was clearly at least somewhat present in his mind and his audience's minds.
> These reactors can be made safer, but they all still have a foundational design flaw which means the ultimate goal should be replacing rather than continually spending money reinforcing.
This was about the Fukushima reactors that were completely destroyed? In response to a discussion of Belgian reactors that are completely different?
> the underlying working principles are the same as GPT-2
I don't think anyone was claiming otherwise. Sonnet is still better at writing code than GPT-2, and worse than Opus. Workflows that work with Opus won't always work with Sonnet, just as you can't use GPT-2 in place of Sonnet to do code autocomplete.
Deep neural networks can generalize well even when they're far into the overparametrized regime where classical statistical learning theory predicts overfitting. This is usually called "double descent" and there are many papers on it.
Versions numbers for LLMs don't mean anything consistent. They don't even publicly announce at this point which models are built from new base models and which aren't. I'm pretty sure Claude 3.5 was a new set of base models since Claude 3.
What do mean by "it's a 1.0" and "3rd iteration"? I'm having trouble parsing those in context.
Not really. Dirac's trick works entirely at a depth of two logs, using sqrt like unary to increment the number. It requires O(n) symbols to represent the number n, i.e. O(2^n) symbols to represent n bits of precision. This thing has arbitrary nesting depth of logs (or exps), and can represent a number to n bits of precision in O(n) symbols.
I think it's better phrased as "find the best rule", with a tacit understanding that people mostly agree on what makes a rule decent vs. terrible (maybe not on what makes one great) and a tacit promise that the sequence presented has at least one decent rule and does not have multiple.
A rule being "good" is largely about simplicity, which is also essentially the trick that deep learning uses to escape no-free-lunch theorems.
There are two kinds of naturalness principle in physics, sometimes called "technical naturalness" and "Dirac naturalness" respectively.
Dirac naturalness is as you describe: skepticism towards extremely large numbers, end of story. It has the flaw you (and every other person who's ever heard it) point out.
Technical (or t'Hooft) naturalness is different, and specific to quantum field theory.
To cut a long story short, the "effective", observable parameters of the Standard Model, such as the mass of the electron, are really the sums of enormous numbers of contributions from different processes happening in quantum superposition. (Keywords: Feynman diagram, renormalization, effective field theory.) The underlying, "bare" parameters each end up affecting many different observables. You can think of this as a big machine with N knobs and N dials, but where each dial is sensitive to each knob in a complicated way.
Technical naturalness states: the sum of the contributions to e.g. the Higgs boson mass should not be many orders of magnitude smaller than each individual contribution, without good reason.
The Higgs mass that we observe is not technically natural. As far as we can tell, thousands of different effects due to unrelated processes are all cancelling out to dozens of decimal places, for no reason anyone can discern. There's a dial at 0.000000.., and turning any knob by a tiny bit would put it at 3 or -2 or something.
There are still critiques to be made here. Maybe the "underlying" parameters aren't really the only fundamental ones, and somehow the effective ones are also fundamental? Maybe there's some reason things cancel out, which we just haven't done the right math to discover? Maybe there's new physics beyond the SM (as we know there eventually has to be)?
But overall it's a situation that, imo, demands an explanation beyond "eh, sometimes numbers are big". If you want to say that physical calculations "explain" anything -- if, for example, you think electromagnetism and thermodynamics can "explain" the approximate light output of a 100W bulb -- then you should care about this.
Three options:
(1) Both companies are lying through their teeth and the entire thing is bullshit
(2) OpenAI hacked another company's servers and deliberately gambled on felony charges for the sake of PR
(3) OpenAI is telling the truth and a rogue AI agent hacked another company, opening them up to legal liability accidentally
All three of these are a lot more interesting than the "skeptical" story that this is just OAI doing business-as-usual PR hype, nothing to see here, move along.
(FWIW, I strongly doubt HF was in on it.)