I’ve found that I’m better at math than I used to be when working with AI. It really helps me to get my head out of the weeds, to be curious in the right direction, to focus on how to drive the development of ideas towards a goal without getting hung up on technicalities. The LLM fills in the details, I check them, provide corrections, and realize what I’d missed: loop. Then I get on with the real plot-setting.
(Admittedly, I work in applications, where the formalism describes the trees better than the forest.)
From the abstract, I’m curious about two things. First, is `\phi` an arbitrary function? Second, why ‘Sophie Germain’ rather than just ‘Germain’?
Right, that’s why I wrote, “I don’t mean that most neural networks are invertible functions.”
For a neural network that is not bijective, you can obtain an input that maps to a desired output by the following algorithm.
1. Start with a trained neural network. (The weights will not change throughout this procedure.)
2. Pick a random input.
3. Given an output for which you want to compute an associated input, feed the input into the network to compute the output.
4. Compute the loss of the computed output relative to the target output (e.g., mean-square error). If the loss is sufficiently small, you’ve found an input that maps to an output close to your target output and you’re done.
5. Otherwise, compute the gradient of the loss with respect to the input (e.g., by backprop).
6. Update the input according to a gradient-update rule. And go back to Step 3.
In theory, you can recover a “representative” prompt for the output of an LLM in this manner. For outputs that could have been generated by a large set of disparate prompts, obviously this won’t work well.
The obvious solution is to run things in reverse, inputting the AI-generated output to recover the prompt that generated it.
Most generative models can be run in reverse by algorithms that already exist [0], but you have to have the model weights. For closed-weight models, or for a process that can handle unknown models, you’d have to do some engineering.
But do we have the technology to build models that back out the prompt from suspected AI output? Yes.
0. I don’t mean that most neural networks are invertible functions. They’re not. But you can do backprop in reverse, from output to input, to train a model to generate an input to the original model that best predicts its output.
The inflection point was 2012, when AlexNet [0], a deep convolutional neural net, achieved a step-change improvement in the ImageNet classification competition.
After seeing AlexNet’s results, all of the major ML imaging labs switched to deep CNNs, and other approaches almost completely disappeared from SOTA imaging competitions. Over the next few years, deep neural networks took over in other ML domains as well.
The conventional wisdom is that it was the combination of (1) exponentially more compute than in earlier eras with (2) exponentially larger, high-quality datasets (e.g., the curated and hand-labeled ImageNet set) that finally allowed deep neural networks to shine.
The development of “attention” was particularly valuable in learning complex relationships among somewhat freely ordered sequential data like text, but I think most ML people now think of neural-network architectures as being, essentially, choices of tradeoffs that facilitate learning in one context or another when data and compute are in short supply, but not as being fundamental to learning. The “bitter lesson” [1] is that more compute and more data eventually beats better models that don’t scale.
Consider this: humans have on the order of 10^11 neurons in their body, dogs have 10^9, and mice have 10^7. What jumps out at me about those numbers is that they’re all big. Even a mouse needs hundreds of millions of neurons to do what a mouse does.
Intelligence, even of a limited sort, seems to emerge only after crossing a high threshold of compute capacity. Probably this has to do with the need for a lot of parameters to deal with the intrinsic complexity of a complex learning environment. (Mice and men both exist in the same physical reality.)
On the other hand, we know many simple techniques with low parameter counts that work well (or are even proved to be optimal) on simple or stylized problems. “Learning” and “intelligence”, in the way we use the words, tends to imply a complex environment, and complexity by its nature requires a large number of parameters to model.
> Even for billion-parameter theories, a small amount of vectors might dominate the behaviour.
We kinda-sorta already know this is true. The lottery-ticket hypothesis [0] says that every large network contains a randomly initialized small network that performs as well as the overall network, and over the past eight years or so researchers have indeed managed to find small networks inside large networks of many different architectures that demonstrate this phenomenon.
Nobody talks much about the lottery-ticket hypothesis these days because it isn’t practically useful at the moment. (With the pruning algorithms and hardware we have, pruning is more costly than just training a big network.) But the basic idea does suggest that there may be hope for interpretability, at least in the odd application here or there.
That is, the (strong) lottery-ticket hypothesis suggests that the training process is a search through a large parameter space for a small network that already (by random initialization) exhibit the overall desired network behavior; updating parameters during the training process is mostly about turning off the irrelevant parts of the network.
For some applications, one would think that the small sub-network hiding in there somewhere might be small enough to be interpretable. I won’t be surprised if some day not too far into the future scientists investigating neural networks start to identify good interpretable models of phenomena of intermediate complexity (those phenomena that are too complex to be amenable to classic scientific techniques, but simple enough that neural networks trained to exhibit the phenomena yield unusually small active sub-networks).
Sandvault [0] (whose author is around here somewhere), is another approach that combines sandbox-exe with the grand daddy of system sandboxes, the Unix user system.
Basically, give an agent its own unprivileged user account (interacting with it via sudo, SSH, and shared directories), then add sandbox-exe on top for finer-grained control of access to system resources.
OK, let’s survey how everybody is sandboxing their AI coding agents in early 2026.
What I’ve seen suggests the most common answers are (a) “containers” and (b) “YOLO!” (maybe adding, “Please play nice, agent.”).
One approach that I’m about to try is Sandvault [0] (macOS only), which uses the good old Unix user system together with some added precautions. Basically, give an agent its own unprivileged user account and interact with it via sudo, SSH, and shared directories.
Milton Friedman wouldn’t have approved of a basic-income scheme restricted to artists. He would have argued that restricting the benefit to artists would distort incentives for choosing a profession in a way likely to reduce social welfare, and that eligibility by profession is a “welfare trap”: it’s hard to stop being an artist and start being something else when it means losing your guaranteed income.
But Friedman would have supported a broad basic-income scheme. We know this because he did support one. It was his proposal in 1962 of a “negative income tax” [0] (in Capitalism and Freedom) that gave rise to the movement to replace traditional social welfare programs with simple schemes that just give money to poor people. (This movement led to the Earned Income Tax Credit [1] in the United States.)
Friedman’s negative income tax is equivalent to the contemporary notion of a guaranteed basic income (but not to a universal basic income, as only people earning below some threshold would receive it). Like most economists, Friedman believed that people (even poor people) can typically make their own economic choices better than a government program can make those choices for them. (He was likewise not opposed to redistributive policies per se.) That was the root of his advocacy for market-based mechanisms of organizing the economy.
From what I understand, which is very incomplete, the leading hypothesis at the moment is that ingested prions are a bit hard to digest (because they’re malformed proteins), so they end up making it out of the gastrointestinal tract somehow, interacting with the nervous system via the intestinal lining or lymphatic system. Then they travel to the brain via nervous pathways, by-passing the usual blood–brain barrier.
But transmission of prions by ingestion is thought to be quite rare, as that mechanism suggests. Transmission by any means seems to be quite rare, even heritable transmission (e.g., vCJD). So that’s why it seems unlikely that whatever is happening in New Brunswick is CVD.
But if it’s not some minor mass hysteria, then maybe prions.
I’m not qualified to comment intelligently on what might be going on here, but I’d like to add some background color that the article lacks.
Creutzfeldt–Jakob Disease is a prion disease [0] for which there is no definitive diagnosis in vivo. A confident diagnosis can be made only after examining brain tissue under a microscope.
Prions are an unusual type of mis-folded protein that induce other proteins to take on a similar mis-folded shape when they come into contact with them. The mis-folded shape of the prion itself is what causes the mis-folding in adjacent proteins. It’s a chemical-bonding thing at the molecular level. It’s the shape of the prion that causes other proteins to take on a similar shape and become prions, etc.
Some prion diseases occur spontaneously (when a protein takes on a mis-folded configuration due to mis-transcription or random energetic impulses) and some are transmitted, typically by eating some part of an animal that contains prions, which then end up in your own body, inducing proteins in your body to take on prion configurations.
Prion diseases are the only known transmissible diseases that do not involve the replication of a pathogen’s genetic material in a host cell. The only known prion diseases affect nervous tissues, and in humans the only known prion diseases affect brain tissues.
I’m not an expert on prion diseases, but I’ve had a bit of a fascination with them since having to report on a bunch of USDA surveillance lectures on mad-cow disease (bovine spongiform encephalopathy, BSE) and to summarize a bunch of symposia on prion diseases in a previous life. The symptoms reported in the article sound very much like a prion disease, and the tests for CJD indicate that the doctors in the region suspect as much.
But we simply don’t have good tests for prion diseases in vivo. And prion diseases are not well understood in general, so it wouldn’t be surprising that a new one would present as something of a mystery.
It is also the case that I know very little about New Brunswick, but I will mention that prion diseases in humans are thought to be far more commonly acquired than spontaneous. The most common cause of acquisition is eating animals with endemic prion diseases; this is most often nervous tissue of venison, but rarely nervous tissue of cattle infected with BSE, which is present in Canada more than anywhere else (by a small margin).
It is also possible (but not likely) that a prion disease can arise de novo.
For some definition of “sufficiently introductory”, I’d recommend starting with the first chapter of John Nolan’s book Stable Distributions [0] (20 pages), which presents the class of distributions to which sums of iid random variables converge and builds up to a version of the generalized CLT.
Note that this generalization of the classical CLT relaxes the requirement of finite mean and variance but still requires that the summed random variables are iid. There are further generalizations to sums of dependent random variables. John D. Cook has a good blog post that gives a quick overview of these generalizations [1].
I’m not sure what the author had in mind when he wrote the paper, but its light-hearted content reveals well how to structure a math paper: (1) state the problem and why the reader should care about it, (2) state how the paper builds on prior work, (3) summarize the main results, and then (4) develop them in theorem-proof style.
An economics paper in the same spirit (though of course economics addresses more practical problems than mathematics, in this case whether one should leave the toilet seat up or down) shows quite well how to present and develop an economic model [0].
The general version of this is called inverse transform sampling [0], which uses the fact that for the cdf F of any random variable X the random variable Y = F(X) has a standard uniform distribution [1]. Since every cdf increases monotonically on the unit interval, every cdf is invertible [2]. So apply the inverse cdf to both sides of the previous equation and you get F^-1(Y) = X is distributed like X.
Sampling from a standard uniform distribution and then using the inverse transform is the commonest way of generating random numbers from an arbitrary distribution.
Yes, it’s a bit surprising. The whole point of a constitution is to provide a groundwork and a set of meta-rules governing how the ordinary rules of society may be made and modified, and how disputes about those sorts of thing should be resolved. The judiciary is precisely the mechanism for entertaining and deciding these disputes.
The odd thing, really, is that it took a decade and a half, until 1803, for the Supreme Court first to find it necessary to tell another of the other branches of government that it had broken the rules.
> The judicial Power shall extend to all Cases, in Law and Equity, arising under this Constitution …
Let’s not continue this conversation. The view I’m presenting is long established and can be found better and much more extensively made in a great many sources which no one should have any trouble finding, should he wish. You, I infer, do not wish.
The role of the courts in deciding the constitutionality of legislation was very much in the zeitgeist of the American experiment. Federalist, No. 78:
> It is not otherwise to be supposed, that the Constitution could intend to enable the representatives of the people to substitute their will to that of their constituents. It is far more rational to suppose, that the courts were designed to be an intermediate body between the people and the legislature, in order, among other things, to keep the latter within the limits assigned to their authority. [0]
It is absurd to say that the Supreme Court “magicked out of thin air” the notion of judicial review in Marbury v. Madison. That was simply the Court's first assertion of the power the constitution gave it to wield, the wielding of which is the primary purpose of the judiciary as a separate but equal branch of American government.