Your looking at this from a legal perspective. Consider a diplomatic one instead.
Model training is expensive. Model distillation is cheaper. If you train models, and that lets your adversary distill them for much cheaper, that gives your adversary an advantage. Giving your adversary an advantage is bad, so you address it.
It doesn’t matter whether the advantage was gained legally, not even if it was fair. The advantage in and of itself is bad, and sufficient reason to act. You couch it in less nakedly power hungry terms, but this is the reasoning.
It’s more like eating your own fiber supplement in public, and then shitting your pants and telling everyone about it. Sure, it’s embarrassing. But it shows how potent your product is.
I especially wonder about the quality of N big models in a delegation harness against M smaller models in a delegation harness with N and M tuned to use the same amount of total compute. (So a few agents of big models against a lot of big models). I wouldn't be surprised if the advantages of delegation and the corresponding compression of context outweighs the lower quality of the smaller models.
Or put differently a wide exploration of long chains of obvious insights might be more valuable than a more narrow exploration of shorter chains of deeper insights. But perhaps that is a wrong sense of the difference between a small model and a large model.
It states regarding a visa:
"A visa allows a foreign citizen to travel to a U.S. port-of-entry (generally an airport) and request permission to enter the United States. A visa does not guarantee entry into the United States.
The Department of Homeland Security (DHS), U.S. Customs and Border Protection (CBP) officials at the port-of-entry have authority to permit or deny admission to the United States. "
Under the assumption that this framework is somewhat sane and only shares biometric data of those people who actually request an ESTA, I don't see the issue.
This data already leaks whenever you travel legally to the US, because they take it from you as a requirement of entry. All this does is let them check the data they take on entry against the provided data. Which seems fair. The fact this data is already in passports is a very different matter, but that is something the EU has done voluntarily.
That's a valuable extra distinction between Mathematics and Software.
In Mathematics there is much more clarity on what question you want to answer. It's much less likely you get an answer to a question, and then realize that the question was useless. Whereas in software its almost guaranteed that your first implementation, correct or otherwise, will solve the wrong problem.
It seems to me you hooked onto the wrong part of proofs vs software compared to what OP meant.
The difference OP cares about isn’t how much one cares about style. Instead the important difference lies in validation. A proof can be validated as either correct or wrong. That type of hard feedback really helps combat the optimism and desire for shortcuts of modern models.
Now, that still doesn’t help an LLM distinguish between good and bad correct proofs. But it still really helps a lot. On top of that, taste in proofs is a lot more uniform than taste in coding. That helps LLMs be better at judging the quality of a proof, because there’s less disagreement in the wider world.
A significant risk with ZKPs is that they can make a lot of explicit control rules palatable by making the side effects much less extreme.
If a company can come up with a reasonable reason to check something, then a ZKP enables that check without wider harm to privacy. But it still gives the narrow harm to privacy that otherwise wouldn’t exist.
Put differently, privacy concerns have shielded us not just from surveillance, but also from powerful control. Control in the form of “you are only allowed X if you meet criteria Y”. With ZKPs, that shield of “why would I tell you enough information to determine Y” stops working. But those conditions on X are harmful more broadly.
Take the US border checks of Social Media. Are those more OK if all of a sudden there’s a ZKP you have to give of never having called Trump a cunt in any private message?
The attestor gives you a credential once. You can think of it as a dob (in e.g. days since 1900) with a digital signature.
Then when you want to access something age gated, you locally generate a proof that says “I have a credential signed by X, with DoB N, and N < CutOff date” where X and the CutOff date are public but the credential and the value N are hidden through the ZkP.
The attestor isn’t involved so doesn’t learn anything. The Verifier only learns the public information, and generally won’t be able to tell if two proofs are made with the same credential.
We definitely need identity attestation in some places. A basic example is when authorising tax return payments. Or when opening a line of credit. Also when getting e.g. a concealed carry license. And similar for government agents executing search warrants, tow truck drivers picking up cars, etc.
Identity attestation isn’t a bad thing per se. It’s just something that can be abused.
Our whole education system is setup as a skinner box for grades. People finding tricks for better grades are massively rewarded. Study methods are optimised for recall within a month. And educators are accountable only to the measured outcomes.
Very little of the designed system of education is aimed at teaching. Most of it is aimed at measuring and certifying. Treating students as adversarial grade maximisers will likely teach a lot of them they are expected to grade maximise adversarially.
This isn’t about keeping people from having the power of frontier LLMs. So tricks that let others have it aren’t a defeat of this policy.
This is export control, where the US government seeks to leverage the fact that these frontier models are US made. This is then leveraged against opponents, and likely also just for grift. There’s also perhaps a little legitimate worry about the implications of free access to, but that is secondary to the real goal.
There's a very strong podcast series about the common failure of Epidurals: "The retrievals" season 2, by serial productions.
I found it rather eye-opening. The series includes reporting on a conference of the Society for Obstetric Anesthesia and Perinatology (SOAP) where the professionals in this field also found it eye opening. (mostly episode 2)
I don't bring this up to say that actually Epidurals suck, just to bring attention to the fact that they can fail, and that the system has historically handled such failure really poorly, and that the system itself isn't very well aware of this issue. This isn't just opinion from some podcast, but also admitted by the professionals working within this field.
It's also something valuable to be aware of when you or your partner is planning to have an epidural, because there is real space (and even a need) for advocacy for the patient when an epidural fails and the woman giving birth is in excruciating pain.
Not vacuous, but tautological.
Which is different, because tautologies can actually be quite directly informative. Whereas vacuous truths tend to be oblique.
Also, “Microsoft is lying” is not a logically stronger statement, because they might be lying about something other than whether they distilled or trained on AI output.
It doesn’t fully guarantee that. But it guarantees you don’t have a huge class of bugs. And it makes concurrency a lot easier to reason about.
No system will likely ever guarantee that software does what you expect. That runs into the halting problem, and practically runs into a verbosity problem. But that doesn’t mean systems that give scoped guarantees aren’t amazing for building (and iterating on) reliable software.
Currently working for TNO in the department Cyber Security and Robustness.