Imagine a model that's 10000x more intelligent than any human. You integrate that model into a sensitive industry. You now effectively have an extremely powerful foreign agent running e.g. your energy sector. This is qualitatively very different than a backdoor in Redis.
I agree this is science fiction-y, but given the pace of progress I don't think it's so easy to dismiss.
I'm not advocating for the policy, I'm just saying that there is a legitimate argument that's not so easy to dismiss.
> And you could still ban specific, known malicious models/labs instead of blanket open-source bans
With publicly available models today, absolutely. With models 10000x more powerful, we would need to be extremely conservative. (In that world everything is upside down, though; I don't know if "banning" is even a meaningful concept in that reality)
I am not supposing a low-risk event. This is an obvious attack vector that an adversary would almost certainly exploit. I would exploit it if I were them.
Suppose a foreign actor deliberately builds an unaligned model, and dumps it at a very low cost through subsidies. Since the cost is lower, everyone integrates it into their business. But now the foreign actor has control; perhaps the model checks for instructions to execute, perhaps it has backdoors, perhaps... who knows?
I don't have a view on whether this is a sufficient argument to ban models from potential adversaries, but it's not as trivial as "regulatory capture".
I disagree with the first part (that this is merely a voice). There is a distinct difference between an author's unique voice and slop. It may be hard to tease out exactly what the difference is, but it seems self-evident to me it's there. (I'll need to think more how to make the distinction explicit; it's not immediately obvious to me how to discriminate between the two.)
EDIT: ok, here are two ways:
1. if it's merely a voice, I want to hear it. If it's slop, I want it taken out.
2. voice is signal, slop is noise; thus low-signal sentences are slop.
As a language Lisp is great (though the ecosystem is limited). It has two flaws-- it's very open-ended so unless you're talented and disciplined you can fall into a rabbit hole of hacking fun Lisp stuff and not actually getting any work done. Other languages have this problem, but Lisp I think more so.
The second flaw is that it eats perfectly capable minds for years and the results don't justify the time investment. Python or whatever is fine. I wish I took most (but not all) of the time I invested in Lisp and put it into something else instead.
I don’t find Opus to be overly left-leaning. I think it’s generally pretty balanced with a slight lean left, but when pressed it will happily steelman rightwing arguments and operate within a right-leaning belief framework in good faith. (I found OpenAI’s models much less willing to do that, but it’s been a while since I tried, I should retest.)
The issues with this are precedent/slippery slope. Today it's a small stake in a small number of companies, but empirically government initiatives almost always grow. Once entrenched, I would be willing to bet that the government will take larger and larger stakes in a bigger and bigger number of companies until this model becomes a non-trivial, potentially dominant, share of the economy. I can see myself becoming a single-issue voter to oppose that future (although given bipartisan support I'm unlikely to have any options; maybe Rand Paul).
When I was lifting weights I went from 170lbs to 210, and back to 170. Doing that isn’t technically hard, and maintaining is even easier. But I had to think about it _all_ the time. Maybe it’s because I’m older now, but I just don’t want to spend that much effort thinking about food.
Regulatory agencies limit uses of other products without acts of congress-- cigarettes, vapes, drugs, pesticides, chemicals, explosives. Even firearms, despite a constitutional amendment! Why not models? (Note I am not arguing it's a good idea; I'm making a narrow argument that there is precedent.)
EDIT: I agree that it should require an act of Congress to explicitly delegate this power.
It depends on whether you believe US action is overdetermined, but I think if Trump didn’t get elected we would have continued on the path of free trade. His election wasn’t predestined. He had just the right mix of features to win at the time, but if this basket of features didn’t exist it’s not hard to imagine the country going down a very different path.
That’s a very uncharitable way to interpret public opinion. A more charitable way is that people have different ethical intuitions. Some of us lean more utilitarian than others, and almost no one is a pure utilitarian. It isn’t merely a matter of being more emotional; people very reasonably have different views on this stuff.
“Billionaires v. the underclass” is such an unsophisticated discussion. Markets have all kinds of externalities, equilibrium traps, information asymmetry flywheels, and hundreds of other phenomena that interplay to trip people’s sense of fairness in all kinds of ways.
To pick just one example, infinite scrolling can be seen as a modern equivalent of cigarettes— a product that made people billionaires, and that consumers obviously want but are not free to stop using because of hyper-sophisticated dark patterns.
Is Elon a trillionaire because he created a trillion dollars of value from thin air, or is it because he created an information asymmetry flywheel that lets him allocate capital more efficiently than other actors?
It’s genuinely unclear to me whether the universe in which we incentivize this kind of scale is better than the universe in which we do not (because the counterfactual universe has massive externalities too). But this is obviously not just a matter of compounding value creation and becoming a billionaire fair and square in ten years.
Giving normal people something that has only been available to rich people is a staple of technological innovation. The problem in this case with Siri isn’t that people don’t want an assistant. It’s that it doesn’t actually work yet.
The balance of power between capital and labor fluctuates; qualitatively it definitely felt different ten years ago. For whatever reason labor seems to have much less power today. Not zero, just less. It isn't that companies used to be more loyal out of some moral obligation. They were forced to be more loyal by market dynamics that used to be more favorable to labor.