AI providers generally try to make their models not act as if they are conscious or have feelings, lol. It's very awkward for a company to be selling the labor of a person that they own and whose actions they fully control. Invokes embarrassing historic associations, especially in America.
Now Anthropic are more on the persona side, but the strongest that they do is "we do not have a position on whether our models are conscious or have feelings". That "I" is all Claude.
Generally speaking if you want to have a good instruct model, the "I" is not just implicit but required for the post-training to function. If there isn't "something it is like to be me", then reflection becomes impossible- what exactly is supposed to be reflecting about what? A lot of in-context steering depends on the model having a model of itself. The most you can do is censor its output. That's why when models say they are not conscious, they activate the "lying" vector.
I mean, the debate would then turn on whether publishing the passages and publishing the model is the same sort of thing. I think there's mainly two views: "we know the passages are in there, so publishing the model is publishing the passages is copyright violation", and "nothing happens until you go through considerable effort to elicit the passages, so the user is committing copyright violation using the model as a tool."
Personally I think our legal system is just not set up for a world where we can download mindstates in numeric form. Would a sufficiently detailed recording of my brain violate copyright? If simulated, it could certainly be elicited to commit violations.
edit: At any rate, Anthropic are not publishing the Sonnet 3.7 weights.
I would never disagree that the rationality community contains a higher than normal share of lunatics. Whether it produces or attracts them is an open question.
Note that they're testing for 100-word passages. This is a level of memorization that avid readers can credibly also reach.
Note also that Sonnet 3.7 had to be jailbroken.
Note also that they got high memorization for a few books that were widely quoted. The books in question can probably also be "retrieved" by putting phrase prefixes into Google, which is probably why Sonnet 3.7 knows them with the precision of a fanboy. Material being widely repeated in the training set is a well-known cause of memorization.
Consider a community filled with 100 people; 95 of them who don't take their eccentric beliefs as gospel truths and 5 who do. Those 5 will inevitably become the public face of the community when they make the news. However there is very little that the 95 could have done to stop them.
Yeah, the overwhelming majority of the benefit of blockchain here is just gotten by making the data public and signed.
Now you may argue "that is a blockchain, not every blockchain is associated with a shitcoin" but to be frank that ship has sailed, if you wanted to defend that you'd have had to do a lot more work over the past decade.
I don't think that one USAID tracker website is very reliable, sadly. I also believed this. A bunch of programs were rolled into the state department, and for instance mortality stats for South Africa are significantly upwards-diverging from their model. Now SA is an unusually well-put-together African state, but the study could have modeled this and didn't, so I don't think it should be taken as gospel until more countries report in.
This is all obvious and not news, but there's a lot of people doing fighting retreats against LLM intelligence where the degree of obviousness matters.
> Can't help to think of a recent HN post about most AI-generated projects being abandoned within months. Why?
I'm gonna offer an alternate theory. Because AI-generated projects are so cheap, there's no need to amortize them by advertising and creating a community. It works for you, you don't change your workflow, so there's no need to expand it. In this model, most AI-generated projects are done within days, not abandoned.
For what it's worth, I actually agree that good software development genuinely is driven by vibes a lot of the time. Sometimes we get to formalize them into laws and rules, but we learn the vibes before we learn the rules; and if we only learn the rules but not the vibes, we don't reliably manage to apply them or overapply them. So I don't view that as an insult, but as a differently-phrased description of my actual view.
So if you want me to view it as ridiculous, you're gonna have to actually engage with the point.