"So where's this huge price gap coming from? token pricing, prompt caching, and effort-per-task. On SWE for example, K3 works much harder than Fable: roughly 55 turns and 1.3M tokens a task versus 21 turns and 130K. On the long terminal tasks it's the other way around: Fable is the one that spirals, running up 64 turns and 1.5M tokens (sometimes straight into a timeout).
Prompt caching does most of the work of turning that effort into K3's price advantage: even when K3 reads ten times the tokens, with cache hits that means that SWE runs still come in lower cost than Fable. There’s a tradeoff. Tasks with extra turns generally mean more wall-clock time per run i.e. slower runs. If you need an answer in two seconds, that matters; if you're running agents in the background at scale, a bill that's a fraction of the size matters a lot more."
As other have mentioned, and I'm the same. Leaving "Co-Authored by:" feels like open disclosure of how the project was created.
That way if a consumer does not want to use LLM generated/assisted software, it's easy for them to do that without having to search around in the codebase for "AI Tells".
Removing those would feel like I was mis-representing my coding abilities (LLMs are better at most coding than I am) :)
Interesting write-up and I do think LLM assisted/powered exploit disclosure is a real concern (I've been able to get models to create container breakouts from Linux LPEs relatively quickly).
One thing I'm surprised about is that GPT-5.6 didn't block that prompt due to guardrails. My experience is that GPT-5.5 and up does not like offensive security work (similar to Opus 4.7+/Fable).
I didn't notice it but I'd assume that the authors have some level of cyber approvals from OpenAI to relax the guardrails a bit.
You might find some areas to criticize Berkshire Hathaway but I don't see being lazy as one of them. This is one of the most successful investment companies of all time and they got that way by being better than most at judging when the right time to get in and get out of the market, and by putting in the work on researching where/when to buy.
Might there be opportunities they miss? I'm sure there will be, but perhaps finding those is just too risky at the moment, so they've looked at the options and decided not to invest.
I'm not sure I'd call them rando, but Deepseek/Qwen/GLM all have decent models that will work fine for some use cases at much lower costs that the SOTA models from Anthropic/OpenAI.
It's an interesting thing and we can only speculate from the outside, but there's some obvious reasons why they'd literally hand out money in the form of free compute to people who have already committed to paying them.
- They've had to commit to minimum spend with their suppliers and their actual usage is below that level, so they might as well give it to end-users. That implies either their demand is waning or their forecasts were off.
- They want the usage numbers for this period to be higher and are willing to spend money to achieve it. I guess if they're about to IPO maybe showing more usage is good?
- They've got a lot of competition and want to keep market share. With lots of new models coming out, some of them much cheaper than Anthropic's options, you can see why this might make sense but it's effectively burning cash, so they won't want to keep doing that for long I'd guess...
I think until they produce full financial information, which will happen I expect when their S-1 is published, we won't have a good picture on Anthropics true position.
There's a lot of different ways to calculate "profitable" depending on what's included or excluded...
Interesting write-up. Having been a bookkeeper a long time ago, I'm not too surprised at this being susceptible to automation by an LLM backed system.
It seems also that the classes of error they encountered could be handled by improved skills/knowledge base access on the fine points of relevant tax legislation.
The important part for their software ofc is, will they take responsibility for the output if HMRC come calling? Without that users are adopting the risk which they may not be keen to do (dealing with HMRC is not fun), with that it could be a very nice saving for a lot of small companies (and bad for the employees of a lot of accountancy firms)
If you're going to have "blessed" plugins, which seems like a good idea, you'll need a review and possibly hosting process.
- Review to check that the plugin is reasonable quality/isn't malicious.
- hosting (e.g. the plugin is retrieved from a repo. you control) or "known good" checksums so pi will only download the plugin with a version that you've reviewed.
From a security/supply chain aspect, ironically what you're looking to do is deliberately add some friction to the publishing process, which sounds bad, but can be quite effective at mitigating attacks. Most of the recent supply chain attacks get found by automated scanners in < 24 hours, so having a review process for new releases that takes a while will reduce the number that affect users.
I think having this is handy as it'll give security conscious users more confidence in using pi, without the anxiety of pulling a load of additional code from effectively random sources.
It provides a right to privacy if the company allows personal messaging services on the device, but I don't think it provides a right to having personal messaging apps on the device(s).
Personally I think it's much cleaner to keep work stuff on your work device(s) and personal stuff on personal devices. The only place that gets sticky is where companies don't provide the device but want you to have work information on it (e.g. mobile phones)
I have a similar experience, for the last 5+ years I've worked in companies where very few of the people I work with are British which does require care on both language and idiom. Combined with being older than a lot of colleagues, cultural references need to be picked with care :D
The later Opus models (4.7/4.8), Sonnet 5, and particularly Fable 5 will refuse to do tasks related to offensive security.
One example I've hit is working on a benchmark of how well LLMs handle Kubernetes security tasks, there's a section on them exploiting security misconfigurations. Opus 4.6 was fine with that section, 4.7 and 4.8 saw some refusals and Fable point blank refused to do any of it.
The only other model I've seen refuse is OpenAI GPT-5.5, all the open weight models seem fine with it.
Ofc if you need to do that kind of work a lot you might be able to get on OpenAI/Anthropics allow-list for cyber work.
Yep I've got one I built and it's absolutely fine for my use cases has a web interface/API custom kernels and rootfs, even the facility to set-up custom Kubernetes clusters. It's been really useful for other work like testing out vulnerabilities or security features in isolated envs.
Worth noting that, this isn't just a risk with npm or other package managers. If you're using LLM agents in the directory of a cloned repo, there's risks in skills, hooks etc automatically executing..
that probably depends on how much security and resource isolation you need. Multi-Tenant security in Kubernetes is not a simple thing, for a wide variety of reasons, and noisy neighbour problems are also potentially a headache.
The one I remember most is, when experimenting with Opus 3.5 for the first time, I asked it to generate a Firecracker backed local VM creation and management tool, something I'd wanted for a while but not found.
My expectation was that it might get something barely functional but would probably fail, and instead it generated a working piece of software which achieved a lot of what I wanted.
That definitely made me realise that, for at least some classes of software task this was a major change in how things could be done.
More recently when I can give the model a Local Privilege Escalation PoC in Linux and ask it to test whether it can be used for container breakout and then generate a working container breakout, all in one prompt... that definitely changes things.
not really, there are a number of security companies doing analysis of any new packages looking for supply chain attacks, so if you wait a couple of days, till their analysis is complete, you're reducing the risk of hitting a compromised package.
Any opinions expressed are purely my own and not necessarily those of any employer, past, present, or future.
Personal Site - https://www.mccune.org.uk Blog - https://raesene.github.io Mastodon - @[email protected] Blue Sky - @mccune.org.uk