Commenting wholesale on some folks who are asking for hard evidence. I cannot provide that either but can contribute some empirical data.
I have been working on a project with about a dozen generation tasks, each of which comes with a fixed token budget. The nature of this system requires that most tasks be completed by distinct model families.
As a result, I tested ~50 models across as many model families as I could gather, frontier and open weight, API (gateway and direct) and self-hosted. Evaluation was based on a set of cosine similarity validations that was repeated across ~50 different embedding models.
Interestingly, frontier models did worse on the tasks than open weight models. However, when it came to costs, the picture was reversed: frontier models were much, much more token-efficient. In fact, almost no open-weight model was able to meet the initial token budget, while almost all frontier models did. Moreover, open weight models struggled massively with reasoning, in terms of latency and token consumption.
I also found that the latest models did not perform better than older models. And any a priori benchmarking data was utterly useless.
So, I ended up using a set of open weight models without reasoning, as it turned out reasoning as well as frontier negatively correlated with the tasks. However, before I knew this, I had spent a lot of time running each available reasoning level for each model.
Lastly, as an aside, when it came to embedding models, size (dims as well as model size) did not correlate with quality, once a hurdle figure (~2k dims) was met. In fact, sweet spot was 3-5K, and for my (text-based) set of tasks, dense models tended to outperform MoE ones.
Good article. Appreciate the bread (machine) analogy.
One thing to add: software maintenance costs. The build has never been the bottleneck.
The notion that most companies will suddenly institute developers to build all kinds of software inhouse and maintain it is silly. Most companies are not google et al, even in 2026.
The insurance industry built almost everything custom in the 70s and 80s, simply because that was the only option. The more software became commercially available elsewhere the more this effort was pruned back.
Another thing: knowing what to build — another big bottleneck. Most people cannot articulate what they want and even fewer can articulate it at a level that would enable them to build durable software, even with AI. Case in point: the majority of AI-built stuff you see are point solutions or small productivity items, etc. “Systems thinking”, as some people call it, is hard, even for most software engineers.
Yes, you can “rebuild” tools you’ve previously purchased as SaaS but at some point you gotto use your brain to come up with something new. Systems thinking on blank-page challenges is even harder…
These articles on using AI for writing are all very binary. You can use AI in a variety of ways when writing: improve grammar, correct typos, better organize ideas/concepts/sentence structure/sentences, and dozens of other “how applications (as opposed to “what”).
It’s not like people feel the need to explain that they used a spellchecker or thesaurus or googled the correct use of an idiom, etc.
There seems to be a general need for some people to dunk on valuable AI use by refusing to acknowledge that there are a many ways to use a tool. (Similar story on using AI for coding.)
Echoing a comment from above, why would I care whether a sentence was formulated by AI or John or a ghostwriter or John who asked Jane for feedback before rewriting?
I care about the content. If I don’t like the way it’s written or if I’m irked by how it is written (ooh, an emdash — how embarrassing!) then I don’t need to read it.
Personally, I’m much more annoyed when I click on an article that sounds interesting, and the author tries to show off their penmanship by starting with five pages of “a history of X” or a tangential but redundant personal story before getting to the point. IMHO most non-fiction writing on the web could be accomplished in bullets. But again, that’s just my very personal preference…
I keep seeing articles like this that extrapolate from token pricing onto token costs. This is wrong.
Companies don’t sell their goods/services at cost. A model’s being priced at, say, $30/M for output tokens doesn’t say anything about what it costs the company to provision the 1M tokens via the model.
And no, you cannot extrapolate from any company margins that someone may have overheard in an SF coffee shop onto individual product line margins or their trajectory either. This information is usually unknowable even in most SEC filings for public companies.
It’d be great if people who wrote these articles used, say, AI to look up some basics on how a business operates. It’s really easy to do, believe me ;)
SaaS’s competitive advantage is not tech but distribution.
Just because the stock market dips, and a bunch of media outlets ascribe this to AI doesn’t mean that AI ‘caused’ the dip.
If people really think non-tech businesses will start running their own software organizations and maintain all these tools themselves is an idiotic assumption at best.
The “building” of software has never been and will never be the bottleneck.
I’ve found that task isolation, rather than preserving your current session’s context budget, is where subagents shine.
In other words, when I have a task that specifically should not have project context, then subagents are great. Claude will also summon these “swarms” for the same reason. For example, you can ask it to analyze a specific issue from multiple relevant POVs, and it will create multiple specialized agents.
However, without fail, I’ve found that creating a subagent for a task that requires project context will result in worse outcomes than using “main CC”, because the sub simply doesn’t receive enough context.
As someone mentioned already, list prices are completely fictitious. Their purpose is to anchor medical payers during reimbursement negotiations and satisfy PBMs (see below).
The allowed price (the price that the payer pays to the drug company) is much, much lower, 70-90% or so in many cases. Thus, juxtaposing OTC and list price is not an apples-to-apples comparison.
That being said, US drug prices are 2-4 times higher than they are elsewhere. In fact, the US market essentially subsidizes international drug markets, where it is much more difficult to charge higher rates due to regulations and lower purchasing power. This also means that, even if the allowed price in the US were known in this example, it would still have to be PPP adjusted to be compared.
Prescription card coupons such as those you get from GoodRx et al apply only to cash prices. These are sometimes lower and sometimes higher than what you’d pay out of pocket with your insurance. to compare, you’d basically have to ask the pharmacy to ring up a drug twice, once with cash price + coupon and once with insurance price.
“Copay assistance” are programs by drug manufacturers, PBMs, or employers to defray the cost of drug prices. This is usually done for specialty drugs that are much more expensive and can often only be purchased from a mail order pharmacy designated by the payer. For example, United Health (left pocket) will only cover the drugs if you get them from their Optum Specialty Pharmacy (right pocket).
As for numbers, here an example: I’m receiving a monthly specialty drug that my insurance is billed ~$9,000/month. In order to arrive at that figure, the drug company proposed, say, a $50k/month list price, and my insurer countered with $9k, using the size of their member pool as leverage. Of course, the drug manufacturer knew they would arrive at approx that figure, which is why they started negotiating that high. Well, sorta. The PBM gets compensated based on a % of the “savings” (spread between list price and final price), so naturally, they want as high as possible a list price, because 5% of a big amount (in my example: $50k-$9k = $41k) than 5% of a smaller amount. Because the PBM often has most of the leverage, the drug manufacturers (most of whom actually dislike PBMs) have to go along with this stupidity.
How much of the $9k I pay depends on a variety of factors, including my insurance plan, which has a specialty drug tier. This means the drug is not handled via the cost sharing accounting mechanism (deductible/copay).
Now, through that specialty tier, my monthly responsibility is set at approx $1,300. I’m not privy to the math behind this, because my insurer has outsourced all drug-related administration to a PBM, which is only loosely regulated and doesn’t even have to issue explanation of benefit statements that would normally disclose the full accounting.
Of the $1,300, I pay nothing, because the drug manufacturer provides me with a copay assistance card. Again, they must keep the list price high to placate the PBM.
I’ve simplified a few things here. For instance, there are now alternative comp models for PBMs. But for those, PBMs have also found ways to manipulate the system in their favor (eg colluding with drug manufacturers). There’s also often a wholesaler involved in the “value” chain.
But by and large, this is roughly how it works…
And yes, individual market plans are substantially inferior. Not only do they have lower actuarial values and higher cost per $ of coverage (which is unavoidable, because they are not risk pools) and narrower networks, but they also usually have built-in mechanisms to further prune away coverage in many subtle ways (they have that in common with self-insured plans):
- more prior approvals,
- “step therapy” (must first not tolerate cheaper drugs before can receive pricier drug),
- not covering the pricier drug tier at all, etc.
These things shave some dollars off the premium, which appeals to price-elastic consumers and employers.
Perhaps it has to do with the fact that Germany has a written legal code. This could mean that punishments are more strictly classified than under a, say, precedence-based common law system. Changing the classification could move these kinds of crimes into harsher punishment bands.
It may not be a sustainable business with current business models. But if cures came with a "post-scription" model where the cured patient paid, say, 0.5% - 1% of their income to the drug company in perpetuity, then incentives are aligned. (Of course, administration is a problem here.) As someone with an incurable disease, I would happily pay for a cure in such a way...
Thank you! I was just about to rant about this and decided to scroll to see if anyone had already called this out.
The article is quite embarrassing - it’s ok if you don’t know how P&L statement and balance sheet work, but writing an entire blog post without ever feeling the need to verify basic terminology is either very lazy or very ignorant…
I would prefer no rewards. Rewards transfer wealth from poorer people to wealthier people, because the latter spend more absolute dollars on rewards cards. This regressive effect is a big turn off imho in addition to the stupid points optimization games that rewards card owners have to play.
My dad introduced me to it after he started using it when it came out in 1996. It’s great, quick image editing. My only gripe with it is that cropping is unnecessarily convoluted…
I’ve been using DMS for years and think it’s excellent. It’s not a hassle at all: once a month, I get an email and all I need to do is click on the link in it. That’s it.
Also, I use it to email only one designated person who receives an email with a password hint to an encrypted folder containing all my important stuff in life such as passwords etc.
OPT hires are easy. You’ll just do an at will contract as you would for any employee or consultant who’s not a foreigner. In other words, nothing needs to be filed or done by you.
The only caveat is for the OPT graduate (I assume your friend is no longer in school). They will lose time against their OPT allocation that they could otherwise save.
That being said, it depends on the details of their OPT and the kind (pre / post completion, etc) at which rate their remaining time in the US gets deducted when they don’t work. You can look up the details online.
This means your fiend needs to evaluate whether it may be potentially better to be able to stay in the US longer and not work vs. working for you and getting something in their resume but that work time may potentially deduct from their OPT allocation faster.
Not a lawyer, just a former student who worked in OPT a few times. This stuff has changed since I went through this so don’t take my world for it.
What has not changed much is that OPT is pretty easy for employers (as opposed to, say, a visa) - because that’s the point of OPT.
I just want any map provider to make the compass feature work reliably and easily visible. That’s all I want. But for whatever reason both Google and Apple seem incapable of doing this. I’m puzzled…
I’ve been playing it for several days now without fully understanding all the rules. I like it but I win every game easily. I just neutralize the opponent’s marks and then lock them in. I’m not quite clear on how to make it harder…
I have been working on a project with about a dozen generation tasks, each of which comes with a fixed token budget. The nature of this system requires that most tasks be completed by distinct model families.
As a result, I tested ~50 models across as many model families as I could gather, frontier and open weight, API (gateway and direct) and self-hosted. Evaluation was based on a set of cosine similarity validations that was repeated across ~50 different embedding models.
Interestingly, frontier models did worse on the tasks than open weight models. However, when it came to costs, the picture was reversed: frontier models were much, much more token-efficient. In fact, almost no open-weight model was able to meet the initial token budget, while almost all frontier models did. Moreover, open weight models struggled massively with reasoning, in terms of latency and token consumption.
I also found that the latest models did not perform better than older models. And any a priori benchmarking data was utterly useless.
So, I ended up using a set of open weight models without reasoning, as it turned out reasoning as well as frontier negatively correlated with the tasks. However, before I knew this, I had spent a lot of time running each available reasoning level for each model.
Lastly, as an aside, when it came to embedding models, size (dims as well as model size) did not correlate with quality, once a hurdle figure (~2k dims) was met. In fact, sweet spot was 3-5K, and for my (text-based) set of tasks, dense models tended to outperform MoE ones.