I don't think it's that difficult. Their servers are stateless too. S3 is easy to migrate.
Database is more difficult, but tons of people have done it successfully.... meanwhile people who host their own LLMs are relatively small in number in comparison.
Most companies don't do their own data centers mainly because it is more expensive and less reliable. It's something they can just pay for the problem to go away. The calculus for hosting your own LLM is probably similar.
Even Stripe who built their own coding agents and has tons of money/resources still decides not to host their own LLMs.
Still, many people will prefer open-weight models. It is similar to how we prefer linux but still use AWS/Render/and whatever. It doesn't lock us in, and we can move providers if we want to.
It's the same argument why Google isn't never considered a monopoly to the online ads space. Because the whole ads industry is much larger.
Same thing with card payments. There are online card payments and in-store card payments and other kinds of payments. Stripe + Paypal volume is considered a fraction of it.
Open-source models + services. This is more attractive because it doesn't lock in the vendors. If I grow larger, I can decide to deploy the open-source models.
> is barely any more "effort" than buying and holding SPY, as the work is being entirely delegated
Effort is translated to money e.g. you do it yourself or you hire someone to do it; someone is spending that large amount of effort that is valued at millions of dollars/year.
If SPY is so great, then wealthy people would've just bought SPY.
But that's not what they are doing. They pay fund managers top money (think top 0.1% earner) to invest for them.
> Many strategies are dependent on that level of wealth (or designed to address problems that only apply to that level of wealth) for tax-related reasons.
Their main goal is to grow the funds. Tax-saving is secondary at best. Nobody would be okay with shrinking the fund to acquire tax-saving lol.