There are 2 economic arguments for why it would make sense for Chinese State to subsidize the open-sourcing of models beyond undermining Anthropic's and OpenAI's investments (and by proxy the American financial economy, ie capital class):
1. As induced demand for domestic semiconductor production, where the level and diversity (ie number of distinct corporate users) of demand for the hardware is tied to the availability of models you can run yourself, ie open-weight models. If you believe that semiconductors will continue to be an important sector for innovation, productivity growth, and security, then it would make sense to subsidize broadly now, for future gains later. This would be the same export-led manufacturing discipline that allowed China to successfully develop several other sectors over the last 50 years.
2. It is likely that the bulk of value production will happen above (and below, ie #1) the large models. We already know that 90% of the training cost (maybe even closer to 99%) is in the single pre-training, but that an enormous amount of the value is actually in the supervised, RL, constitutional fine-tuning, and harness building that happens afterward. So, if your interest was in maximizing the size of the pie, you may actively subsidize the pre-training so as to maximize the downstream usages. This induces a direct value transfer from the labs specializing in pre-training to all downstream builders and users. There's a similar logic to subsidizing or state-financing the construction of other infrastructure and basic research.
This was an interesting interview. Like a lot of great comedians, Odenkirk has a very grounded and bleak view of the world. I suppose a lot of art, comedy included, is a way of coping with their perspective, for themselves and for the audience.
Should have specified old used thinkpads. I’ve never bought one new. My daily driver is 10+yo, bought for $200 and upgraded mem, battery, and ssd with another $100.
Very Don Norman problem. It is odd that things like this aren't more of a "solved problem." Maybe they are in some places and builders in the US just ignore the best practices.
These are the points I'm making, which I think are fairly one-to-one with my original comment:
- American politics presents a false choice between Democrats and Republicans
- America is both a consumerist and corporatist society
- Anthropic asked for minimal limits on AI usage
- People view Anthropic's stand as heroic, while viewing OpenAI as villainous
- The false choice between Anthropic and OpenAI mirrors the false choice in
American politics.
- People at OpenAI, Anthropic, and elsewhere used to view ethical deployment of
AI as paramount, but those goalposts have shifted as financial and political
incentives changed.
- Specifically, the ethics of AI have become conveniently synonymous with the
current financial and political moment.
- The current political moment is fascist.
- Technology is broadly neutral and it is politics that primarily dictates how
technology is actually used and deployed, and therefore its broad impacts.
- The internet was developed in the neoliberal era, which began with the
election of Ronald Reagan and extended through the Obama presidency.
- The structure and dynamics of the internet over the last 30 years is more
reflective of neoliberal politics than it is of anything inherent in the
technology. Extreme privitization and the refusal to use public institutions
to provision or regulate public goods.
- AI is being developed in a new political era, begun with the first Trump
presidency, and taking more full shape under the second Trump presidency.
- We are likely to find that AI's trajectory is similarly dictated largely by
politics rather than anything inherent to the technology.
- With this political era being fascist and explicitly
neo-imperial/neo-colonial, I fear for the technology's impact on humanity.
It's funny how the false choice of American politics (Red vs Blue) also makes it into its consumerist corporatist life. That Anthropic's threadbare "limits" on government usage are seen as a heroic stand is a testament to just how far the goalposts on "ethical" deployment of AI have moved to the (fascist) right. As ever, politics precedes technology. We have Reagan's internet, we will have Trump's AI. God help us.
I think this is spot on. It’s interesting how rhetoric about “liberty” seems to practically serve oligarchy. I suppose an alternative to bans and regulations is to genuinely pursue the elimination of deprivation, orient our collective capacities towards our collective well-being, and then let people do what they will. Anything short of that seems to be a rather false liberty (and a rather false democracy, while we’re at it).
I always find it odd when media (and others) consider consumerism as somehow "helping" the economy. The economy is entirely about the collective activity of humans serving humans. Everything we make or do is really about prioritizing that activity over others. Why would it be advantageous to prioritize barely-distinguishable "new" devices over the myriad other things human labor and capital could be put to?
"Metaphorically, I think about any job given to Claude as having 3 dimensions. There's the breadth of the task (roughly how many lines of code it will touch), the depth of the task (the complexity, the layers of abstraction needed, the decision making involved, etc.), and the time spent working on it. Those three axes define a cube, and the size of the cube is how much entropy I'm shoving into the project."
That's an interesting conceptualization that tracks with my experience using CC. And they were able to get an impressive amount of work done:
"""
The specifics don't matter too much here, but for context, some of what I had it do:
- Research all the available on-device speech-to-text models with permissive licences
- Demo the transcription speed of each one on an android device attached to the PC
- Write a C wrapper for the best one (Moonshine) and build an embeddable dynamic library
- Build this for iOS, Android, Linux, and macOS, and integrate it with my app code using the FFI
- Build a Nim wrapper for the fdk-aac library
- Integrate it with miniaudio, so I can play AAC audio and pipe the audio into Moonshine
Thinking of bunny huang’s open source hardware efforts, a 28nm nommu system seems like a good long term open source target. How much complexity in the system is the mmu, and so how much complexity could we cut out while still having the ability to run third-party untrusted code?
I’m sorry, I did not intend to accuse you of being part of the evangelical community. Your article only prompted the thought I shared.
On the technical point, I think I do disagree, but open to changing my mind. What would be better? I’m working on an async runtime currently, written in C, and I’m using several intrusive doubly linked lists because of their properties I mentioned.
That would be the “arbitrarily bounded” allocation. Arbitrary because now you have to make a decision about how many items you’re willing to maintain despite that number being logically determined by a sum over an unknown set of modules.
> The last time I used [a doubly linked list] was in the nineties. I know the Linux kernel uses them, but that design was also laid down in the nineties; if you were designing a kernel from scratch today, you would probably not do it that way.
This and sibling comments about the supposed uselessness or outdatedness of linked lists taught me something about the disconnect between many systems engineers resistant to Rust and many members of the (evangelical) Rust community.
To address the technical topic first: a doubly linked list is what you reach for when you’re looking for an intrusive data structure with fast append and fast delete. You need to iterate over it, but don’t need random access. Queues where you may need to remove elements. Think a list of completions that can be cancelled.
Why an intrusive data structure? Because you don’t want to, or can’t allocate. Because you want only a non-owning reference to the object, and you don’t want to have to manage any other lifetimes or dynamic (potentially unbounded, or arbitrarily bounded) allocations. Intrusive data structures are a great tool to minimize allocations, and to keep allocations (and ownership) logically separated across modules.
And now onto the cultural topic. That a member of the Rust community might not know this (which is of course totally fine) is what taught me something, which was maybe obvious to many: Rust brings safety to a C++ style language and audience. For the same reasons many dislike C++, many dislike Rust. For the same reasons many would opt for C over C++ when they have the choice (either across all their projects or for certain projects), many continue to opt for C over Rust.
Rust will not be taking over systems programming for the same reasons C++ did not. Well, by the numbers, it probably did (and Rust probably will too), so maybe better to say that it will not stamp out C for systems programming for the same reasons C++ did not. And it’s not just because of the stubbornness of C programmers.
Systems programming has 2 cultures and practice groups: the C camp, and the C++ camp. The C++ camp is a big tent. I’d argue it includes Rust and Swift (and maybe D). Zig belongs to the C camp.
Safety is important, and maybe Rust has the right model for it, but it is not a solution for the C camp. It likely did not intend to be, and again, by the numbers, replacing C++ is probably the more important task.
Certainly onto something but misses how much large organizations are actually controlled by small organizations operating in the “large complex system” environment. It is only individuals and small organizations that have agency at all. Large organizations and large complex systems are both emergent, one with hierarchical control, and one with distributed control. What has really changed is how unequal small organizations have become in their influence and power. The small cadres of people at the “top” (of organizations, media, government, tech, etc) control/influence more and more, not only at the expense of other small organizations (power is zero sum) but also at the expense of the decentralized mechanism, ie the large complex system becomes increasingly hierarchically/centrally controlled (vs distributed/decentralized control).
Yes being able to have compilers as libraries inline in the same code and same language. That feels like what all these call for. Which really is the Lisp core I suppose. But with static types and heterogenous backends. MLIR I think hoped (hopes?) to be something like this but while C++ may be pragmatic it’s not elegant.
Maybe totally off but would dependent types be needed here? The runtime value of one “language” dictates the code of another. So you have some runtime compilation. Seems like dependent types may be the language of jit-compiled code.
Anyways, heady thoughts spurred by a most pragmatic of libraries. Cloudflare wants to sell more schlock to the javascripters and we continue our descent into madness. Einsteins building AI connected SaaS refrigerators. And yet there is beauty still within.
1. As induced demand for domestic semiconductor production, where the level and diversity (ie number of distinct corporate users) of demand for the hardware is tied to the availability of models you can run yourself, ie open-weight models. If you believe that semiconductors will continue to be an important sector for innovation, productivity growth, and security, then it would make sense to subsidize broadly now, for future gains later. This would be the same export-led manufacturing discipline that allowed China to successfully develop several other sectors over the last 50 years.
2. It is likely that the bulk of value production will happen above (and below, ie #1) the large models. We already know that 90% of the training cost (maybe even closer to 99%) is in the single pre-training, but that an enormous amount of the value is actually in the supervised, RL, constitutional fine-tuning, and harness building that happens afterward. So, if your interest was in maximizing the size of the pie, you may actively subsidize the pre-training so as to maximize the downstream usages. This induces a direct value transfer from the labs specializing in pre-training to all downstream builders and users. There's a similar logic to subsidizing or state-financing the construction of other infrastructure and basic research.