By "capitalist," I'm referring to investors whose primary contribution is capital, not making a political statement about capitalism itself.
Capital is crucial when tools and infrastructure are expensive. Consider publishing: pre-internet, starting a newspaper required massive investment in printing presses, materials, staff, and distribution networks. The web reduced these costs dramatically, allowing established media to cut expenses and focus on content creation. However, this also opened the door for bloggers and digital news startups to compete effectively without the traditional capital requirements. Many legacy media companies are losing this battle.
Unless AI systems remain prohibitively expensive (which seems unlikely given current trends), large corporations will face a similar disruption. When the tools of production become accessible to individuals and small teams, the traditional advantage of having deep pockets diminishes significantly.
The typical AI economic discussion always focuses on job loss, but that's only half the story. We won't just have corporations firing everyone while AI does all the work - who would buy their products then?
The disruption goes both ways. When AI slashes production costs by 10-100x, what's the value proposition of traditional capital? If you don't need to organize large teams or manage complex operations, the advantage of "being a capitalist" diminishes rapidly.
I'm betting on the rise of independents and small teams. The idea that your local doctor or carpenter needs VC funding or an IPO was always ridiculous. Large corps primarily exist to organize labor and reduce transaction costs.
The interesting question: when both executives and frontline workers have access to the same AI tools, who wins? The manager with an MBA or the person with practical skills and domain expertise? My money's on the latter.
AI is an existential threat to tech companies not software engineers.
In many domains, the scope and complexity of software systems goes beyond the ability of a single software engineer to manage. A coordination layer becomes necessary when the number of engineers required goes beyond a threshold (say 5 or so). When the development effort must be coordinated over extended periods (say several months or years), mechanisms to raise capital and manage risk become necessary. These functions are why companies exist.
Consider that a massive increase in software engineer productivity will make coordination unnecessary for many kinds of software. In the market that opens up, companies with expensive executives, middle management and coordination inefficiencies will not be competitive. Smaller shops with a solo engineer or a team of less than 5 will outcompete larger players because their costs will be significantly lower. Massive one-size-fits-all products will be harder to justify when a small dev shop can quickly build or customise software for the unique requirements of a business or niche.
Before the CEOs stop needing engineers, engineers will stop needing CEOs and managers to coordinate their efforts and raise capital.
If AI increases the productivity of a single engineer between 10-100x over the next decade, there will be a seismic shift in the industry and the tech giants will not walk away unscathed.
There are coordination costs to organising large amounts of labour. Costs that scale non-linearly as massive inefficiencies are introduced. This ability to scale, provide capital and defer profitability is a moat for big tech and the silicon valley model.
If a team of 10 engineers become as productive as a team of 100-1000 today, they will get serious leverage to build products and start companies in domains and niches that are not currently profitable because the middle managers, C-Suite, offices and lawyers are expensive coordination overhead. It is also easier to assemble a team of 10 exceptional and motivated partners than 1000 employees and managers.
Another way to think about it is what happens when every engineer can marshal the AI equivalent of $10-100m dollars of labour?
My optimistic take is that the profession will reach maturity when we become aware of the shift in the balance of power. There will be more solo engineers and we will see the emergence of software practices like the ones doctors, lawyers and accountants operate.
The government is getting rid of tax exemptions for non-domiciled individuals who up to this point did not need to pay tax on foreign source income that was not brought into the UK. The economy overall is in a dire state, Labour are projected to win the election this week in a landslide. The rich are worried about further tax raids from a left wing government.
In the UK, the loss of subsidised childcare and tax free allowances creates an effective marginal tax rate of over 100% for a parent earning £100,000-£125,000.
This isn't quite fair. The problem is that JavaScript expertise is still volatile. It is a use it or lose it skill. It quickly becomes alien unless you stay on the threadmill and keep up with the developments, patterns and tooling. I admit syntax is not actually difficult to relearn but patterns take time to acquire.
I've been programming for over 20 years and the languages and frameworks I favour have a sticky quality. These are the ones I can put away for years, return, get back into it and pick up new features in a few hours at most. The cognitive load of getting back into them are low.
I picked up django around 2011 and moved on to other things including JavaScript at the time. When I needed to build a Django project in 2016, there were some improvements and new features but the syntax, tooling and patterns were the same. It was possible to just jump right back in after a quick skim of the docs. It is possible to develop mastery of this stack, set it down for a few years and return to it with familiarity and continue on that journey.
The JavaScript ecosystem in that period however has pulled off a massive amount of transformation. I learned the basics of jQuery, yui, knockout, emberjs, backbone, coffeescript, ecmascript X, knockout, baconjs, rxjs, react with classes, react with hooks, redux, typescript, nextjs, react server components, react native, graphql, assemblyscript and on and on. It has been impossible for me to develop any deep mastery of anything in the JavaScript space. This does not even include the insane tooling with vague overlapping responsibilities. The closest I have found to reasonable stability in this area has been clojurescript with reagent.
I am constantly stuck in tutorial mode when I need to dive back into the frontend. I recently upgraded a 4 year old Django codebase in 2 hours. I am ripping out all of the react code because it is just not worth the effort any more. The tooling, dependencies and patterns are now outdated. HTMX has that simplicity and elegance that will keep the cognitive load manageable while I focus on the real problems.
This is exactly right. The lakehouse is a custom data warehouse you can build out of these cloud primitives to suit the specific data needs of an organisation. Think of it as a database scaled up by several orders of magnitude. Everything from storage costs to latency can be optimised as design choices. The common core in this architecture is data held in standard file formats such as parquet, delta tables, avro etc.
One of the most powerful ways to integrate LLMs with existing systems is constrained generation. Libraries such as outlines[1] and instructor[2] allow structural specification of the expected outputs as regex patterns, simple types, jsonschema or pydantic models. Llama.cpp supports bnf grammars.
These outputs often consume significantly fewer tokens than chat or text completions.
There are well described links between intelligence and information theory. Intelligence is connected to prediction and compression as measures of understanding.
Intelligence has nothing specific to do with The Universe as we known it. Any universe will do, a simulation, images or a set of possible tokens. The universe is every possible input. The training set is a sampling drawn from the universe. LLMs compress this sampling and learn the processes and patterns behind it so well that they can predict what should come next without any direct experience of our world.
All machine learning models and neural networks are pure functions. Arguing that no function can have intelligence as a property is equivalent to claiming that artificial intelligence is impossible.
We have an algorithm and computational hardware that will tune a universal function approximator to fit any dataset with emergent intelligence as it discovers abstractions, patterns, features and hierarchies.
So far, we have not yet found hard limits that cannot be overcome by scaling the number of model parameters, increasing the size and quality of training data or, very infrequently, adopting a new architecture.
The number of model parameters required to achieve a defined level of intelligence is a function of the architecture and training data. The important question is, what is N, the number of model parameters at which we cross an intelligence threshold and it becomes theoretically possible to solve mathematics problems at a research level for an optimal architecture that we may not yet have discovered. Our understanding does not extend to the level where we can predict N but I doubt that anyone still believes that it is infinity after seeing what GPT4 can do.
This claim here is essentially a discovery that N may be much closer to where we are with today's largest models. Researchers at the absolute frontier are more likely to be able to gauge how close they are to a breakthrough of that magnitude from how quickly they are blowing past less impressive milestones like grade school math.
My intuition is that we are in a suboptimal part of the search space and it is theoretically possible to achieve GPT4 level intelligence with a model that is orders of magnitude smaller. This could happen when we figure out how to separate the reasoning from the factual knowledge encoded in the model.
A strategic play is to release the models - the nuclear option. Go full OpenAI. Even if Microsoft has an exclusive license, leaks and breaches happen.
The board should have led with this as an implied threat behind closed doors and given both Altman and Nadella a face-saving public retreat. Altman goes off to start GPTSoft with equity, a license to GPT4 and access to Microsoft's cloud compute. Everyone gets to call this a win. Public humiliation was a mistake.
Absolutely agree with this. Real skin in the game would require CEOs and executives to face ruin in the event of failure.
Pilots have skin in the game. There is rarely any doubt that their priority is landing that plane safely. Now imagine a world where airline pilots are able to safely eject with a golden parachute and minimal damage to their professional reputation. What are the chances of achieving the same safety record?
Capital is crucial when tools and infrastructure are expensive. Consider publishing: pre-internet, starting a newspaper required massive investment in printing presses, materials, staff, and distribution networks. The web reduced these costs dramatically, allowing established media to cut expenses and focus on content creation. However, this also opened the door for bloggers and digital news startups to compete effectively without the traditional capital requirements. Many legacy media companies are losing this battle.
Unless AI systems remain prohibitively expensive (which seems unlikely given current trends), large corporations will face a similar disruption. When the tools of production become accessible to individuals and small teams, the traditional advantage of having deep pockets diminishes significantly.