Most technologies aren't races(astralcodexten.substack.com)
astralcodexten.substack.com
Most technologies aren't races
https://astralcodexten.substack.com/p/most-technologies-arent-races
15 comments
Two things can be true at once:
* being first doesn't guarantee success (otherwise we would all own Xerox laptops right now)
* hard research and development is always a race in terms of getting to a product before running out of money
Basic economic theory shows there is always a scarcity of resources - money, time, market, people. The faster you are in front of the pack, the more likely you will get the lions share of money, the more people buying your product, and the more people wanting to join your venture.
The secret is to never fool yourself that you have unlimited time or money. That pathology leads companies to burn cash and take too much time choosing the "perfect" over the "really good", never getting to market. That's where I believe most startups failed in the past few years, and we're seeing that now more clearly with the desert of VC investment. You can fool yourself, but you cannot pretend basic economic forces don't exist.
* being first doesn't guarantee success (otherwise we would all own Xerox laptops right now)
* hard research and development is always a race in terms of getting to a product before running out of money
Basic economic theory shows there is always a scarcity of resources - money, time, market, people. The faster you are in front of the pack, the more likely you will get the lions share of money, the more people buying your product, and the more people wanting to join your venture.
The secret is to never fool yourself that you have unlimited time or money. That pathology leads companies to burn cash and take too much time choosing the "perfect" over the "really good", never getting to market. That's where I believe most startups failed in the past few years, and we're seeing that now more clearly with the desert of VC investment. You can fool yourself, but you cannot pretend basic economic forces don't exist.
These arguments make sense.
I think there isn't going to be a single dominant LLM. There will be multiple competing ones. It's difficult to see how companies will achieve lock in. Natural language is the most flexible interface. A front end app could easily switch from one LLM to a different LLM behind the scenes with very little effort. It's basically one API endpoint...
How intelligent can the LLMs get and still be useful to humans? There is a point beyond which intelligence won't have any use for us. Humans may have already reached this point in some industries. Intelligence above a certain threshold doesn't have much value.
I think there isn't going to be a single dominant LLM. There will be multiple competing ones. It's difficult to see how companies will achieve lock in. Natural language is the most flexible interface. A front end app could easily switch from one LLM to a different LLM behind the scenes with very little effort. It's basically one API endpoint...
How intelligent can the LLMs get and still be useful to humans? There is a point beyond which intelligence won't have any use for us. Humans may have already reached this point in some industries. Intelligence above a certain threshold doesn't have much value.
I think Elon is trying to delay AI because he is so far behind in his AI ambitions. Altman is getting so much press and Elon is noticeably jealous.
The pause would give him time to spin up a startup and attempt to compete.
The pause would give him time to spin up a startup and attempt to compete.
Not sure if his latest call is to buy more time, but to say that it is his only motive is just wrong. Musk has always voiced concern about AI, even if he pushes for it at the same time. This is from 2017 when OpenAI was still in his plan:
https://www.cnbc.com/2017/09/04/elon-musk-says-global-race-f...
> “Last month he co-signed a letter with other leaders in robotics including Google Deep Mind’s Mustafa Suleyman, calling on the United Nations to ban killer robots”
https://www.cnbc.com/2017/09/04/elon-musk-says-global-race-f...
> “Last month he co-signed a letter with other leaders in robotics including Google Deep Mind’s Mustafa Suleyman, calling on the United Nations to ban killer robots”
Elon's been hesitant about AI for a while now, so I don't think it's only jealous, but I do agree that I think that's part of it.
Except he isn't hesitant about the AI that is beneficial to him and his company. It doesn't make him wrong but it does make him a bit hypocritical.
Totally agree that he's a hypocrite, but I've always assumed that "okay for me, but not for thee" mindset came mainly from ego.
"If Tesla builds an AI, I'm smart enough that I can keep it in check. If someone else builds an AI their CEO is probably not smart enough to do the same thing."
Elon has a long history of thinking he's smarter than everyone else and that even people who are experts in their field aren't as capable as he is (cave submarine).
"If Tesla builds an AI, I'm smart enough that I can keep it in check. If someone else builds an AI their CEO is probably not smart enough to do the same thing."
Elon has a long history of thinking he's smarter than everyone else and that even people who are experts in their field aren't as capable as he is (cave submarine).
Most likely, yes. With Tesla and autopilot, he was getting a lot of press. Now, no one is talking about Tesla.
hasn't he been saying these things even before GPT, though?
>China is currently about 2 years behind the US in AI.
How can this be true if AI is not a race? What does it even mean to be behind in a technology that isn't a race?
How can this be true if AI is not a race? What does it even mean to be behind in a technology that isn't a race?
Traffic isn’t a race either, but you can still be behind other cars.
The first country to develop AI will either permanently rule the world or be destroyed with the rest of us.
This isn't rocket science.
This isn't rocket science.
> But suppose AI is “only” a normal transformative technology, no more important than electricity, automobiles, or computers.
Most people who think the AI "race" is important aren't supposing this. So I don't think you can just "suppose" this away in the second sentence.
Most people who think the AI "race" is important aren't supposing this. So I don't think you can just "suppose" this away in the second sentence.
I'll throw out that two additional motivations that entities have to frame things as a race:
- Delay thoughtful regulation of the technology - if you can delay it long enough until the entity gets big enough to fund lobbying efforts, its much easier to reach steady state/too big to fail status
- Gain access to government funding - as a consultant, if there's no existential crisis of being forever stuck in second place (no one wants to be behind), its much harder to provide thought leadership seminars/get fully paid vacations (i.e. speaking events).
- Delay thoughtful regulation of the technology - if you can delay it long enough until the entity gets big enough to fund lobbying efforts, its much easier to reach steady state/too big to fail status
- Gain access to government funding - as a consultant, if there's no existential crisis of being forever stuck in second place (no one wants to be behind), its much harder to provide thought leadership seminars/get fully paid vacations (i.e. speaking events).
Also, if you goal is full employment, think of the GDP advantage of a country having access to these technologies. It's a huge workforce multiplier.
High GDP does not in itself result in high employment. It is simply a measure of wealth produced in the measured economy.
If I owned a fully-robotic factory that produced all the goods this country consumed, unemployment would be at 100%, but GDP would be the same (Because the amount of wealth produced would be the same as it is today.)
Capitalists aren't going to share their profit until we take it from them, either through taxes[1], or, ah, 'land reform'[2].
[1] The carrot.
[2] The stick.
If I owned a fully-robotic factory that produced all the goods this country consumed, unemployment would be at 100%, but GDP would be the same (Because the amount of wealth produced would be the same as it is today.)
Capitalists aren't going to share their profit until we take it from them, either through taxes[1], or, ah, 'land reform'[2].
[1] The carrot.
[2] The stick.
Yes and No.
Output always creates more work, due to second law of thermodynamics (entropy). So, the more output you have, the more work you need to fight entropy. May be the Robots can deal with entropy too, but we aren't there yet.
So, AI will increase need of workforce / labor
Output always creates more work, due to second law of thermodynamics (entropy). So, the more output you have, the more work you need to fight entropy. May be the Robots can deal with entropy too, but we aren't there yet.
So, AI will increase need of workforce / labor
> Output always creates more work, due to second law of thermodynamics (entropy).
...You're gonna have to explain that one, because I don't see how those things are connected at all.
...You're gonna have to explain that one, because I don't see how those things are connected at all.
every good and service created need additional maintenance/fixing/organizing. It also increases complexity
If we're positing a fully robotic factory to create the products, I feel very comfortable positing that either a) the products are fully disposable, or b) there can also be fully robotic maintenance/fixing/organizing of said products.
That's only true if we're at 100% efficiency. Which I can say with good certainty, we are not.
Apple didn't "win" the smartphone race, but they had a five year head start and ended up becoming the most valuable (publicly traded) company in the world.
Tesla didn't "win" the EV market (and will likely dwindle), but it did become the most valuable car company and made Musk the richest person in the world (for a time).
OpenAI likely won't win the AI market, but they're landing multi-million deals left right and center. They've got the home advantage as well - US and EU companies don't want to deal with Chinese hard- or software.
It's not so much about winning, it's about getting a head start.
Tesla didn't "win" the EV market (and will likely dwindle), but it did become the most valuable car company and made Musk the richest person in the world (for a time).
OpenAI likely won't win the AI market, but they're landing multi-million deals left right and center. They've got the home advantage as well - US and EU companies don't want to deal with Chinese hard- or software.
It's not so much about winning, it's about getting a head start.
It's easy to get a "head start" effect if you pick the top companies and make a cutoff line just before those companies launched their products.
If you include pre-iphone smartphones and pre-tesla EVs, you'll get the opposite conclusion: Apple and Tesla demonstrated that a head start isn't necessary to surpass competitors and achieve market dominance. By making better technology you can leapfrog companies (like Nokia) that have been around for decades.
If you include pre-iphone smartphones and pre-tesla EVs, you'll get the opposite conclusion: Apple and Tesla demonstrated that a head start isn't necessary to surpass competitors and achieve market dominance. By making better technology you can leapfrog companies (like Nokia) that have been around for decades.
Apple did not have any head start, when they entered the market there were already smartphones on the market (e.g.; BlackBerry and Palm Treo). Apple just brought out a significantly better product, and it took years for a good response to come out (and for the power of Apple's change to be appreciated fully).
So this is evidence that the first move is not always the winning stroke.
So this is evidence that the first move is not always the winning stroke.
You could even argue the delay is what let Apple win. Apple jumped in the moment they could build a UI paradigm around fingers and capacitive screens. Its competitors had spent a decade building a UI scheme and software ecosystem around either trackballs and keyboards or resistive touchscreens and styluses, and weren't willing to throw that away when multitouch came along.
> You could even argue the delay is what let Apple win.
Apple won because of superior design and engineering. It became glaringly obvious when BlackBerry released the Storm one month or so before or after Apple released the iPhone 3G (and the App Store).
I remember thinking it was a prototype when someone showed a Storm to me. Scrolling was janky, the whole phone was sluggish and slow and the keyboard barely worked. Blackberry couldn't even figure out how to get Wifi on the thing, so it was stuck using cellular while my iPhone was just streaming video. And of course, no apps. It was 2 years behind the original iPhone and shipped two years later.
Apple won because of superior design and engineering. It became glaringly obvious when BlackBerry released the Storm one month or so before or after Apple released the iPhone 3G (and the App Store).
I remember thinking it was a prototype when someone showed a Storm to me. Scrolling was janky, the whole phone was sluggish and slow and the keyboard barely worked. Blackberry couldn't even figure out how to get Wifi on the thing, so it was stuck using cellular while my iPhone was just streaming video. And of course, no apps. It was 2 years behind the original iPhone and shipped two years later.
And there were plenty of people in 2007 happy to dunk on Apple for not having a physical keyboard and various other aspects of the design that didn't represent tradeoffs that people were accustomed to but turned out to be appropriate (for most customers).
Blackberry and Palm Treo and the rest were called "smartphones", sure, but let's be real Apple completely rebooted the category. Everything that existed before iPhone was instantly irrelevant. Its competitors has to start from scratch to incorporate large multitouch screens and reimagine the basic UI and apps.
The first iPhone could be barely considered “smart” comparatively to its competitors. They weren’t instantly irrelevant.
Every iPhone released in the last 15 years has been an incremental upgrade on the first one. And in turn every other smartphone has been a copy of the equivalent iPhone. No one is drawing inspiration from the top of the line Nokia or Blackberry from 2008. That entire segment is dead.
iPhone obviously copied lots of features from Android [1], but the even back at the original iPhone they copied features the old Nokia or Blackberry segment. Not just talking about email or calendars, but things like iPhone 2 copying the idea of an app store.
Sure the iPhone introduced lots of cool new stuff. But some people on hacker news seem to think it happened in a vacuum and nobody else had ideas that mattered.
[1]: https://www.androidpolice.com/2020/06/27/features-apple-borr...
Sure the iPhone introduced lots of cool new stuff. But some people on hacker news seem to think it happened in a vacuum and nobody else had ideas that mattered.
[1]: https://www.androidpolice.com/2020/06/27/features-apple-borr...
Instantly irrelevant is certainly one way of looking at it. You are talking about a platform where developers originally had to beg for an appstore. Even Kim Kardashian was using a blackberry until 2015.
Form factor wise there's no argument. The iPhone completely changed the design of smartphones. Every pre-iPhone smartphone form factor except for the Blackberry was dead within 2 years. Even the Blackberry died out over time, it was just sustained by the governments and businesses with multi-year contracts a bit longer.
It’s very unrealistic to believe that it was developers begging that caused Apple to create the App Store. It’s possible the begging might have accelerated the process because apple wanted to capitalize on the demand, but there is no way there wouldn’t have been an App Store.
> Even Kim Kardashian was using a blackberry until 2015.
How much did this cost BlackBerry?
How much did this cost BlackBerry?
I'm willing to bet they paid nothing; only for the reason that RIM completely screwed the pooch on mass market smartphones and exclusively focusing on enterprise. I remember BBM having incredible penetration in college-aged women but Blackberry didn't seem interested in anything other than enterprise until it was too late.
Edit: I did a quick Google Search and she was buying them on Ebay. I think it was one of the many misplays by the company.
Edit: I did a quick Google Search and she was buying them on Ebay. I think it was one of the many misplays by the company.
Apple didn't "win" the smartphone race, but they had a five year head start and ended up becoming the most valuable (publicly traded) company in the world.
The iphone was in 2007. The Blackberry preceded it by many years. Same for the Palm Pilot.
The iphone was in 2007. The Blackberry preceded it by many years. Same for the Palm Pilot.
This photo is a good if incomplete depiction of cellphones before and after iPhone [0]
Many traditional phone makers ceased to exist in the iPhone era — Blackberry, Nokia, Palm, Motorola, Ericsson, etc. Many new players took their places — Google, Samsung, Xiaomi, HTC. All smartphones are similar today. In that context Apple is a pioneer, with a head start. Wasn’t for long, mind you.
[0]-https://www.cultofmac.com/wp-content/uploads/2012/02/beforea...
Many traditional phone makers ceased to exist in the iPhone era — Blackberry, Nokia, Palm, Motorola, Ericsson, etc. Many new players took their places — Google, Samsung, Xiaomi, HTC. All smartphones are similar today. In that context Apple is a pioneer, with a head start. Wasn’t for long, mind you.
[0]-https://www.cultofmac.com/wp-content/uploads/2012/02/beforea...
I had both, and frankly it’s like comparing a TRS-80 to a Mac.
The point of the article isn't that some companies/countries won't have advantages - he specifically points out all the examples where they do - but just that it isn't critical to AI alignment concerns.
You can think like that in the USA or China, but in the rest of the world its quite obvious we’re not winning any races and we know the consequences of not winning them
Yeah, but the context of the article is whether we should work on alignment because of the possible existential threat to humanity if a hard takeoff scenario is plausible. If it is, AI-alignment becomes paramount for everyone. If not, then winning the race matters more, for countries like the US and China.
There seems to be some unfortunate social herding around AI beliefs in the 'rationalist' community that seems very un-rationalism-like to me. Some of these arguments are really poor.
"when the US invented nukes, its enemies were stuck with normal bombs; there is no slightly-worse-nuke that can only destroy half a city."
I'm not an expert on what happened to Dresden but I heard it was ... a lot.
I'm not an expert on what happened to Dresden but I heard it was ... a lot.
There is a bit of a difference between needing to commit dozens or hundreds of expensive bombers to kill a city, and only needing to commit one. You go from being able to knock out a city in a day (... and with, probably, weeks worth of munitions production) with an air wing, to being able to knock out all the cities in a country in a day with an air wing. With enough nukes and a modest number of bombers at one's disposal, all but overwhelmingly-strong air defenses become incapable of protecting a country from destruction—enough will get through to cause catastrophic damage (talking 40s and 50s tech here—as air defense tech got better, ballistic and cruise missiles overtook bombers in this kind of scenario).
Concentrating the damage like that gives the enemy far less time to adapt or rebuild, and less capacity to do so—you can out-produce heavy damage that's spread out over a long period of time, you can adapt to it, you can shift tactics and resources around to reduce the rate of damage in the future, but such efforts are hopeless against nukes.
It wasn't especially in-play during WWII since even the US couldn't (yet) build the bombs fast enough for them to really matter that much versus what could be done with conventional weapons, but it did matter soon after as stockpiles began to grow and production rate increased. Nukes represented a huge multiplier in the amount of destructive power a state could realistically stockpile and deploy in short order.
Concentrating the damage like that gives the enemy far less time to adapt or rebuild, and less capacity to do so—you can out-produce heavy damage that's spread out over a long period of time, you can adapt to it, you can shift tactics and resources around to reduce the rate of damage in the future, but such efforts are hopeless against nukes.
It wasn't especially in-play during WWII since even the US couldn't (yet) build the bombs fast enough for them to really matter that much versus what could be done with conventional weapons, but it did matter soon after as stockpiles began to grow and production rate increased. Nukes represented a huge multiplier in the amount of destructive power a state could realistically stockpile and deploy in short order.
I was going to quote that exact sentence. A night of bombing Tokyo killed approximately as many people as the nuclear bomb on Hiroshima. https://en.m.wikipedia.org/wiki/Bombing_of_Tokyo
I feel like the fact that that was 334 planes dropping 1665 tons of mostly 500lb bombs over the course of a night is an important distinction from a single bomb that does the same damage.
Not really as even with WWII tech the bombs could target specific areas. A nuclear bomb is more destructive for a few blocks around the blast, but get a couple miles out in the city and the damage is much less than a smaller bomb hit.
Modern military doctrine is hitting a whole city is bad. You want to hit specific targets and not collateral damage to other things. Discredited WWII doctrine was killing civilians would make them surrender so hitting the whole city was good. Which lens you look at military doctrine says which is better.
Modern military doctrine is hitting a whole city is bad. You want to hit specific targets and not collateral damage to other things. Discredited WWII doctrine was killing civilians would make them surrender so hitting the whole city was good. Which lens you look at military doctrine says which is better.
Obviously there are huge differences and it ushered in a new era of weaponry. What I want to point out is that atomic weapons were not an immediate step-change in destructive power. They’re actually not an exception to the greater point the author was making.
The destruction caused by a single military operation was in the same league. The logistics were actually harder in some ways than traditional bombing. It would be months before the US would have more atomic weapons after the first two. So the one bomb versus hundreds doesn’t make a big difference when you have zero.
This would all change dramatically over the coming years. But at the time of its first use, the atomic bomb was not as much of a leap forward in capability as you’d probably think.
The destruction caused by a single military operation was in the same league. The logistics were actually harder in some ways than traditional bombing. It would be months before the US would have more atomic weapons after the first two. So the one bomb versus hundreds doesn’t make a big difference when you have zero.
This would all change dramatically over the coming years. But at the time of its first use, the atomic bomb was not as much of a leap forward in capability as you’d probably think.
He's not saying there weren't other technologies equally destructive as nukes at the time. Just that in the world of nuclear bombs there is no half version.
Seems like other replies are trying to harp on this to undermine the entire article, when they are just deliberately misinterpreting the point. Maybe no analogy is perfect?
Seems like other replies are trying to harp on this to undermine the entire article, when they are just deliberately misinterpreting the point. Maybe no analogy is perfect?
The paragraph is a bit muddled. I took it as pointing to American dominance at the end of the war, preventing Soviet over-reach or immediate direct confrontation with the Allies. After all, "Americas enemies" in the sense of Germany, Japan, and Italy never got The Bomb—they still don't have it, even if they could all likely get there pretty fast if they wanted to, these days—in the sense of building their own nuclear weapons, so he can hardly mean them as the enemies who got the bomb "soon after"—that's the Soviets, our wartime frenemy, and I expect nukes were a significant factor in shaping the wrap-up of that war, even if they played only a minor role in bringing the actual victory over the axis powers.
But I do think the broader point of nukes being a "you have it, or you don't and can't even really approach the capability of those who do" sort of tech is accurate.
But I do think the broader point of nukes being a "you have it, or you don't and can't even really approach the capability of those who do" sort of tech is accurate.
To emphasize the main substantive point, the US having access to nuclear weapons while the other side did not was not a factor in who won WW II. By the time the US had nukes, Germany had already surrendered and Japan was beaten. (And Japan knew it; they had been making peace feelers through various avenues for months.) WW II was won by conventional weapons.
In other words, the basic historical claim being made here, that the US's "brief period of dominance" in nukes won WW II, is false. Not only that, it's "schoolboy error" false: the sort of mistake nobody who claims any sort of knowledge about the subject they're writing about should make.
In other words, the basic historical claim being made here, that the US's "brief period of dominance" in nukes won WW II, is false. Not only that, it's "schoolboy error" false: the sort of mistake nobody who claims any sort of knowledge about the subject they're writing about should make.
It was possible to destroy a city pre-nukes with a dedicated bombing campaign, but it was not possible to destroy a city with a single plane/missile/bomb.
Nuclear powers can credibly threaten to not just destroy a single city, but to wipe a modern nation off the map. That's something you can't do with conventional bombers, even a lot of them.
I think his specific claim is not quite right here, but the thrust of his argument is. There's a reason that nuclear powers dare not enter direct conflict with each other in a way that major military powers used to casually do.
Nuclear powers can credibly threaten to not just destroy a single city, but to wipe a modern nation off the map. That's something you can't do with conventional bombers, even a lot of them.
I think his specific claim is not quite right here, but the thrust of his argument is. There's a reason that nuclear powers dare not enter direct conflict with each other in a way that major military powers used to casually do.
> Nuclear powers can credibly threaten to not just destroy a single city, but to wipe a modern nation off the map. That's something you can't do with conventional bombers, even a lot of them.
I think the key insight here is rate-of-production (measured in, say, tons of TNT equivalent, ready for delivery to a target), ease of stockpiling huge levels of destructive power, and cost of delivering that destructive power. The first two are incredibly higher for nukes vs. conventional weapons, and the last, far lower.
500 late-WWII Allied heavy bombers couldn't wipe out all the major cities in, say, the Soviet Union in a couple days. Impossible. The logistics simply don't work, there's no halfway-plausible scenario in which they could even get close to achieving that, there are tons of obstacles standing in the way.
.. with a few hundred nukes, though? Now it's totally possible.
I think the key insight here is rate-of-production (measured in, say, tons of TNT equivalent, ready for delivery to a target), ease of stockpiling huge levels of destructive power, and cost of delivering that destructive power. The first two are incredibly higher for nukes vs. conventional weapons, and the last, far lower.
500 late-WWII Allied heavy bombers couldn't wipe out all the major cities in, say, the Soviet Union in a couple days. Impossible. The logistics simply don't work, there's no halfway-plausible scenario in which they could even get close to achieving that, there are tons of obstacles standing in the way.
.. with a few hundred nukes, though? Now it's totally possible.
Not to mention that
> there is no slightly-worse-nuke that can only destroy half a city.
Is itself false. The "Little Boy" that dropped on Hiroshima was the "slightly-worse-nuke" with a moderate blast damage radius (5 psi) of 1.67 km. The Dong Feng 5, China's largest nuclear warhead currently deployed, has a moderate blast damage radius (5 psi) of 24.5 km.
> there is no slightly-worse-nuke that can only destroy half a city.
Is itself false. The "Little Boy" that dropped on Hiroshima was the "slightly-worse-nuke" with a moderate blast damage radius (5 psi) of 1.67 km. The Dong Feng 5, China's largest nuclear warhead currently deployed, has a moderate blast damage radius (5 psi) of 24.5 km.
But that's still a nuke.
You still need to do every bit as much science and engineering to make it; the only difference is you need slightly less fissile material.
You still need to do every bit as much science and engineering to make it; the only difference is you need slightly less fissile material.
Dresden took literally hundreds of thousands of bombs...
Astral Codex Ten (or Slate Star Codex, or,...) would be a lot more compelling if one didn't always find some obvious historical error like this in the typical article. If only he could afford a fact-checker.
This is not a historical error. It is objectively true that there is no conventional bomb which can destroy half a city.
Insofar as it is objectively true, it is irrelevant to his point; insofar as it is relevant to his point, it is historically false.
Dresden, Tokyo, Hanoi—how many more counterexamples do you need? The means to level a city with conventional bombs already existed. The nuclear bomb's innovation was as a weapon of intimidation, not raw destructive force.
Dresden, Tokyo, Hanoi—how many more counterexamples do you need? The means to level a city with conventional bombs already existed. The nuclear bomb's innovation was as a weapon of intimidation, not raw destructive force.
> Dresden, Tokyo, Hanoi—how many more counterexamples do you need? The means to level a city with conventional bombs already existed. The nuclear bomb's innovation was as a weapon of intimidation, not raw destructive force.
Conventional strategic bombers and high explosives would never have given us MAD. The raw destructive force is absolutely relevant, it was (and remains) infeasible to do anything like wipe out a whole country in a day or two with conventional weapons, that'd take months or years and enormous dedication of resources. With nukes? Now it's possible.
I'm really not getting the reading of this as "LOL we could blow up cities with regular bombs, he's wrong!" Like, WTF was all the worry over nukes in the cold war, then? They're extremely obviously a major game-changer. A functioning nuke program means multiplying the max plausible destructive power a country can deploy inside a week by a very large number—no later developments in conventional weapons have really closed that gap, it's a whole other category of weapon, and you either have it, or you don't.
Conventional strategic bombers and high explosives would never have given us MAD. The raw destructive force is absolutely relevant, it was (and remains) infeasible to do anything like wipe out a whole country in a day or two with conventional weapons, that'd take months or years and enormous dedication of resources. With nukes? Now it's possible.
I'm really not getting the reading of this as "LOL we could blow up cities with regular bombs, he's wrong!" Like, WTF was all the worry over nukes in the cold war, then? They're extremely obviously a major game-changer. A functioning nuke program means multiplying the max plausible destructive power a country can deploy inside a week by a very large number—no later developments in conventional weapons have really closed that gap, it's a whole other category of weapon, and you either have it, or you don't.
Nuclear warheads on ICBMs are wildly different from what the Allies had during WWII, and far beyond the scope of this debate. They also represent an incremental advance in the state of the art, and not the quantum leap Scott is failing to invoke here.
He shouldn't need to hire a fact checker. He just needs to realize that his error rate is too high when he just tosses off these historical claims based on his own background knowledge, and stop doing that. If the claim is really central to the argument, then take the time to check it. If it's not that important, then just leave it out.
Unfortunately, public intellectuals in our current society don't hold themselves to these kinds of standards (which would have been considered obvious for public intellectuals a couple of centuries ago), and no one else holds them to it either.
Unfortunately, public intellectuals in our current society don't hold themselves to these kinds of standards (which would have been considered obvious for public intellectuals a couple of centuries ago), and no one else holds them to it either.
> Unfortunately, public intellectuals in our current society don't hold themselves to these kinds of standards (which would have been considered obvious for public intellectuals a couple of centuries ago)
Have you read anything a couple of centuries old? The problem was, if anything, much worse!
Have you read anything a couple of centuries old? The problem was, if anything, much worse!
He's not a public intellectual and he doesn't want to be. But people treat him like one regardless, because he happened to attain a little clout as a "Rationalist".
He doesn't get to decide whether he's a public intellectual. If you publish your thoughts and ideas about issues that are of interest to the public, you're a public intellectual, whether you like it or not. Even more so if many people read what you publish, discuss it, and give it weight.
> If you don’t believe in crazy science fiction scenarios like these, fine. But then why are you so sure that it’s crucial to “win” the AI “race”?
The whole point of the article is that either AI is not that big of a deal and like many other technologies (e.g. electricity, cars, ... ), or it's so revolutionary that's it's so hard to align that we're doomed.
Well, can't it be something else? Can't it be something very safe which gives a country a 200 year head start?
The whole point of the article is that either AI is not that big of a deal and like many other technologies (e.g. electricity, cars, ... ), or it's so revolutionary that's it's so hard to align that we're doomed.
Well, can't it be something else? Can't it be something very safe which gives a country a 200 year head start?
Sure. We just don't have examples in history where the head start is as big as 200 years.
Well, there is a difference between the "developed" and "developing" world, but that doesn't seem to be due to access to a single specific technology...
Well, there is a difference between the "developed" and "developing" world, but that doesn't seem to be due to access to a single specific technology...
We also don't have examples of one of the two contemplated options, we haven't seen a technology that has destroyed the world either.
I disagree whole heartedly. There are winning countries for various technologies. Yes there is tech that can be outsourced to another country, passing the winning baton to that country. Countries can also spend a lot on R&D to reach parity with established winners, hence making multiple winners.
Lets use car manufacturing example. Yes North America and Germany became early winners, and eventually Japan and Korea have become part of the winning club. But many other countries are not winning here. There are countries that can only make subpar cars for the local market, or make no cars at all and must import them all.
So if they are winning, what exactly are these countries winning? They are winning money and influence. US and Japan sell more cars overseas, that's more money into their country and into the government's coffers.
Lets use car manufacturing example. Yes North America and Germany became early winners, and eventually Japan and Korea have become part of the winning club. But many other countries are not winning here. There are countries that can only make subpar cars for the local market, or make no cars at all and must import them all.
So if they are winning, what exactly are these countries winning? They are winning money and influence. US and Japan sell more cars overseas, that's more money into their country and into the government's coffers.
Thanks this was my immediate objection, as well.
And who can say that being an early computer & Internet center didn't privilege America? The majority of big Internet companies are American, and they're now collecting rent across the globe!
And who can say that being an early computer & Internet center didn't privilege America? The majority of big Internet companies are American, and they're now collecting rent across the globe!
> You may have GPT-3, GPT-4, some future GPT-5,
One thing that is a little different with these AIs is the ability to improve programming.
I bet, GPT-4 can make it simpler for humans to build/deploy GPT-5. GPT-5, could do even more of the heavy lifting for GPT-6.
One thing that is a little different with these AIs is the ability to improve programming.
I bet, GPT-4 can make it simpler for humans to build/deploy GPT-5. GPT-5, could do even more of the heavy lifting for GPT-6.
Imagine the Germans nuking London in 1941. Imagine the chilling effect this might have on the US public's willingness to enter WWII?
To say something had no impact is very different than it could have a MAJOR impact. And nuclear weapons are clearly an example of this.
AI is much the same, though I think 70% of the shift occurred with unmanned vehicles. The ability to send something with no person in it, to fight and kill a person is a HUGE game changer. When that 'something' gets better, smaller, and smarter... the edge becomes all the more decisive.
Also the ability to take in much more data... Imagine having 100,000 sensors you want to understand reasonably well. You may not have enough trained humans. But an AI? Maybe... And maybe it can correlate the signals etc.
AI is a game changer, and those saying it is as powerful as the bomb may not be underselling it. Mainly because of the precision and deniability the end results may offer.