Amazon’s Chips Threaten Intel(nytimes.com)
nytimes.com
Amazon’s Chips Threaten Intel
https://www.nytimes.com/2018/12/10/technology/amazon-server-chip-intel.html
349 comments
Mike Tyson said: "Everyone has a plan until they get punched in the face." When it comes to semiconductors I'd say: "Everyone wants to make their own chips until they have to do so at scale". (Doesn't roll of the tounge as well!)
There is definitely a threat from Apple, Amazon, Google and especially China that will put Intel's market share in target distance, but making chips at scale is incredibly difficult. It's hard to see Amazon transitioning their AWS machines to Amazon built chips, but if they display competency they'll certainly be able to squeeze more out of Intel.
There is definitely a threat from Apple, Amazon, Google and especially China that will put Intel's market share in target distance, but making chips at scale is incredibly difficult. It's hard to see Amazon transitioning their AWS machines to Amazon built chips, but if they display competency they'll certainly be able to squeeze more out of Intel.
I'm by no means an expert in this, and maybe it's a bit obvious, but hadn't seen this mentioned yet.
I think as we run out of gains to be had from process size reductions, the next frontier for cloud providers is in custom silicon for specific workloads. First we saw GPUs move to the cloud, then Google announced their TPUs.
Behind the scenes, Amazon's acquisition of Annapurna Labs has been paying off with their Nitro (http://www.brendangregg.com/blog/2017-11-29/aws-ec2-virtuali...) ASIC, nearly eliminating the virtualization tax, and providing a ton of impressive network capabilities and performance gains.
The Graviton, which I believe also came from the Annapurna team, is likely just the start. Though a general purpose CPU, it's getting AWS started on custom silicon. AWS seems all about providing their customers with a million different options to micro-optimize things. I think the next step will be an expanding portfolio of hardware for specific workloads.
The more scale AWS has, the more it makes sense to cater to somewhat niche needs. Their scale will enable customization of hardware to serve many different workloads and is going to be yet another of Amazon's long-term competitive advantages.
I think that will show up in two ways. Hardware narrowly focused on certain workloads, like Google's TPUs that show really high performance, and general purpose CPUs like these Gravitons that are more cost efficent for some workloads.
I see echoes of Apple's acquisition of P.A. Semi that lead to the development of the A series CPUs. My iPhone XS beats my MacBook (early 2016) on multi-core Geekbench by 37%. (And on single core, it's only 10% slower than a 2018 MacBook Pro 15.)
If Amazon is able to have similar success in custom silicon, this will be a big deal.
I think early next year we'll test the a1 instances for some of our stateless work loads and see what the price/performance really looks like.
It does make me worry that this sort of thing will cement the dominance of large cloud providers, and we'll be left with only a handful (3?) of real competitors.
I think as we run out of gains to be had from process size reductions, the next frontier for cloud providers is in custom silicon for specific workloads. First we saw GPUs move to the cloud, then Google announced their TPUs.
Behind the scenes, Amazon's acquisition of Annapurna Labs has been paying off with their Nitro (http://www.brendangregg.com/blog/2017-11-29/aws-ec2-virtuali...) ASIC, nearly eliminating the virtualization tax, and providing a ton of impressive network capabilities and performance gains.
The Graviton, which I believe also came from the Annapurna team, is likely just the start. Though a general purpose CPU, it's getting AWS started on custom silicon. AWS seems all about providing their customers with a million different options to micro-optimize things. I think the next step will be an expanding portfolio of hardware for specific workloads.
The more scale AWS has, the more it makes sense to cater to somewhat niche needs. Their scale will enable customization of hardware to serve many different workloads and is going to be yet another of Amazon's long-term competitive advantages.
I think that will show up in two ways. Hardware narrowly focused on certain workloads, like Google's TPUs that show really high performance, and general purpose CPUs like these Gravitons that are more cost efficent for some workloads.
I see echoes of Apple's acquisition of P.A. Semi that lead to the development of the A series CPUs. My iPhone XS beats my MacBook (early 2016) on multi-core Geekbench by 37%. (And on single core, it's only 10% slower than a 2018 MacBook Pro 15.)
If Amazon is able to have similar success in custom silicon, this will be a big deal.
I think early next year we'll test the a1 instances for some of our stateless work loads and see what the price/performance really looks like.
It does make me worry that this sort of thing will cement the dominance of large cloud providers, and we'll be left with only a handful (3?) of real competitors.
This can be powerful. They don't have to build general CPU right away. Start with storage and by the time you have database boxes on ASICS designed to match your software you're already winning.
I'm surprised there's still not much effort to make FPGAs more affordable and base everything on it. With this scale it seems like it should be a win on the long run over deploying new ASICs every few years.
I'm surprised there's still not much effort to make FPGAs more affordable and base everything on it. With this scale it seems like it should be a win on the long run over deploying new ASICs every few years.
The article is non technical. For those who search for that:
https://www.theregister.co.uk/2018/11/27/amazon_aws_graviton...
"Semiconductor industry watcher David Schor shared SciMark and C-Ray benchmarks for the 16-core Graviton. In the SciMark testing, the AWS system-on-chip was twice as fast as a Raspberry Pi 3 Model B+ on Linux 4.14."
http://codepad.org/wZe5SrjI
""It does well on the Phoronix Test Suite," he said. "It does poorly benchmarking our website fully deployed on it: Nginx + PHP + MediaWiki, and everything else involved. This is your 'real world' test. All 16 cores can't match even 5 cores of our Xeon E5-2697 v4.""
"The system-on-chips use a mix of Arm's data-center-friendly Neoverse technology, and Annapurna's in-house designs. The 16 vCPU instances are arranged in four quad-core clusters with 2MB of shared L2 cache per cluster, and 32KB of L1 data cache, and 48KB of L1 instruction cache, per core. One vCPU maps to one physical core."
https://www.theregister.co.uk/2018/11/27/amazon_aws_graviton...
"Semiconductor industry watcher David Schor shared SciMark and C-Ray benchmarks for the 16-core Graviton. In the SciMark testing, the AWS system-on-chip was twice as fast as a Raspberry Pi 3 Model B+ on Linux 4.14."
http://codepad.org/wZe5SrjI
""It does well on the Phoronix Test Suite," he said. "It does poorly benchmarking our website fully deployed on it: Nginx + PHP + MediaWiki, and everything else involved. This is your 'real world' test. All 16 cores can't match even 5 cores of our Xeon E5-2697 v4.""
"The system-on-chips use a mix of Arm's data-center-friendly Neoverse technology, and Annapurna's in-house designs. The 16 vCPU instances are arranged in four quad-core clusters with 2MB of shared L2 cache per cluster, and 32KB of L1 data cache, and 48KB of L1 instruction cache, per core. One vCPU maps to one physical core."
Designing an ARM Chip from ARM blueprint and TSMC is relatively simple and cheap for Amazon. And there is enough market and hype to justify the investment as they will probably break even within 24 months. It has become obvious that Intel isn't really willing to lower price that affect margin and sales. So Amazon needs to make a statement to Intel to say they have lots of options, EPYC and ARM.
I don't want to hype Zen 2 / EPYC 2, but I do think it will be very competitive. And that is a Threat to Intel. And fundamentally, the REAL threat is neither ARM, AMD or even RSIC-V, it is TSMC.
I don't want to hype Zen 2 / EPYC 2, but I do think it will be very competitive. And that is a Threat to Intel. And fundamentally, the REAL threat is neither ARM, AMD or even RSIC-V, it is TSMC.
The article misses that months before the ARM announcement, AWS announced AMD based instances being available in m5 and r5 classes, and are cheaper than the default Intel offerings. If anything Intel might be afraid is that because the workloads that can be achieved are comparable.
Pretty funny to see “Dave Patterson” described as “a Google chip specialist”!
Why ARM though? The article touts how this is a homegrown chip, and Amazon obviously has the resources to build a truly homegrown, optimized CPU. Why use ARM instead and import all of its idiosyncrasies?
I guess I could ask the question more broadly. Why does every company that "designs its own chip" use ARM instead of designing its own ISA? How much work does it save? How much optimization does it forsake? I'm reminded of John Regehr's post on discovering the optimal instruction set: https://blog.regehr.org/archives/669
I guess I could ask the question more broadly. Why does every company that "designs its own chip" use ARM instead of designing its own ISA? How much work does it save? How much optimization does it forsake? I'm reminded of John Regehr's post on discovering the optimal instruction set: https://blog.regehr.org/archives/669
Well, guess Intel's thinking "there's always Microsoft (Azure)."
I don't think it's settled which of Azure or AWS captures the most market share in the next decade. AWS has a lot going for it but MS is coming in hard and fast on the OSS to cloud integration front. Probably would have made sense for Amazon to pickup Red Hat from IBM.
I don't think it's settled which of Azure or AWS captures the most market share in the next decade. AWS has a lot going for it but MS is coming in hard and fast on the OSS to cloud integration front. Probably would have made sense for Amazon to pickup Red Hat from IBM.
Apologies in advance for the layperson question:
It's my understanding that a lot of CPU gains come from caching. That suggests to me that there is potential performance to be gained by caching across a larger number of machines.
Is that something Amazon could do here? Somehow connect all their machines and cache in a huge space?
Maybe individual physical machines would be more like a front end for a cache space, and when I get an Amazon instance, it's actually a "virtual" CPU that pieces together instructions that are mostly already cached in various places throughout the network?
Is that even theoretically possible, or is it total fantasy?
It's my understanding that a lot of CPU gains come from caching. That suggests to me that there is potential performance to be gained by caching across a larger number of machines.
Is that something Amazon could do here? Somehow connect all their machines and cache in a huge space?
Maybe individual physical machines would be more like a front end for a cache space, and when I get an Amazon instance, it's actually a "virtual" CPU that pieces together instructions that are mostly already cached in various places throughout the network?
Is that even theoretically possible, or is it total fantasy?
It's not exactly a threat. Amazon use cortex A72 in their CPUs and there's no way they can replace most Intel CPUs with that. The performance isn't there.
(Tangential) What I'd really like to see is a competitor to Google's TPU that I can actually buy (vs rent on the cloud)
Amazon sell compute to the world. This is vertical integration and makes sense at a certain scale. That scale has to be massive, but they appear to have reached it. They may have chosen to execute sooner if they also plan to sell the chips to hardware vendors like Dell and the financials check out.
Truly lame question, is there any possibility for Amazon or other cloud providers to monitor the executed instructions and their distribution? Could this allow more optimized architectures for specific loads, or would this not bring any actual benefit?
I thought Amazon is just licensing a custom design from ARM and manufacturing from TSMC. It's a step in the right direction but it'll be a good amount of years before Amazon has their own Fab making their own chips.
To bad they are not doing this with RISC-V. Getting a large costumer and high performance implementations would been a great boost.
Since this is about vertical integration for them it would make a certain amount of sense.
Since this is about vertical integration for them it would make a certain amount of sense.
In the near future we will see more and more chips coming from Asia. Not just final silicon production, test, and packaging, but also complete hardware design. They will produce GPUs, FPGAs, and CPUs of all classes.
Look how many students in Computer Design, Digital Design, Electrical Engineering graduate every single year. Multiply that by low costs with high productivity and you will find Silicon Valley will face a very strong competitor.
Look how many students in Computer Design, Digital Design, Electrical Engineering graduate every single year. Multiply that by low costs with high productivity and you will find Silicon Valley will face a very strong competitor.
Open source has won the software, however it mostly runs on closed-source and proprietary hardware. Perhaps open hardware is the ultimate answer.
They're ignoring the fact that it's not even the same ISA, not even the same use cases or maturity for these ISAs, and that for them to actually produce a x86 chip, the only way to do that is to licence either AMD or Intel as IIRC they're the only ones to hold the x86 patent.
That or change the way a majority of the software stack was written for the past 15 years.
That or change the way a majority of the software stack was written for the past 15 years.
As much as it is easy to license from ARM and use TSM as a foundry, it does not make much economic sense.
Look at Google's purchase of Motorola. They ended up selling Motorola to Lenova.
The hardware industry and specifically the chip industry is very specialized. If your code competency is not chip making, it really does not make sense to try to enter the industry.
Look at Google's purchase of Motorola. They ended up selling Motorola to Lenova.
The hardware industry and specifically the chip industry is very specialized. If your code competency is not chip making, it really does not make sense to try to enter the industry.
You might compare this with Google's partnership with IBM, using POWER chips in their data centers.
https://www.fool.com/investing/2018/03/22/googles-data-cente...
https://www.fool.com/investing/2018/03/22/googles-data-cente...
I wanted to try to use an AWS ARM server as a bastion host running Wireguard. But a t2.nano was cheaper.
Anyone get it working? (I haven’t tried)
Anyone get it working? (I haven’t tried)
How does competition work with regard to trade secrets like chip design etc? If I hire the top dogs at a chip maker and they design a similar chip for my company from memory, how is this prevented from happening?
And Amazon is not alone. Other potential candidates to add to list include AMD, Apple and Qualcomm. Maybe a few others that don't come to mind right away.
Not only Amazon's but Apple's, Jiāngnán's as well. I think Intel really should crank up innovation if they would like to stay competitive.
I could almost hear Larry Ellison snickering in the background as I read this. I wonder how much Amazon pays Oracle, anyway? I think in '16 is was around $60M (according to Ellison: https://www.forbes.com/sites/bobevans1/2017/12/12/oracles-la... )
Yawn, the amount of publicity ARM gets for free for the past decade to enter the data center is perplexing. The economy of these designs are not good versus the competition full stop.
There are three chips that work well in the data center: EPYC, Xeon, and POWER. All of these are billion dollar designs. Nothing about the ARM ecosystem supports designing to these same constraints or spending that amount of money to enter this space seriously.
There are three chips that work well in the data center: EPYC, Xeon, and POWER. All of these are billion dollar designs. Nothing about the ARM ecosystem supports designing to these same constraints or spending that amount of money to enter this space seriously.
This is oversimplification.
Binary compatibility, of millions of third party libraries out there, is the biggest hurdle.
How many years did it take to switch from Python 2.7 to Python 3 ?
AWS lockin is a bigger threat.
I prefer to keep my binaries cloud agnostic and X86 and X64 compatible rather than AWS only code or even ARM binary.
By the way what happened to Amazon phones ?
Binary compatibility, of millions of third party libraries out there, is the biggest hurdle.
How many years did it take to switch from Python 2.7 to Python 3 ?
AWS lockin is a bigger threat.
I prefer to keep my binaries cloud agnostic and X86 and X64 compatible rather than AWS only code or even ARM binary.
By the way what happened to Amazon phones ?
Um, and AMD.
For example, Apple has been able to own nearly their entire iDevice stack from manufacturing to silicon to firmware to OS to ecosystem. They have very little incentive to interoperate with external organizations via open standards because they own many of the pieces which need to be interoperable. Thus, they can force users and developers to use their tooling, the applications they approve of, dictate what code will run on their platform, and how easily you can inspect, modify, or repair their products.
This is all to say, it is easy to imagine a future where all performance-competitive technology is entirely built upon proprietary, locked down stacks – AND – it will be at a level of complexity and specificity that independent people simply cannot gain access to the ability to create competitive solutions. It could be back to the days of the mainframe, but worse, where only the corporations who create the technology will have access to the knowledge and tools to experiment and innovate on a competitive scale.
Amazon wants developers to build solutions entirely on top of their platform using closed source core components. They also want to control the silicon their own platform runs on. In 10 years, what else will they own, how much will this effect the freedom offered by competitors, and what impact will it all have on our freedom to build cool shit?