48 Core ARM-64 processors from Cavium to challenge Xeons?(eetimes.com)
eetimes.com
48 Core ARM-64 processors from Cavium to challenge Xeons?
http://www.eetimes.com/document.asp?doc_id=1322565
8 comments
Wonder why Cavium jumped from MIPS to ARM for this? At least, all the Cavium kit I have is MIPS based.
The biggest road bump for 48+ core SoC will be parallel programming. Because in truth nobody is (or at least doesn't want too). If you happened to be an early adopter of the SPARC T5 Niagra you likely noticed most of your utils ran very slowly.
I just remember generating SSL keys at a speed that my pentium would laugh at, because SSL didn't supporting threading itself out to all 512 cores.
I just remember generating SSL keys at a speed that my pentium would laugh at, because SSL didn't supporting threading itself out to all 512 cores.
At SUN I was working with 256-thread SPARCs regularly and it was fairly simple to program using Java, except for some oddities in Solaris regarding networking stack. So if you need fast web servers, app servers or ETL, these chips can be pretty awesome. SPARC's threads were slower than ARM processors in mainstream phones nowadays, yet they were pretty good in handling web load.
I would buy these chips in an instant, similarly to Parallela's 64-core ARM.
The competition would be Xeon Phi - I assume that is much more expensive and the memory is too limited (8GB).
I would buy these chips in an instant, similarly to Parallela's 64-core ARM.
The competition would be Xeon Phi - I assume that is much more expensive and the memory is too limited (8GB).
> Parallela's 64-core ARM
Parallela is not a 64-core ARM, it's a 2-core ARM with a bunch of co-processors which have to be programmed specially. The co-processors have little local memory and the memory model is very weak, there are no memory barriers precluding the use of general-purpose code.
Parallela is not a 64-core ARM, it's a 2-core ARM with a bunch of co-processors which have to be programmed specially. The co-processors have little local memory and the memory model is very weak, there are no memory barriers precluding the use of general-purpose code.
These chips are designed for networking equipment, not general purpose code. I don't understand how come the article fails to make this explicitly clear.
Networking code has no problem with this high core count mostly because it was already written to work on already high-core count MIPS systems. For example, the previous MIPS64 offerings from Cavium pack 48 cores on a chip...
Networking code has no problem with this high core count mostly because it was already written to work on already high-core count MIPS systems. For example, the previous MIPS64 offerings from Cavium pack 48 cores on a chip...
Directly from the article: "Cavium plans four families of Thunder chips, targeting storage, networking, and security systems in addition to servers. Each family will use a different set of I/O and accelerator blocks from Cavium's existing silicon library."
So no, these are not just designed for networking equipment, they are planned to target 4 different areas.
So no, these are not just designed for networking equipment, they are planned to target 4 different areas.
If I question the legitimacy of the article, you can't quote the article in as an argument for the article. See my other reponse and windexh8er's.
Why do you claim that these chips are designed for networking? The article isn't just omitting the target market like you imply, but is very clear about Cavium targeting servers rather than just networking appliances.
(I'm a lot pessimistic than most of the people in this thread. Network processors haven't really made a lot of sense compared to Xeons even for networking applications for a few years. Why would they do any better in the general purpose market?)
(I'm a lot pessimistic than most of the people in this thread. Network processors haven't really made a lot of sense compared to Xeons even for networking applications for a few years. Why would they do any better in the general purpose market?)
> Why do you claim that these chips are designed for networking?
Because this is Cavium's niche and biggest asset, and all their customers are doing networking. Of course they might want to expand their business, but they have always done that, really, and they only had success in networking. The special features of this chip (things that are Cavium-specific and not ARM64 specific) are features that make a lot of sense for networking equipment, and are not very important for general purpose servers. These special features are also present in their existing MIPS64 line.
> Network processors haven't really made a lot of sense compared to Xeons even for networking applications for a few years.
Really? Where's that 100GbE Xeon switch?
Because this is Cavium's niche and biggest asset, and all their customers are doing networking. Of course they might want to expand their business, but they have always done that, really, and they only had success in networking. The special features of this chip (things that are Cavium-specific and not ARM64 specific) are features that make a lot of sense for networking equipment, and are not very important for general purpose servers. These special features are also present in their existing MIPS64 line.
> Network processors haven't really made a lot of sense compared to Xeons even for networking applications for a few years.
Really? Where's that 100GbE Xeon switch?
> Because this is Cavium's niche and biggest asset, and all their customers are doing networking.
That's their current niche. They're trying to get out of that niche, as the article clearly shows. (Let's be clear here -- the article might be a PR fluff piece, but it hasn't been written in an ignorance of the position of Cavium in networking). If they try to market this stuff for general purpose servers, they need to have an answer to the problem of parallelizing general purpose workloads. I don't see why this is such an offensive thing to point out.
I work at a company making networking equipment (mainly for telecoms), targeting the 20Gbps range at the top end, with Xeons. We could scale much higher for trivial workloads like switching, but the bottleneck is in actual processing where the wimpy MIPS cores aren't particularly tempting. The offerings of companies like Cavium have really not looked compelling compared to standard Xeons during the last 4 years or so.
Maybe if we'd started before that, we'd have started with a network processor and would naturally stick to it. But many of the more established telecom suppliers we've interacted with are likewise doing significant amounts of stuff with Xeons (standard hardware, or ATCA blades at the most exotic end). The only Cavium card I remember anyone using was doing 10Gbps of traffic that was so trivial, that even the low-spec 1U Xeon server the card was installed in could have handled it with the proper userspace NIC drivers.
That's their current niche. They're trying to get out of that niche, as the article clearly shows. (Let's be clear here -- the article might be a PR fluff piece, but it hasn't been written in an ignorance of the position of Cavium in networking). If they try to market this stuff for general purpose servers, they need to have an answer to the problem of parallelizing general purpose workloads. I don't see why this is such an offensive thing to point out.
I work at a company making networking equipment (mainly for telecoms), targeting the 20Gbps range at the top end, with Xeons. We could scale much higher for trivial workloads like switching, but the bottleneck is in actual processing where the wimpy MIPS cores aren't particularly tempting. The offerings of companies like Cavium have really not looked compelling compared to standard Xeons during the last 4 years or so.
Maybe if we'd started before that, we'd have started with a network processor and would naturally stick to it. But many of the more established telecom suppliers we've interacted with are likewise doing significant amounts of stuff with Xeons (standard hardware, or ATCA blades at the most exotic end). The only Cavium card I remember anyone using was doing 10Gbps of traffic that was so trivial, that even the low-spec 1U Xeon server the card was installed in could have handled it with the proper userspace NIC drivers.
They are not just designed for networking, but the bulk of their customers are: A10, F5, Sonicwall, Cisco, Juniper, Palo Alto Networks (http://www.cavium.com/winning_products.html)
"...haven't really made a lot of sense..." - You are completely wrong here. Most data center grade networking equipment does not run Xeon and the products that do are generally significantly less performant. Especially when it comes to security or packet inspection.
"...haven't really made a lot of sense..." - You are completely wrong here. Most data center grade networking equipment does not run Xeon and the products that do are generally significantly less performant. Especially when it comes to security or packet inspection.
If you narrow the statement down to "the highest performance network gear don't use Xeons", I suppose you get to be right. Almost all of my data-center security and packet inspection devices have a Xeon or two in them. The Cisco ASAs, Fireeye APT probes, Imperva WAFs, F5 load balancers, Sourcefire IDS. A couple of them also have packet processing support hardware (Endace cards, mostly), but that's mostly for taking the packet processing off the Xeons and letting them do just the inspection. But hey, we're only doing a couple of gig sustained, so we aren't the cool kids.
I assume these kinds of chips are ideal for naturally parallel workloads like web servers. I wonder what the mWh/request looks like between one of these and a normal x86 server.
Niagara was too much ahead of its time. And a crypto accelerator would have been helpful (even if shared by a group of cores). I imagine memory bandwidth will become a problem as soon as we start designing our software to branch out enough threads to feed all the 48 mouths of this chip, but, still, this is how things will be in the future so it'd be wise to start designing for it right now.
Niagara included a crypto accelerator. Its successor chips had fancier ones.
https://blogs.oracle.com/ningsun/entry/cryptography_accelera...
https://blogs.oracle.com/ningsun/entry/cryptography_accelera...
Errr, some of us have been in the business of writing parallelised code for quite some time now.
A lot of enterprise* server stuff is written to take advantage of multiple cores/hardware threads and has been for a while.
(*yeah, I hate that word)
A lot of enterprise* server stuff is written to take advantage of multiple cores/hardware threads and has been for a while.
(*yeah, I hate that word)
I've written a fairly wide variety of multithreaded code in the past ten years or so, and I think the only real trouble I've had is with algorithms that work step by step, each step dependent on the previous results. That's effectively impossible to parallelize without fundamentally changing the algorithm.
Otherwise, even if you have large chunks of shared memory, it's usually possible to work around that without locks.
Otherwise, even if you have large chunks of shared memory, it's usually possible to work around that without locks.
> The biggest road bump for 48+ core SoC will be parallel programming.
And if you're okay with parallel programming (which, let's face it, is the only way to do high performance computing), then you'll possibly be hit by memory bandwidth or cache size. Because if you have 48 cores on a single lump of silicon, they won't have very large dedicated caches each. However, if you have an appropriate job that will scale out to all the cores with a small working set then the performance of this thing could be rather interesting.
And if you're okay with parallel programming (which, let's face it, is the only way to do high performance computing), then you'll possibly be hit by memory bandwidth or cache size. Because if you have 48 cores on a single lump of silicon, they won't have very large dedicated caches each. However, if you have an appropriate job that will scale out to all the cores with a small working set then the performance of this thing could be rather interesting.
[deleted](1)
When you have tens of CPU cores, you really do start to run up against Amdahl's Law and soon don't get the speedup that you might expect.
e.g. let's say 90% of your task can be perfectly parallelised. The remaining 10% of the time will not be improved. So you won't ever get more than a 10x speedup no matter how many tens or hundreds of extra CPU cores you have...
e.g. let's say 90% of your task can be perfectly parallelised. The remaining 10% of the time will not be improved. So you won't ever get more than a 10x speedup no matter how many tens or hundreds of extra CPU cores you have...
Well, you're thinking in terms of parallelising a single task. For a server, you're talking about a large number of concurrent tasks (users), so more cores are better in that case.
Actually, this holds only up to certain extent. Synchronization, locking, memory access patterns and context-switching from many threads with accompanied cache thrashing can bring the effective parallelization to very small levels.
I remember from benchmarking a high performance transactional messaging system in Java I was working on seeing how it hit a flat line at some point where it didn't scale at all due to CPU/memory contention (this depended on maximal number of threads allowed). We had to look at alternative designs such as 0MQ to see how they overcame it - basically they put one working/client thread per core to achieve their 1M messages/s. We were using 10s of thousands of threads in our middleware when necessary, and that was faster than async NIO in Java 6 at that time.
I remember from benchmarking a high performance transactional messaging system in Java I was working on seeing how it hit a flat line at some point where it didn't scale at all due to CPU/memory contention (this depended on maximal number of threads allowed). We had to look at alternative designs such as 0MQ to see how they overcame it - basically they put one working/client thread per core to achieve their 1M messages/s. We were using 10s of thousands of threads in our middleware when necessary, and that was faster than async NIO in Java 6 at that time.
That's basically a truism. If you have parallelisable tasks, then they will parallelise. No surprise there.
I was trying to make the point that as we step up the number of cores, the actual gain is less and less.
I was trying to make the point that as we step up the number of cores, the actual gain is less and less.
You're missing the point. I think everyone knows what you're saying is true for a mobile phone or even a desktop PC. Where it's NOT true is on servers, where tasks are "parallelized" since you have millions of concurrent users, so whether you have 1 million 8-core servers, or 125,000 64-core servers it doesn't matter much, because it's pretty much the same thing.
But from a whole datacenter perspective, I believe having servers with more cores is more efficient than having more servers with fewer cores, since data travels faster between more of the cores.
But from a whole datacenter perspective, I believe having servers with more cores is more efficient than having more servers with fewer cores, since data travels faster between more of the cores.
This whole thread is talking past each other.
Servers absolutely can have sequential dependencies, especially if there is a shared resource (eg, databases, IO, etc). If your application can make full use of all your cores without any inter-dependencies, great -- but that is not always the case.
Servers absolutely can have sequential dependencies, especially if there is a shared resource (eg, databases, IO, etc). If your application can make full use of all your cores without any inter-dependencies, great -- but that is not always the case.
"But from a whole datacenter perspective, I believe having servers with more cores is more efficient than having more servers with fewer cores, since data travels faster between more of the cores."
How are you looking at this?
From limited CompE background, my understanding was that moving data around core -> core burned a significant amount of power, even on the same die. (Using the comparison of one core vs two equally performing but separate cores)
How are you looking at this?
From limited CompE background, my understanding was that moving data around core -> core burned a significant amount of power, even on the same die. (Using the comparison of one core vs two equally performing but separate cores)
Again, as it was pointed out earlier, no. If you have 48 runnable threads from 48 different tasks running on your machine, you'll get pretty much linear performance increases (memory bottleneck excepted). It's not a processor designed to run Office or Firefox, but it may be very interesting on a server or as a competitor for the Xeon Phi.
BTW, considering even my humble laptop can do 8 threads at once, it's about time we started building office suites and browsers to take advantage of these processors.
BTW, considering even my humble laptop can do 8 threads at once, it's about time we started building office suites and browsers to take advantage of these processors.
> If you have 48 runnable threads from 48 different tasks running on your machine, you'll get pretty much linear performance increases
Only if the tasks are trivially parallelizable. Any sequential dependencies among those tasks limit the effectiveness of parallelization. This is exactly what Amdahl's Law says.
> my humble laptop can do 8 threads at once, it's about time we started building office suites and browsers to take advantage of these processors.
Multiple browser tabs or open office documents are trivially parallelizable (eg, processes). The work within those tabs or documents is not linearly parallelizable -- running JS/macros, handling input, rendering, etc. Existing browser/office suites already do a pretty good job of splitting these tasks up, but they can't be split up arbitrarily into $core_count chunks.
Only if the tasks are trivially parallelizable. Any sequential dependencies among those tasks limit the effectiveness of parallelization. This is exactly what Amdahl's Law says.
> my humble laptop can do 8 threads at once, it's about time we started building office suites and browsers to take advantage of these processors.
Multiple browser tabs or open office documents are trivially parallelizable (eg, processes). The work within those tabs or documents is not linearly parallelizable -- running JS/macros, handling input, rendering, etc. Existing browser/office suites already do a pretty good job of splitting these tasks up, but they can't be split up arbitrarily into $core_count chunks.
> Only if the tasks are trivially parallelizable
Which is a very common scenario on web and application servers, specially if you design your web applications to be scalable.
> Multiple browser tabs or open office documents are trivially parallelizable
They would be if I could pay attention to more than one tab or document at any given time. In reality, I spend most of my time with full-screen apps and there is little point to keep other tabs or documents running.
Which is a very common scenario on web and application servers, specially if you design your web applications to be scalable.
> Multiple browser tabs or open office documents are trivially parallelizable
They would be if I could pay attention to more than one tab or document at any given time. In reality, I spend most of my time with full-screen apps and there is little point to keep other tabs or documents running.
> Which is a very common scenario on web and application servers, specially if you design your web applications to be scalable.
Doesn't matter. You can (and probably will) still hit horrible bottlenecks in the network stack. Another flashback to developing on the 64+ Niagara cores. We were running an embarassingly scalable web application - loads of independent concurrent requests - but the Solaris OS just couldn't keep up.
Has your NIC driver been optimised to scale for tens/hundreds of CPUs, I wonder?
Doesn't matter. You can (and probably will) still hit horrible bottlenecks in the network stack. Another flashback to developing on the 64+ Niagara cores. We were running an embarassingly scalable web application - loads of independent concurrent requests - but the Solaris OS just couldn't keep up.
Has your NIC driver been optimised to scale for tens/hundreds of CPUs, I wonder?
There is a lots of interesting work on building office suites and browsers that can take advantage of parallelism. But the general consensus is that we need to invent new lower level tools first. See Rust + Servo and MSFT + M#
[deleted]
I don't know why people are down-voting you. Amdahl's Law is absolutely applicable here.
Having said that, I've been using a similar aarch64 machine for a few months (sorry, can't name it -- NDA), and single thread performance is just fine. I think people once used an RPi and now think every ARM has terrible performance. 64 bit ARM is a completely different architecture and far better performing.
Having said that, I've been using a similar aarch64 machine for a few months (sorry, can't name it -- NDA), and single thread performance is just fine. I think people once used an RPi and now think every ARM has terrible performance. 64 bit ARM is a completely different architecture and far better performing.
I'm a little biased through once having developed on Sun's Niagara processors. Early versions of these had pretty terrible single core (thread? I forget their exact terminology) performance. The theoretical power of the processor was huge but any time you hit a single-threaded program you felt it. ssh logins were slow, for example!
It shouldn't be surprising that Niagra is slow on single threads. The whole point of the design was to support parallel, non-interactive tasks (e.g. databases) at the expense of non-threaded workloads. It was a design trade-off that Sun was very up front about. Even N1 & N2 gave remarkable price/performance numbers when used for the tasks it was designed for.
I've been compiling libguestfs, libvirt, the kernel, qemu, and other programs, and using the aarch64 machine interactively, and it certainly doesn't feel slow.
I have to be careful about exactly what I say here because the NDA prevents me from doing any kind of performance benchmarking (for good reasons: this is pre-release hardware and the performance doesn't reflect what the final hardware will be capable of), but ... performance is fine, I've had no problem with either single-threaded tasks or parallel compiles. Given I'm using it as a remote machine over ssh, it doesn't "feel" any different from an x86-64 server.
I have to be careful about exactly what I say here because the NDA prevents me from doing any kind of performance benchmarking (for good reasons: this is pre-release hardware and the performance doesn't reflect what the final hardware will be capable of), but ... performance is fine, I've had no problem with either single-threaded tasks or parallel compiles. Given I'm using it as a remote machine over ssh, it doesn't "feel" any different from an x86-64 server.
n.b. I wasn't very clear in my comment; my experience was on the original Niagara 1 & 2, this was about 7-8 years ago, I'd guess. Hopefully things have improved since :)
[deleted](1)
For the optimist, Amdahl's law is just a restatement of Zeno's halfway paradox ;)
But if you have only 10 processors, you'll only get a 5× speedup. 36 processors will get you an 8× speedup. (After that point it's really diminishing returns; you'd need 81 processors to get a 9× speedup.)
Cavium announced a 48 x 64-bit chip running 2.5GHz almost two and a half years ago: the Octeon III.
http://www.cavium.com/newsevents_Cavium_Unveils_48-core_OCTE...
http://www.cavium.com/newsevents_Cavium_Unveils_48-core_OCTE...
Worth noting that Intel already has a many weaker core design -- the Xeon Phi. While it exists as an add-in "co-processor" right now, in 2015 it will become a stand-alone platform via Knights Landing.
Xeon Phi seems to be targeting two markets at once - many-core (x64) server computing and GPGPU with their 512-bit SIMD. For general server the memory limitation to 8GB is too low, which is however very competitive in GPGPU market.
Their new models are 16GB, but you are exactly right that the model is very similar to the GPGPU (less memory, but what it does have is incredibly fast), albeit fewer but higher performance cores than something like a Tesla.
Knights Landing is when the model becomes a server unto itself, with 384GB of addressable memory, albeit still maintaining the ultra-fast 16GB of local memory per CPU.
Knights Landing is when the model becomes a server unto itself, with 384GB of addressable memory, albeit still maintaining the ultra-fast 16GB of local memory per CPU.
Exciting times ahead ;-)
Also, there is a difference in required memory architecture when it comes to optimizing CPU thread performance vs SIMD - CPU is more sensitive to latency, SIMD to bandwidth. You usually have to pick the memory type (e.g. DDR3 vs GDDR5) that optimizes one of those and that would tell you what market you target.
I am not sure how AMD handles their HSA - they seem to have discontinued plans for GDDR5 so their Fusion GPU is going to be bandwidth-starved.
Also, there is a difference in required memory architecture when it comes to optimizing CPU thread performance vs SIMD - CPU is more sensitive to latency, SIMD to bandwidth. You usually have to pick the memory type (e.g. DDR3 vs GDDR5) that optimizes one of those and that would tell you what market you target.
I am not sure how AMD handles their HSA - they seem to have discontinued plans for GDDR5 so their Fusion GPU is going to be bandwidth-starved.
The chip is vaporware. They've been talking about it for two years, but to quote the article, "Cavium has yet to share performance figures for the chip, which is not yet in first silicon."
I will be very excited to see some hard numbers when (if) this thing is ever released.
I will be very excited to see some hard numbers when (if) this thing is ever released.
http://www.ubuntu.com/download/server/arm
Links to "Project Thunder SDK":
http://www.cavium.com/thundersdk_access_application.html