Apple Announces Apple Silicon Mac Pro Powered by M2 Ultra(9to5mac.com)
9to5mac.com
Apple Announces Apple Silicon Mac Pro Powered by M2 Ultra
https://9to5mac.com/2023/06/05/apple-announces-apple-silicon-mac-pro-powered-by-m2-ultra/
15 comments
Intel Mac Pros max out at 1.5TB of RAM, and these are max 192GB. Seems like a large drop. I've heard that Apple Silicon is more memory-efficient, maybe because of efficiencies from the tight integration with macOS. Is that enough to make up for the difference?
Apple Silicon isn't more efficient in terms of memory capacity, there's really no way to do that unless you go for some crazy memory architecture that nobody wants to program for.
Apple Silicon has a lot more memory bandwidth though, maybe that's what you heard?
Apple Silicon has a lot more memory bandwidth though, maybe that's what you heard?
Apple silicon has better support (hardware instructions) in saving / restoring memory. Helps with swapping "unused" things to the background and back.
What does a hardware instruction for swapping do?
Compression/encryption/hashing acceleration.
A reverse case being that x86 has a memcpy instruction and ARM didn't until recently.
A reverse case being that x86 has a memcpy instruction and ARM didn't until recently.
I think the parent comment meant there isn't a way to make memory more efficient when it comes to app data. And code takes up so much less memory that x86 vs ARM code size isn't going to make a measurable difference.
In any case, 1.5TB is only going to help a few app niches. It is hard to even reliably predict which kinds. Even, say, editing 8K video could be well-enough accommodated with much less memory if file system bandwidth is high enough.
In any case, 1.5TB is only going to help a few app niches. It is hard to even reliably predict which kinds. Even, say, editing 8K video could be well-enough accommodated with much less memory if file system bandwidth is high enough.
Unlikely. 1.5TB of RAM is the kind of crazy amount that you'd use if you were handling some immense workload and you need to hold a huge amount of data in memory. You couldn't just cut the amount of available RAM down to a fraction and expect things to continue to work.
Not only that, it's shared memory with the on-chip GPU. Isn't this a big downgrade in both memory capacity and GPU capability?
One thing I was surprised to not see any comments from Apple on is whether GPUs will still be supported in the PCIe slots. It'd require work from either AMD or Nvidia to get an ARM64 macOS driver so maybe that's the holdup there. It's hard to compete with max builds which have many GPUs in raw power without the ability to go beyond the built in GPU. That said, 192 GB of 800 GB/s VRAM is pretty crazy at the price point.
It could be a positive for GPU capability - such as models that would be constrained on GPUs of say 24 GB.
This will be a positive development for LLMs like Llama.
This will be a positive development for LLMs like Llama.
Yes, you only need faith.
This looks like a product meant to serve a very niche market.
During lockdown I had the Intel Mac pro in my home office and my electric bill doubled with that thing on. It also pushed out so much heat, but at least the chassis design was whisper quiet. I don't know, by the end of it all, I was so happy to get that heavy thing out of my house.
Sure, it was definitely the fastest thing to enable the tools I needed back in 2020 and I was extremely happy to have it. It also became obsolete once the M1 had native tooling ready a couple of years later. Soooo yea.
I guess with all that excessive space in that chassis they can overclock the system and squeeze out every bit of performance. The PCI expansion could be good for production studios that need it. At that cost level though, the system will have to outperform anything you can find on the windows side.
During lockdown I had the Intel Mac pro in my home office and my electric bill doubled with that thing on. It also pushed out so much heat, but at least the chassis design was whisper quiet. I don't know, by the end of it all, I was so happy to get that heavy thing out of my house.
Sure, it was definitely the fastest thing to enable the tools I needed back in 2020 and I was extremely happy to have it. It also became obsolete once the M1 had native tooling ready a couple of years later. Soooo yea.
I guess with all that excessive space in that chassis they can overclock the system and squeeze out every bit of performance. The PCI expansion could be good for production studios that need it. At that cost level though, the system will have to outperform anything you can find on the windows side.
> This looks like a product meant to serve a very niche market.
The Mac Pro always was a niche-market device for professionals with big compute needs in a workstation and money to burn.
> At that cost level though, the system will have to outperform anything you can find on the windows side.
You can count on that. It's Apple. Not only is the M2 designed to outperform any other desktop-class chip out there, but they've also bought out almost all of TSMC's entire 3nm process capacity, shutting Intel, AMD, and other ARM vendors out of the highest tier of CPU performance entirely.
The Mac Pro always was a niche-market device for professionals with big compute needs in a workstation and money to burn.
> At that cost level though, the system will have to outperform anything you can find on the windows side.
You can count on that. It's Apple. Not only is the M2 designed to outperform any other desktop-class chip out there, but they've also bought out almost all of TSMC's entire 3nm process capacity, shutting Intel, AMD, and other ARM vendors out of the highest tier of CPU performance entirely.
I don't know what kind of work you do, but in some ways it is a niche market. It's a professional workstation that is usually meant for compute intensive tasks like scientific computing, 3d rendering and animation, etc. It's certainly overkill for most software engineers, for example.
Is it overkill? I use all 32 of my CPU cores when building C++ or Go, and while fuzz testing. I'd take 64 if I could get them. There is no reason why I'd ever want fewer cores, even if it doesn't make Chrome or Emacs any faster. Latency is crucial during the edit/compile/debug cycle; making compile as close to 0 as possible is always a productivity win.
Finally if you think Intel power saving is bad, wait until you see AMD's. My Threadripper system idles at 270W. This is exceedingly wasteful. Before upgrading to a Threadripper I had an i7-6950X which idled under 100W. The difference is quite noticeable.
Finally if you think Intel power saving is bad, wait until you see AMD's. My Threadripper system idles at 270W. This is exceedingly wasteful. Before upgrading to a Threadripper I had an i7-6950X which idled under 100W. The difference is quite noticeable.
I too subscribe to the Tim "The Toolman" Taylor ideology of more power. My only point was that most developers don't "need" a Mac Pro, even if they neeeeed it.
I’m going to be using that one. Thank you.
> During lockdown I had the Intel Mac pro in my home office and my electric bill doubled with that thing on.
Total lifetime electricity use of a professional laptop is something like $30, FWIW.
Total lifetime electricity use of a professional laptop is something like $30, FWIW.
Mac Pro is a desktop, not a laptop.
Indeed, I read "macbook". Still, going from 30W average over 40 hours a week to 60W continuous[1] doesn't change the calculus that much, and if anything a 3Y amortization is probably too short for a workstation. Really my point was that this was a bit too far down the hyperbole hole. There are reasons to like new efficient hardware, but "electricity bills" aren't one of them.
[1] Measured average consumption of my 64-core Zen 2 box, which is surely going to be more than that Mac.
[1] Measured average consumption of my 64-core Zen 2 box, which is surely going to be more than that Mac.
I googled “intel mac pro electricity usage” and this was the top hit: https://support.apple.com/en-us/HT201796
The first (and highest) entry at 302W at idle would consume ~50% our household’s monthly usage. It’s at least the right order of magnitude.
The first (and highest) entry at 302W at idle would consume ~50% our household’s monthly usage. It’s at least the right order of magnitude.
[deleted]
Without Nvidia drivers, I don't see the point in having a $7k Mac Pro.
It does feel a little like a product they made because people kept asking for it... As to who actually needs it, I have no idea.
I was hoping it would have expandable memory. Not because I need the Mac Pro, or insane amounts of memory, I just wanted to see how they'd deal with it.
Technically I suppose Nvidia could write the drivers, not sure if they can justify the cost though. Seems like most people would just get a Linux box.
I was hoping it would have expandable memory. Not because I need the Mac Pro, or insane amounts of memory, I just wanted to see how they'd deal with it.
Technically I suppose Nvidia could write the drivers, not sure if they can justify the cost though. Seems like most people would just get a Linux box.
"This completes the transition from Intel to Apple Silicon." --Apple presenter talking about the new M2 Ultra Mac Pro
I wonder how much of the mainboard is essentially blank? (Just a plane or essentially traceless) With unified memory, that certainly simplifies the “desktop” format considerably.
I'm a bit naive when it comes to how interconnects like PCI work, but other than OS level drivers being available, is there any technical reason why CPU architecture could restrict PCI device support?
That is to say, if the drivers were signed and supported on macOS, could you theoretically take any modern PCIe device from a PC and run it in an ARM-based PC, Mac, whatever?
That is to say, if the drivers were signed and supported on macOS, could you theoretically take any modern PCIe device from a PC and run it in an ARM-based PC, Mac, whatever?
Yes, apart from GPUs which have limitations regarding cache coherency that have historically been a problem with some ARM architectures.
There's also the I/O ports (PCI has three address spaces: configuration, memory, and I/O, with the I/O space being a legacy x86 thing which does not exist on ARM and most other CPU architectures), which some cards might require; and the PCI address window (BAR space), which might be too small for some cards (usually GPUs; see for instance https://www.jeffgeerling.com/blog/2020/external-gpus-and-ras... for a couple of examples where it was too small.
I did not expect them to recycle the old Mac Pro enclosure. I get it, if it works it works, it just looks hilariously empty and unwieldy compared with the Mac Studio that could house the same chip (Yes I know the expansion slots need room).
That was my thought as well. I still prefer the original cheese grater :-/
> The Mac Pro features eight Thunderbolt ports and six PCI slots for modular expansion. The base model config Mac Pro starts at $6999.
I'm guessing you can't use those PCI slots to add more ram or an external graphics card (maybe I'm wrong?!)
If not, what are they for?
I'm guessing you can't use those PCI slots to add more ram or an external graphics card (maybe I'm wrong?!)
If not, what are they for?
I’d assume you can add any expansion card with drivers for Apple Silicon macos. Now that this config exists I guess some companies will make the necessary drivers for their products.
CXL ram expansion cards actually are a thing!
It is no free lunch though, it will very very slow even if they offer it.
It is no free lunch though, it will very very slow even if they offer it.
I think CXL devices in general will become more and more common given that other players on the hardware side are now supporting CXL 1.1 / 2.0 protocols (Sapphire Rapids / 4th Gen EPYC[2]) and it's a way to scale memory past the limitations of a single machine and the number of memory slots / DIMM slots.
For now it's PCIe 5.0, but given that PCIe 6.0[2] will come around with PAM4 signalling, FEC and FLIT mode on CXL 3.0 spec it may be not be as bad or slow with this future configuration.
[1] https://www.amd.com/en/press-releases/2022-11-10-offering-un... [2] https://www.rambus.com/blogs/pcie-6/
For now it's PCIe 5.0, but given that PCIe 6.0[2] will come around with PAM4 signalling, FEC and FLIT mode on CXL 3.0 spec it may be not be as bad or slow with this future configuration.
[1] https://www.amd.com/en/press-releases/2022-11-10-offering-un... [2] https://www.rambus.com/blogs/pcie-6/
Given that macOS has memory compression, it already has two "classes" of memory pages in a storage-hierarchy relationship, with active promotion/eviction. Just mount the CXL "external memory" to a different part of the address space, and default to writing compressed/cold pages there.
[deleted]
So Apple wasn't able to increase performance (or even ram capacity) beyond the Ultra series chips. Thats disappointing.
Particularly given they took an entire extra year to deliver it after the original Ultra release in March 2022.
Particularly given they took an entire extra year to deliver it after the original Ultra release in March 2022.
Its already known that the M2 line is more of a refresh than a big step over the M1.
Sure, but you'd think that an extra years delay would deliver more than just a Mac Studio chip in a new form factor.
Not really. Apple doesn't change form factors much - the Mac Mini has tons of empty space but Apple hasn't redesigned it at all from the Intel era.
Also, this is Pro equipment. Form factor changes cause frustration, not joy.
Also, this is Pro equipment. Form factor changes cause frustration, not joy.
Also frustrating is that until this morning Apple would happily charge you full new retail (and margins) on a 2019 CPU.
I love my cheesegrater, and it's great for the workloads. But I can't see any real reason to go to this Mac Pro versus the Studio.
I love my cheesegrater, and it's great for the workloads. But I can't see any real reason to go to this Mac Pro versus the Studio.
> So Apple wasn't able to increase performance (or even ram capacity) beyond the Ultra series chips. Thats disappointing.
Did we watch the same presentation? The M1 Ultra maxed out at 128GB of RAM, the M2 Ultra can handle 192GB. The CPU cores go from 20 to 24. GPU cores go from 64 to 76.
Did we watch the same presentation? The M1 Ultra maxed out at 128GB of RAM, the M2 Ultra can handle 192GB. The CPU cores go from 20 to 24. GPU cores go from 64 to 76.
These are minor improvements over the M1 series in line with the other released M2 chips. For a 1 year delay to the original transition timeline and skipping the first generation it's reasonable to expect something more than the high end Ultra chip in a new form factor.
How about a 4x, 8x and 16x M2 Max chiplet design which would have truly been next level...
How about a 4x, 8x and 16x M2 Max chiplet design which would have truly been next level...
You said:
> So Apple wasn't able to increase performance (or even ram capacity)
You were simply wrong. Why not just admit that?
> So Apple wasn't able to increase performance (or even ram capacity)
You were simply wrong. Why not just admit that?
I never mentioned a comparison with M1 Ultra. I was referring to a step above the Ultra series. From the moment the M2 chip was announced with increased RAM, CPU and graphics cores it was obvious all these benefits would come to Pro/Max and Ultra series.
The point is that Ultra remains the top of the line chip. There's no Mac Pro exclusive chip that lives on another level of performance.
The point is that Ultra remains the top of the line chip. There's no Mac Pro exclusive chip that lives on another level of performance.
This is true, but I also don’t see why it matters. In practice most users who want the highest performance level will choose the Mac Studio form factor, which is far more convenient.
The Pro chassis is an alternative enclosure for use cases that require PCI expansion cards for niche applications.
The Pro chassis is an alternative enclosure for use cases that require PCI expansion cards for niche applications.
In the pre Apple Silicon days the highest performance was always exclusive to the Mac Pro.
It’s disappointing that Apple wasn’t able to make something better than an Ultra chip and the RAM limit for the new Mac Pro has decreased and is no different to the studio.
It’s disappointing that Apple wasn’t able to make something better than an Ultra chip and the RAM limit for the new Mac Pro has decreased and is no different to the studio.
Why? Nostalgia for an old product range? It seems better not to have to buy that clunky enclosure if you don’t need it.
I think what the parent meant is that, once Apple decided that the Mac Pro should have the M2 chips, they must have targetted feature parity with the intel versions, aka in the amount of RAM. But presumably, they couldn't get there despite the extra year.
> How about a 4x, 8x and 16x M2 Max chiplet design which would have truly been next level...
That was entirely Bloomberg BSing. The people working on reverse-engineering Apple Silicon said that idea was infeasible all the way back in 2021.
That was entirely Bloomberg BSing. The people working on reverse-engineering Apple Silicon said that idea was infeasible all the way back in 2021.
Its true that Hector Martin said they didn't have the ability to add another interconnect like they did to double the Max to get Ultra.
But that ignores the entirely different chip design T6500 which was assumed to be for the new Mac Pro. No technical details of it ever leaked but its reasonable to assume it would offer higher performance than the T6002 chips which became M1 Max/Ultra
But that ignores the entirely different chip design T6500 which was assumed to be for the new Mac Pro. No technical details of it ever leaked but its reasonable to assume it would offer higher performance than the T6002 chips which became M1 Max/Ultra
> But that ignores the entirely different chip design T6500 which was assumed to be for the new Mac Pro.
The thing about leaks is that they aren't confirmed and might, literally, not exist. In which case, you might be upset at Apple for not releasing a chip that they literally never made. Anyone can make up a fake chip number, create a Chinese account, and make a fake leak for clicks or the lulz, or reach out to a legitimate leaker and mislead them.
But let's say it did exist. You are assuming that it can be mass-produced, even in small quantities, without immense waste or excessive resources, or at a R&D cost or tooling cost so large that it will never break even. Some things are just experiments.
The thing about leaks is that they aren't confirmed and might, literally, not exist. In which case, you might be upset at Apple for not releasing a chip that they literally never made. Anyone can make up a fake chip number, create a Chinese account, and make a fake leak for clicks or the lulz, or reach out to a legitimate leaker and mislead them.
But let's say it did exist. You are assuming that it can be mass-produced, even in small quantities, without immense waste or excessive resources, or at a R&D cost or tooling cost so large that it will never break even. Some things are just experiments.
[deleted]
Is this now the best hardware (or at least pareto frontier) for local LLMs?
Not really? Inference on Mac GPUs is barely in its infancy.
I have no idea about training... I have never even seen a metal llm training repo, but you can bet the new innovations/papers will not work ootb.
But setting the software (and OSX quirks) aside, the M2s would be fantastic... If they didn't cost a bazillion dollars for the higher memory configs.
I have no idea about training... I have never even seen a metal llm training repo, but you can bet the new innovations/papers will not work ootb.
But setting the software (and OSX quirks) aside, the M2s would be fantastic... If they didn't cost a bazillion dollars for the higher memory configs.
OK but I asked "Is this now the best hardware (or at least pareto frontier) for local LLMs?" and it sounds like you're mostly talking about software issues.
Hmmm... theoretically maybe? You can't argue with 192GB hooked up to a single GPU, even if the die is split. I'm not sure if it behaves more like a single GPU or 2 NVLink GPUs when running or training llms.
But again, at that price point and with theoretical software support, you are starting to look at AMD MI300s, Intel GPU max/gaudi boxes, 4x 4090 boxes and such, so I am not sure.
But again, at that price point and with theoretical software support, you are starting to look at AMD MI300s, Intel GPU max/gaudi boxes, 4x 4090 boxes and such, so I am not sure.
If you need RAM more than you need compute maybe? The M1 Ultra was much slower than the 3090, and by all accounts the 4090 made a significantly bigger step forward in performance over the 3090 than the M2 Ultra does over the M1 Ultra.
If the M2 Ultra can double the performance of an M2 Max it will be on par with the 4070ti in compute, or slightly ahead of a 3090. But we'll have to wait for benchmarks to see.
What are some use cases for PCI in the M2 Mac Pro Silicon?
All I can think of is people who need more ethernet ports. It doesn't seem like it'll work for video cards, right?
All I can think of is people who need more ethernet ports. It doesn't seem like it'll work for video cards, right?
Single M2 Ultra only.
I wonder: will they start signing Nvidia drivers again?
Could you even develop a non Apple gpu driver with the way extensions are allowed to work now?
Almost certainly not.
This is the announcement you are looking for:
> The base model config Mac Pro starts at $6999.
> Mac Pro with M2 Ultra features a 24-core CPU, up to 76-core GPU and 192 GB RAM. It also features two HDMI ports, dual 10-gigabit Ethernet, and a 32-core Neural Engine for machine learning tasks.
Yes Please. Great Deal.
> The base model config Mac Pro starts at $6999.
> Mac Pro with M2 Ultra features a 24-core CPU, up to 76-core GPU and 192 GB RAM. It also features two HDMI ports, dual 10-gigabit Ethernet, and a 32-core Neural Engine for machine learning tasks.
Yes Please. Great Deal.
You aren't getting 76 cores and 192GB of RAM for $6999. Probably twice that fully loaded.
Yup. Apple wanted $3,000 for the Intel cheesegrater to go from 32 to 192GB.