AMD Instinct MI300X Accelerators(amd.com)
amd.com
AMD Instinct MI300X Accelerators
https://www.amd.com/en/products/accelerators/instinct/mi300/mi300x.html
2 comments
What card could you have bought that lost support months later? I can't think of any new card you could have bought that'd lose support that quickly as far back as 2016
https://github.com/ROCm/ROCm/issues/1353
Bought in 2020. Stopped working in 2020. Not the latest, but in-production, advertised ROCm-capable, and what I could find during the Great GPU Shortage of 2020.
Bought in 2020. Stopped working in 2020. Not the latest, but in-production, advertised ROCm-capable, and what I could find during the Great GPU Shortage of 2020.
Wow, not even an industry-standard-for-many-decades deprecation notice up front, to give people a heads up before hand.
That's pretty fucked. :(
That's pretty fucked. :(
It's even worse.
Order of operations:
1. It just silently stopped working, started crashing, and I had no idea what was going on or why. Just sort of intermittent f-age where some older versions would work better, and newer ones would work worse. I never got it reliably working. In most cases, at some point, the system would become unresponsive, and then crash hard.
2. AMD removed it from the supported list (with no notice).
3. The github ticket above was filed, which explained what was going on.
Lots of time wasted debugging. This interacted with a half-dozen other AMD bugs and issues (such as learning the GPU only worked for compute headless; I needed to drive my monitor with a different card).
Human time is more expensive than equipment, so the total cost of this stuff was astronomical.
Yes, notice is industry-standard, but at the very least, when support broke / was later removed, the driver could try warning me "We no longer support this card, do you REALLY want to proceed?" rather than letting me know by hard-crashing my system.
Order of operations:
1. It just silently stopped working, started crashing, and I had no idea what was going on or why. Just sort of intermittent f-age where some older versions would work better, and newer ones would work worse. I never got it reliably working. In most cases, at some point, the system would become unresponsive, and then crash hard.
2. AMD removed it from the supported list (with no notice).
3. The github ticket above was filed, which explained what was going on.
Lots of time wasted debugging. This interacted with a half-dozen other AMD bugs and issues (such as learning the GPU only worked for compute headless; I needed to drive my monitor with a different card).
Human time is more expensive than equipment, so the total cost of this stuff was astronomical.
Yes, notice is industry-standard, but at the very least, when support broke / was later removed, the driver could try warning me "We no longer support this card, do you REALLY want to proceed?" rather than letting me know by hard-crashing my system.
People don't seem to understand how ROCm fails. Some inaccessible list somewhere buried deep down or a GitHub issues says your card is dropped. Apparently the average user is supposed to spend ages researching this. When you try it out, as any sane person does, you don't get a nice "unsupported GPU" message from ROCm. The failure when you use ROCm is instability of your entire OS, not some clean crashing of the program you ran. This invites lots of messing around and desperately trying to get it to work and is a very frustrating experience and then all people do is say "look in this obscure GitHub issue they dropped support for you GPU, you're the one in the wrong".
Not GP, but they dropped support for SKUs which were still sold at that point. The MI50 would be a fun accelerator to play with and easily worth 2000$ if it were made by Nvidia. Since they are made by AMD and not supported, you can grab them for 200$.
https://github.com/ROCm/ROCm/issues/2308
https://github.com/ROCm/ROCm/issues/2308
Support still exists, but the card is in maintenance mode. This doesn't mean that your models that work on an MI50 today are suddenly going to stop working, or the models ran on my MI25 will cease suddenly.
That's only true if you don't mind sharing your systems with strangers in Russia, Asia, and Africa.
"Bug fixes / critical security patches will continue to be supported for the gfx906 GPUs till Q2 2024"
Worse, a lot of tooling will break before then. If you need a security patch for anything built on top of ROCm or CUDA, you often can't have an ancient version.
That's not to mention that I'd like to do new development and prototyping.
I don't do business with AMD for the same reason I don't do business with Google. They're not a reliable partner.
"Bug fixes / critical security patches will continue to be supported for the gfx906 GPUs till Q2 2024"
Worse, a lot of tooling will break before then. If you need a security patch for anything built on top of ROCm or CUDA, you often can't have an ancient version.
That's not to mention that I'd like to do new development and prototyping.
I don't do business with AMD for the same reason I don't do business with Google. They're not a reliable partner.
The ROCm tooling only supports Ubuntu, Fedora and a few other enterprise distro, but it is on track to make it into the main Debian archive at which point the Debian maintainers will backport fixes as needed.
The tooling for older MI25 cards has yet to break, please avoid creating FUD on hardware you don't own.
The tooling for older MI25 cards has yet to break, please avoid creating FUD on hardware you don't own.
That is the thing about AMD GPUs, nobody sane wants to own the GPUs that work with ROCm.
That the only bad actors are in Russia, Asia and Africa is becoming quite the trope.
Some very spectacular compromises by Lapsus$ aren’t originating from any of the above, but instead from the West, specifically the UK:
https://www.bbc.com/news/technology-67663128
Some very spectacular compromises by Lapsus$ aren’t originating from any of the above, but instead from the West, specifically the UK:
https://www.bbc.com/news/technology-67663128
I think you're missing the point. Bad actors are equally distributed around the world.
The difference, however, is that, in this case, as the article points out, the bad actor is in custody.
My local law enforcement, lawyer, and judicial system are able to help out with the bad actors here and keep the problem somewhat contained.
For a bad actor in Russia or India, you quite literally have no recourse. There are literally Youtube videos (Mark Rober did a special recently) showing out-of-jurisdiction scammer and hacker shops occupying entire office buildings, and there is almost nothing we can do about it.
I'm not even going to go on a diatribe about corruption and the quality of law enforcement, since one can have different opinions. What I will point out is that US+EU have extensive treaties which allow my local law enforcement agency to cooperate closely with law enforcement agencies between US, UK, and EU.
For a bad actor in Iran or North Korea? You're not even getting diplomatic contact.
Footnote: I mention India since they're a democracy and by all standards, try to act responsibly on the global stage. They just don't happen to be in-network for various US/EU-centric international conventions.
The difference, however, is that, in this case, as the article points out, the bad actor is in custody.
My local law enforcement, lawyer, and judicial system are able to help out with the bad actors here and keep the problem somewhat contained.
For a bad actor in Russia or India, you quite literally have no recourse. There are literally Youtube videos (Mark Rober did a special recently) showing out-of-jurisdiction scammer and hacker shops occupying entire office buildings, and there is almost nothing we can do about it.
I'm not even going to go on a diatribe about corruption and the quality of law enforcement, since one can have different opinions. What I will point out is that US+EU have extensive treaties which allow my local law enforcement agency to cooperate closely with law enforcement agencies between US, UK, and EU.
For a bad actor in Iran or North Korea? You're not even getting diplomatic contact.
Footnote: I mention India since they're a democracy and by all standards, try to act responsibly on the global stage. They just don't happen to be in-network for various US/EU-centric international conventions.
At least some of this is "support" meaning different things to different people.
AMD means something like "your engineers can raise jira tickets related to the supercomputer you bought". The general idea outside of that context seems to be "support" means "can run stuff on the GPU". There's some analogy to games dev - if call of duty doesn't run on an amdgpu, you also don't get to complain to AMD about it, but the games studio certainly will do. The games studio is the entity with the contact details.
ROCm is fairly likely to run on unsupported consumer cards if you build the libraries from source or use them from someone who did, which is just about viable from linux distributions now. You don't get to raise official jira tickets about it when it falls over because it's "unsupported". You'll have a moderately bad time if you ignore the docs recommendations about the kernel version for a given release, or if the difficult task of building the libraries from source goes awry.
The established packaging idea of only shipping machine code libraries for a subset of the GPUs you sell, and giving very limited information on how to make those libraries exist for other GPUs, is obviously making essentially everyone very pissed off with AMD. Their engineers are aware of that and disagree about what to do about it.
I personally think that'll resolve itself once Debian packaging is generally usable. People will install ROCm from their distro instead of from the official builds which don't have machine code for gaming GPUs anyway. That will give rise to new annoyances but at least the applications will try to run on whatever GPU you've got locally.
AMD means something like "your engineers can raise jira tickets related to the supercomputer you bought". The general idea outside of that context seems to be "support" means "can run stuff on the GPU". There's some analogy to games dev - if call of duty doesn't run on an amdgpu, you also don't get to complain to AMD about it, but the games studio certainly will do. The games studio is the entity with the contact details.
ROCm is fairly likely to run on unsupported consumer cards if you build the libraries from source or use them from someone who did, which is just about viable from linux distributions now. You don't get to raise official jira tickets about it when it falls over because it's "unsupported". You'll have a moderately bad time if you ignore the docs recommendations about the kernel version for a given release, or if the difficult task of building the libraries from source goes awry.
The established packaging idea of only shipping machine code libraries for a subset of the GPUs you sell, and giving very limited information on how to make those libraries exist for other GPUs, is obviously making essentially everyone very pissed off with AMD. Their engineers are aware of that and disagree about what to do about it.
I personally think that'll resolve itself once Debian packaging is generally usable. People will install ROCm from their distro instead of from the official builds which don't have machine code for gaming GPUs anyway. That will give rise to new annoyances but at least the applications will try to run on whatever GPU you've got locally.
"Support," to me, means not dropping support in newer versions of ROCm. See the current compatibility matrix:
https://rocm.docs.amd.com/projects/radeon/en/latest/docs/com...
I bought my (in-production, advertised ROCm-capable) card during the great GPU shortage of 2020, and literally anything new stopped working a few months later when AMD decided to update ROCm to drop support for GFX8 GPUs:
https://github.com/ROCm/ROCm/issues/1353
All the packages I was using (like Spacy) required later ROCm almost immediately. There was literally no way to make them work. Running older packages is a non-starter if you want things like, say, your machines to not be compromised.
> if call of duty doesn't run on an amdgpu, you also don't get to complain to AMD about it, but the games studio certainly will do
If "Call of Duty" is advertised on the box, and doesn't run, yes, this is false advertising, and "consumers" get to complain to the graphics card maker or AMD. In this case, small claims court would give me a refund if I sued AMD; an advertised ROCm-compatible GPU stopped being ROCm-compatible within the warranty period. It's just not worth complaining or suing over a few hundred bucks.
I didn't need much performance, and this was purely for dev -- to make sure I bought this purely so I could have compatibility with AMD. AMD decided to not have compatibility with me, and I said f--k AMD.
The result is that if you want to run my data pipeline, it won't work on AMD. A major industry will hopefully standardize on it, and guess who is locked out of selling any GPUs to that industry if that happens?
https://rocm.docs.amd.com/projects/radeon/en/latest/docs/com...
I bought my (in-production, advertised ROCm-capable) card during the great GPU shortage of 2020, and literally anything new stopped working a few months later when AMD decided to update ROCm to drop support for GFX8 GPUs:
https://github.com/ROCm/ROCm/issues/1353
All the packages I was using (like Spacy) required later ROCm almost immediately. There was literally no way to make them work. Running older packages is a non-starter if you want things like, say, your machines to not be compromised.
> if call of duty doesn't run on an amdgpu, you also don't get to complain to AMD about it, but the games studio certainly will do
If "Call of Duty" is advertised on the box, and doesn't run, yes, this is false advertising, and "consumers" get to complain to the graphics card maker or AMD. In this case, small claims court would give me a refund if I sued AMD; an advertised ROCm-compatible GPU stopped being ROCm-compatible within the warranty period. It's just not worth complaining or suing over a few hundred bucks.
I didn't need much performance, and this was purely for dev -- to make sure I bought this purely so I could have compatibility with AMD. AMD decided to not have compatibility with me, and I said f--k AMD.
The result is that if you want to run my data pipeline, it won't work on AMD. A major industry will hopefully standardize on it, and guess who is locked out of selling any GPUs to that industry if that happens?
That seems to be a different thing again. You had at least one rocm version that claimed to work with your GPU and then later versions dropped it. Your options then are to stay with the rocm version that claimed to support it or to build a newer rocm yourself, or get it from distros etc, where at some point in the future it'll probably stop working entirely.
I don't know what plays into which GPUs are listed as supported on which versions of ROCm. I'm sure my 6800GT was marked "supported" on some version and isn't on the current one. I think my radeon vii was likewise "supported" on some of them and isn't any more. I used a 5700 XT for a year or so which was never on the blessed list. I obsessively build everything from source instead of using the release binaries though.
I don't know what plays into which GPUs are listed as supported on which versions of ROCm. I'm sure my 6800GT was marked "supported" on some version and isn't on the current one. I think my radeon vii was likewise "supported" on some of them and isn't any more. I used a 5700 XT for a year or so which was never on the blessed list. I obsessively build everything from source instead of using the release binaries though.
But nvidia isn’t pulling this crap. You don’t get one old version immediately dropped. So another choice is to use nvidia. And cuda support covers a large number of their cards. And event recent releases cover a good chunk
Neither of those would have worked, but that's another post. However, I had a third option, which I took: I bought a working NVidia card, put my AMD card in my attic, swore off AMD GPUs and ROCm forever*, and began warn other people against ever relying on AMD.
That's the option I took.
Now, my software is tied to CUDA. Others are building more on top if it. If people want to run my software (or by proxy, and software which uses my software as a platform), they have more-or-less the same option if they accidentally bought AMD.
So far, I haven't heard from any users who accidentally bought AMD.
Right now, I'm just waiting for Arc A770 support to improve, so I can also support Intel. That will be the cheapest way to run my tools, since you get 16GB for $300.
I'm not the only one who took this path. I'm just more vocal about it.
* Some exclusions apply.
That's the option I took.
Now, my software is tied to CUDA. Others are building more on top if it. If people want to run my software (or by proxy, and software which uses my software as a platform), they have more-or-less the same option if they accidentally bought AMD.
So far, I haven't heard from any users who accidentally bought AMD.
Right now, I'm just waiting for Arc A770 support to improve, so I can also support Intel. That will be the cheapest way to run my tools, since you get 16GB for $300.
I'm not the only one who took this path. I'm just more vocal about it.
* Some exclusions apply.
Good for you. AMD is focussing on larger customers who write their own software (that runs on supercomputer and in datacenters). Targeting a high-margin, limited scope market before prosumers/consumers is decent strategy for a company that doesn't have Nvidia's deep pockets.
It is frustrating as a consumer - dealing with ROCm on an consumer card was a massive pain for the period I bothered to put in the effort. However, I remain bullish on AMD, I think their current GPUs are their Bulldozer-equivalent; I'm waiting patiently for the AMD GPU Zen-1-moment
It is frustrating as a consumer - dealing with ROCm on an consumer card was a massive pain for the period I bothered to put in the effort. However, I remain bullish on AMD, I think their current GPUs are their Bulldozer-equivalent; I'm waiting patiently for the AMD GPU Zen-1-moment
What you described would have been a fine strategy. That's not AMD's strategy.
To implement your strategy, you either don't ship mass market GPGPU cards, or you stick a big "beta / prototype" label on ROCm and make sure people use it only caveat emptor. I've bought plenty of half-baked things because I wanted to tinker with them or help a company along. AMD cards are very competitive for gaming too, which really brings volume much more so than GPGPU, so there's no reason to oversell.
AMD's problem is that they're claiming these things work, and they don't. I was fortunate to learn that lesson spending a few hundred bucks. I know people who learned that lesson spending thousands. I wouldn't want to figure out on a data center or a supercomputer whether or not the same lesson applies. If a company is willing to cheaply torch relationships as it figures out a strategy, ship half-baked ill-thought-out software, and make bold, unsubstantiated claims, I'll pick another vendor if at all competitive.
AMD is losing mind share, credibility, and branding. That will hurt if they ever have a product ready for the broader market, and it will also hurt (although not as much) with the strategy you described.
To implement your strategy, you either don't ship mass market GPGPU cards, or you stick a big "beta / prototype" label on ROCm and make sure people use it only caveat emptor. I've bought plenty of half-baked things because I wanted to tinker with them or help a company along. AMD cards are very competitive for gaming too, which really brings volume much more so than GPGPU, so there's no reason to oversell.
AMD's problem is that they're claiming these things work, and they don't. I was fortunate to learn that lesson spending a few hundred bucks. I know people who learned that lesson spending thousands. I wouldn't want to figure out on a data center or a supercomputer whether or not the same lesson applies. If a company is willing to cheaply torch relationships as it figures out a strategy, ship half-baked ill-thought-out software, and make bold, unsubstantiated claims, I'll pick another vendor if at all competitive.
AMD is losing mind share, credibility, and branding. That will hurt if they ever have a product ready for the broader market, and it will also hurt (although not as much) with the strategy you described.
AMD hardware is amazing. AMD software needs catching up to do. It's pretty amazing how Nvidia's strategy of making compute tools available on most consumer grade cards is paying off latent dividends -- consumers learn on their own hw and become developers/researchers using same stack, something AMD needs to realize. The result is Nvidia is a trillion dollar company and AMD only 1/5 of that.
> AMD hardware is amazing
How did you reach the conclusion? In almost all the metric, AMD is just 20-30% ahead or wildly behind. e.g. AMD is so much behind in tensor cores in consumer grade GPU or raytracing.
How did you reach the conclusion? In almost all the metric, AMD is just 20-30% ahead or wildly behind. e.g. AMD is so much behind in tensor cores in consumer grade GPU or raytracing.
I'm just saying AMD hardware is (namely their compute line of cards). Nvidia has great scientists in light transport research and it shows in their Real-time ray tracers.
Did you read his post?
Software isn't catching up. For most AMD GPU owners, the software quality is getting worse over time.
Software isn't catching up. For most AMD GPU owners, the software quality is getting worse over time.
AMD need to fully support their cards as long as Nvidia does.
(Note: I know they never will. This story, multiplied by the number of people building stuff now, is why I'm bearish on AMD and bullish on competitors. This would be a very nice card -- 192GB is sexy -- but I just get headaches thinking about compatibility, software, and drivers issues. To quote Steve Balmer, what AMD is missing is: "Developers, developers, developers." That's not a small hole to dig out of.)