The Secret Apple M1 Coprocessor(medium.com)
medium.com
The Secret Apple M1 Coprocessor
https://medium.com/swlh/apples-m1-secret-coprocessor-6599492fc1e1
108 comments
It should also be noted that ARM has multiple types of licences. Most manufacturers only license cores and adjacent IPs, with no possible "tweaks" or the license with minimal "tweaks". However, my understanding is that apple has an "ISA" architecture, where they basically can do whatever they want, since they can build a custom CPU that uses the ARM ISA, whereas Samsung for example, can only "tweak" the cores.
Basically, Apple doesn't use core IP and "only" licenses the ISA (which is far more expensive if my memory is not lying) and so they can do whatever they want.
Basically, Apple doesn't use core IP and "only" licenses the ISA (which is far more expensive if my memory is not lying) and so they can do whatever they want.
It would make sense that Apple is exempt from some of the rules. After all, Apple co-founded ARM.
Right, none of this matters for companies just using ARM core IP.
The broadest and rarest type of ARM license that is publicly documented is an ARM architecture license, which allows the licensee to implement one of the ARM ISAs with their own microarchitecture.
Apple and Samsung both have architecture licenses, though IIRC Samsung recently closed down their CPU design team and so is no longer making use of their architecture license.
The normal terms of architectural licenses do not permit the licensees to do whatever they want, they are typically bound by the ARM ISA specification. Normal architecture licenses don't allow the licensee to add custom instructions.
The broadest and rarest type of ARM license that is publicly documented is an ARM architecture license, which allows the licensee to implement one of the ARM ISAs with their own microarchitecture.
Apple and Samsung both have architecture licenses, though IIRC Samsung recently closed down their CPU design team and so is no longer making use of their architecture license.
The normal terms of architectural licenses do not permit the licensees to do whatever they want, they are typically bound by the ARM ISA specification. Normal architecture licenses don't allow the licensee to add custom instructions.
Samsung and Apple have, afaik, same license and did fully custom implementation of ARM spec (short story - Apple A-series and Samsung's Exynos have common ancestry).
But even if you have full license, it doesn't necessarily mean you have time and resources to do full custom implementations all the time.
But even if you have full license, it doesn't necessarily mean you have time and resources to do full custom implementations all the time.
They were designed to the same spec, and had to pass the same suite of compatibility tests. However, they handled the implementation details differently.
>Architectural licensees get a set of specs and a testing suite that they have to pass, the rest is up to them. If they want to make a processor that is faster, slower, more efficient, smaller, or anything else than the one ARM supplies, this is what they have to do.
https://semiaccurate.com/2013/08/07/a-long-look-at-how-arm-l...
>Architectural licensees get a set of specs and a testing suite that they have to pass, the rest is up to them. If they want to make a processor that is faster, slower, more efficient, smaller, or anything else than the one ARM supplies, this is what they have to do.
https://semiaccurate.com/2013/08/07/a-long-look-at-how-arm-l...
Hummingbird and A4 were co-developed originally by Samsung and Apple, but the companies parted ways before finishing either one. That's why I mention common ancestry (which they don't share with Qualcomm's custom design, for example)
Apple did use the Hummingbird part (which was developed by Samsung and optimized by Intrinsity using their custom EDA software) and Apple did later buy Intrinsity.
However, neither Apple or Intrinsity had any IP rights over Samsung's core, as far as I ever saw.
However, neither Apple or Intrinsity had any IP rights over Samsung's core, as far as I ever saw.
My understanding of the event chain is that originally there was a common project, and both parties had rights to use the IP from it, but without any rights to further changes after the split.
Didn't know the part about Intrinsity, thanks for that :)
Didn't know the part about Intrinsity, thanks for that :)
Apple does indeed have an architcture license, which grants them the rights to modify&ship ARM IP.
Standard architecture licenses require the custom microarchitecture to comply with ARMs architectural specifications, but M1 apparently does not.
An architecture license is not usually a free pass to modify the architecture and fragment ARM's ecosystem at will.
An architecture license is not usually a free pass to modify the architecture and fragment ARM's ecosystem at will.
Apple was one of the codevelopers of the arm6 and at that time Acorn spun out ARM holdings to create the ARM of today - so while architectural licences might come with restrictions, Apple most very likely have even deeper rights into the ARM ecosystem, up and including - "if you don't let us make the M1, we will deep 6 the entire arm ecosystem and use our "cocreator" patents offensively"
It's easy to forget that Apple is/was one of the original computer companies, and had a founding hand in two ISA/architectures: PowerPC and ARM. They also used the Motorola 68000 and x86 (32+64), but in those cases they were relying on the manufacturer for the chips and architectural direction.
It's easy to forget that Apple is/was one of the original computer companies, and had a founding hand in two ISA/architectures: PowerPC and ARM. They also used the Motorola 68000 and x86 (32+64), but in those cases they were relying on the manufacturer for the chips and architectural direction.
> "if you don't let us make the M1, we will deep 6 the entire arm ecosystem and use our "cocreator" patents offensively"
I think those patents would have expired by now.
I think those patents would have expired by now.
Apple is effectively a co-founder of ARM. I am absolutely certain they have a 'do whatever the hell you want with it, and have most-favoured on licensing' agreement in a drawer somewhere in Cupertino.
I would think that Apple is quite in a position to strongARM them and get what they want; even with Jobs gone I doubt they want to repeat the PowerPC fiasco.
This only applies if Apple would sell their chips as ARM Cpus to other companies etc.
Do you know this, or is this a guess?
This would be surprising to me.
Suppose Amazon acquires an architecture license, designs a custom core for Graviton3, and deploys it in AWS. Could they add custom instructions at will?
I would think ARM would be very unhappy when there are applications that run on amazon's arm64 CPU that don't run on ARM's arm64 CPUs, and I would imagine that that would be reflected in the license terms.
This would be surprising to me.
Suppose Amazon acquires an architecture license, designs a custom core for Graviton3, and deploys it in AWS. Could they add custom instructions at will?
I would think ARM would be very unhappy when there are applications that run on amazon's arm64 CPU that don't run on ARM's arm64 CPUs, and I would imagine that that would be reflected in the license terms.
I recall a relevant piece of history where the Intel 286 didn't have the ability to drop from the new protected mode back to real mode without a full system reset. A team at IBM eventually figured out that with a well crafted dance of instructions this could be achieved by taking advantage of specific quirks within the chip.
By the time the 386 was released, this method was so common that Intel had to preserve these quirks to maintain compatibility with existing software. Apple may find themselves in a similar situation if they're not careful.
By the time the 386 was released, this method was so common that Intel had to preserve these quirks to maintain compatibility with existing software. Apple may find themselves in a similar situation if they're not careful.
It is well known that Apple does not follow such policy. If you didn't follow their standards and your software break, then that is on you. Apple will not lift a finger. This has been Apple practice for over 20 years, and if you develop on the Mac platform you should be well aware of it or suffer the consequences.
Apple actually has implemented manual compatibility fixes for several third party apps. https://worthdoingbadly.com/appkitcompat/
In the early years, Microsoft had written some system hacks that they used to run Word and Excel on Apple computers. Apple had to be careful not to break those hacks when releasing OS updates. Eventually Apple got tired of a vendor dictating changes on it's own OS and got Microsoft to remove these hacks.
Yep, some vendors have that kind of pull, which is exactly why the Carbon API was created and persisted for a long while. But, like Carbon, Apple will eventually sunset anything that drags at them evolving the platform.
I'm curious as to what hacks these were?
I can remember when Microsoft decided to port some of the underpinnings of early versions of Windows to the Mac so they could run a unified code base for Word 6 on the Mac and on Windows, but they had to abandon that effort almost immediately as it was bloated, too slow, and didn't use the native UI.
>Mac Word 6.0 was a crappy product. And, we spent some time trying to figure out how not to do that again. In the process, we learned a few things, not the least of which was the meaning of the term “Mac-like.”
https://web.archive.org/web/20040514091238/http://blogs.msdn...
That's the only time I can remember them doing something unique that other vendors weren't also doing.
I can remember when Microsoft decided to port some of the underpinnings of early versions of Windows to the Mac so they could run a unified code base for Word 6 on the Mac and on Windows, but they had to abandon that effort almost immediately as it was bloated, too slow, and didn't use the native UI.
>Mac Word 6.0 was a crappy product. And, we spent some time trying to figure out how not to do that again. In the process, we learned a few things, not the least of which was the meaning of the term “Mac-like.”
https://web.archive.org/web/20040514091238/http://blogs.msdn...
That's the only time I can remember them doing something unique that other vendors weren't also doing.
I only remember that they postponed the move from a 24-bit address bus to a full 32-bit one because about every program would have crashed (https://en.wikipedia.org/wiki/Classic_Mac_OS_memory_manageme.... Many programs flipped bits because it was a lot faster than making the system call that did the same)
I don’t remember any Excel-specific code to keep Excel running, but there was a special error code from the “launch program” system call saying “program has special memory requirements”, but AFAIK, it actually meant “program is Excel version such and such” (changing the programs’s creator code made “launch program” succeed, but the program almost immediately crashed.
I don’t remember any Excel-specific code to keep Excel running, but there was a special error code from the “launch program” system call saying “program has special memory requirements”, but AFAIK, it actually meant “program is Excel version such and such” (changing the programs’s creator code made “launch program” succeed, but the program almost immediately crashed.
Apple does not really care about software using unsupported resources. They don't even care much about backward compat for supported ones...
With the level of vertical integration they have now it is unlikely they will ever start to care.
With the level of vertical integration they have now it is unlikely they will ever start to care.
> A team at IBM eventually figured out that with a well crafted dance of instructions this could be achieved by taking advantage of specific quirks within the chip.
Details are described at [1], tl;dr:
- store register values in RAM
- set magic value in CMOS
- cause triple fault to reset CPU
- BIOS sees magic value and instead of resetting rest of the system, it resumes from your stored state in real mode
So it's more like a BIOS quirk. A20 gate backward compatibility on the other hand ...
[1] - http://rcollins.org/articles/pmbasics/tspec_a1_doc.html
Details are described at [1], tl;dr:
- store register values in RAM
- set magic value in CMOS
- cause triple fault to reset CPU
- BIOS sees magic value and instead of resetting rest of the system, it resumes from your stored state in real mode
So it's more like a BIOS quirk. A20 gate backward compatibility on the other hand ...
[1] - http://rcollins.org/articles/pmbasics/tspec_a1_doc.html
Apple just breaks such Software.
IBM used an external processor, they 8042 which was the keyboard interface, to reset the 286 on demand to go back to real mode. The 286 did have some quirks, but as far as I know none that allowed you to escape protected mode.
We could only wish - Apple doesn’t help getting stuff running (which would be easy) much less give a single thought to breaking third party software not running on their OS. They are switching entire architecture to M1 / ARM
It seems like this is a crucial part of their competitive advantage, while Intel ends up totally hamstrung under the weight of four decades of past design decisions.
It's certainly impressive, they'll have done two full architecture shifts now on their desktop product line pretty seamlessly.
Apple actually did announce this capability in their “Time Flies” keynote which introduced the A14 with the new iPad. They call it “Second-generation machine learning accelerators” in the CPU, with “accelerated matrix multiplication”.
Start at 53:14: https://youtu.be/b13xnFp_Los
Start at 53:14: https://youtu.be/b13xnFp_Los
They've advertised this many times in many venues. It has been a featured part of their keynote. It has been talked about endlessly.
There is positively nothing "secret" about this.
There is positively nothing "secret" about this.
... it's also not a "coprocessor"
Thank you. Mods should change the title to avoid clickbait.
SIMD was probably the wrong direction to take instead of vector / matrix coprocessors, as SIMD can't do as aggressive pipelining.
RISC-V is fixing it, but it's great that Apple is also going into this direction.
RISC-V is fixing it, but it's great that Apple is also going into this direction.
Cool, how is RISC-V dealing with this? Can’t wait to see what processor in 10 years or so on 1nm lithography look like, they might even be able to run a JavaScript test suit at a reasonable pace or compile Rust fast :-)
Generally the idea is to make the instruction set "scalable," such that if the CPU's vector ALU doubles in width, applications don't need to be rewritten or recompiled to benefit. (In contrast to existing SIMD instructions, which are for a specific width, which is why e.g. Intel introduces new instructions for wider and wider vectors - AVX1, AVX2, AVX512). ARM and RISC-V are both pursuing this approach, with SVE and the V extension, respectively.
Though it is unlikely that vector instructions will help very much for typical JS or compilation workloads.
Though it is unlikely that vector instructions will help very much for typical JS or compilation workloads.
Does any of these new vector instruction sets that only differ from the previous one fir the width of the operands still allow for a new patient to be filed?
I would guess yes, but don't really have any idea.
To clarify, though, the changes introduced in newer SIMD extensions are rarely just a width increase. AVX-512, for example, increased the width from 256-bit to 512-bit, but also brought some substantial programmability improvements.
To clarify, though, the changes introduced in newer SIMD extensions are rarely just a width increase. AVX-512, for example, increased the width from 256-bit to 512-bit, but also brought some substantial programmability improvements.
Maybe it's true, but as it can decrease the clock rate, I would never use it, because it completely destroys the predictability of runtime.
Linus was quite angry about it for a good reason.
Linus was quite angry about it for a good reason.
It has an instruction set where the instructions are independent of the size of the register. It's easier to look some videos about it on YouTube than to explain it actually.
This article links to this more detailed post which I’d recommend: https://nod.ai/comparing-apple-m1-with-amx2-m1-with-neon/
Thanks! @dang - can we not make this an automatic HN feature? That is, automatically linking to archive.is for all stories from medium.com, NY Times, etc.
I hope not. archive.is doesn’t resolve on all DNS providers.
You mean it is currently being censored by DNS providers and/or ISP's?
I’m not sure that’s the case.
https://community.cloudflare.com/t/archive-is-error-1001/182...
https://community.cloudflare.com/t/archive-is-error-1001/182...
I found the article could probably be trimmed down somewhat... is the idea that using a separate processor for SIMD style operating allows pipelining and other optimisations that would be difficult to implement in the context of a more general purpose CPU?
There is no "separate processor". These are just private ISA extensions.
Intel recently disclosed an ISA extension for matrix operations called Advanced Matrix Extensions (AMX). It should appear in Sapphire Rapids. https://fuse.wikichip.org/news/3600/the-x86-advanced-matrix-...
The action is in the linked gist https://gist.github.com/dougallj/7a75a3be1ec69ca550e7c36dc75...
# These
# instructions have been reversed from Accelerate (vImage, libBLAS, libBNNS,
# libvDSP and libLAPACK all use them), and by experimenting with their
# behaviour on the M1.
# These
# instructions have been reversed from Accelerate (vImage, libBLAS, libBNNS,
# libvDSP and libLAPACK all use them), and by experimenting with their
# behaviour on the M1.
Oh shiny! As a data scientist who currently is in Pandas day-to-day, under the hood it uses BLAS (and I think LAPACK) which should give some nice acceleration benefits when working with larger dataframes on Apple's M series chips.
To benefit from the acceleration, one should use the Apple implementation of BLAS/LAPACK: the Accelerate framework, for the moment I think numerical python libs link against OpenBLAS ? But I'm not 100% sure.
Numpy recently removed Apple's Accelerate framework as a supported BLAS implementation, due to alleged bugs in Accelerate.
The Accelerate framework is the only thing that has the Matrix co-processor support.
https://github.com/numpy/numpy/pull/15759
The Accelerate framework is the only thing that has the Matrix co-processor support.
https://github.com/numpy/numpy/pull/15759
This article about the undocumented Apple AMX instructions is the missing piece of the puzzle for me. Various HN threads discussed SIMD [1] and ML/DL [2] on the Apple M1. Daniel Lemire's benchmark compared x64 SIMD (SSE, AVX) vs. ARM SIMD (Neon, SVE2 [next gen]). Apple's AMX is a coprocessor that does Matrix math in addition to Vector SIMD math. Maynard Handley clearly understood this in his comment on Lemire's blog [3]:
> You (and other commenters) are aware of NEON, but apparently not of AMX. AMX may not work for the sorts of JSON parsing weirdness for which you use AVX256 (that’ll have to wait for SVE/2, probably next year) but it does solve the problem of “I want to execute dense linear algebra fast”.
> You might want to run some comparisons of that for your M1 vs Intel MacBooks… The API’s to look at are in Accelerate()
The Core ML [4] docs have a nice diagram that demonstrates the relationship with the lower level Accelerate [5] and Accelerate BNNS [6]:
> Core ML is the foundation for domain-specific frameworks and functionality. Core ML supports Vision for analyzing images, Natural Language for processing text, Speech for converting audio to text, and Sound Analysis for identifying sounds in audio. Core ML itself builds on top of low-level primitives like Accelerate and BNNS, as well as Metal Performance Shaders.
Apple's TensorFlow implementation for the M1 is analogous to Core ML. It probably invokes the appropriate Accelerate BNNS and/or Metal library calls. Maynard Handley was exactly right, SIMD/ML/DL code that targets Apple silicon should link against Apple's Accelerate or Core ML libraries. These libraries in turn invoke the appropriate AMX, Neural Engine, and GPU instructions for each target platform.
[1] https://news.ycombinator.com/item?id=25408853
[2] https://news.ycombinator.com/item?id=25773109
[3] https://lemire.me/blog/2020/12/11/arm-macbook-vs-intel-macbo...
[4] https://developer.apple.com/documentation/coreml
[5] https://developer.apple.com/documentation/accelerate
[6] https://developer.apple.com/documentation/accelerate/bnns
> You (and other commenters) are aware of NEON, but apparently not of AMX. AMX may not work for the sorts of JSON parsing weirdness for which you use AVX256 (that’ll have to wait for SVE/2, probably next year) but it does solve the problem of “I want to execute dense linear algebra fast”.
> You might want to run some comparisons of that for your M1 vs Intel MacBooks… The API’s to look at are in Accelerate()
The Core ML [4] docs have a nice diagram that demonstrates the relationship with the lower level Accelerate [5] and Accelerate BNNS [6]:
> Core ML is the foundation for domain-specific frameworks and functionality. Core ML supports Vision for analyzing images, Natural Language for processing text, Speech for converting audio to text, and Sound Analysis for identifying sounds in audio. Core ML itself builds on top of low-level primitives like Accelerate and BNNS, as well as Metal Performance Shaders.
Apple's TensorFlow implementation for the M1 is analogous to Core ML. It probably invokes the appropriate Accelerate BNNS and/or Metal library calls. Maynard Handley was exactly right, SIMD/ML/DL code that targets Apple silicon should link against Apple's Accelerate or Core ML libraries. These libraries in turn invoke the appropriate AMX, Neural Engine, and GPU instructions for each target platform.
[1] https://news.ycombinator.com/item?id=25408853
[2] https://news.ycombinator.com/item?id=25773109
[3] https://lemire.me/blog/2020/12/11/arm-macbook-vs-intel-macbo...
[4] https://developer.apple.com/documentation/coreml
[5] https://developer.apple.com/documentation/accelerate
[6] https://developer.apple.com/documentation/accelerate/bnns
I hate to say it but that article is overly confusing and needs better organization.
I understood most of the information presented, but missed some basic technical point of what the AMX does. Is it a specialized module + its libraries that's very good at dot products, for example? Matrix inversions?
That's what I wanted to know. Maybe I missed it.
That's what I wanted to know. Maybe I missed it.
Let me just say this: if that actually exists, it isn't the first time, by far, Apple has pulled off that trick (or others, like Intel for that matter)
> However due to pressure from customers ARM relented and announced in 2019 that they would allow extensions.
Of course Apple doesn't care and has been shipping custom instructions for longer than that…
Of course Apple doesn't care and has been shipping custom instructions for longer than that…
Reference to support this?
Plus you seem to imply that Apple has done this without Arm's consent which seems unlikely and even if so I would be astonished if that were in the public domain.
Plus you seem to imply that Apple has done this without Arm's consent which seems unlikely and even if so I would be astonished if that were in the public domain.
>Apple has done this without Arm's consent which seems unlikely
On the contrary, ARM has long worked closely with one or more licensees when developing new features.
>ARM picks 2-3 lead licensees for each market segment and works closely with them. This narrows down the pool of initial partners to a manageable number and also ensures that anyone who is picked is up to the task of successfully getting a new core to market correctly. The partners get about a year lead time to market, critical for some, far less important for others.
https://semiaccurate.com/2013/08/07/a-long-look-at-how-arm-l...
On the contrary, ARM has long worked closely with one or more licensees when developing new features.
>ARM picks 2-3 lead licensees for each market segment and works closely with them. This narrows down the pool of initial partners to a manageable number and also ensures that anyone who is picked is up to the task of successfully getting a new core to market correctly. The partners get about a year lead time to market, critical for some, far less important for others.
https://semiaccurate.com/2013/08/07/a-long-look-at-how-arm-l...
I don't know the contractual terms myself, but I suspect that if Apple is only doing this internally for their own products, it may well be none of ARM's business. It's only an issue if Apple sells their processors to other companies as components. In that case they are required to comply with ARM's certification requirements, because ARM doesn't want the market for ARM cpus to become fragmented. If Apple are only using these CPUs for their internal products I don't think those requirements apply.
It's a bit like the way the GPL has requirements that apply if you distribute the software (you have to release source code), but if you use your modified code internally the requirement doesn't apply. In this case "distributing" is equivalent to selling the CPUs as components, in contrast to using them in your own products (even ones you sell to customers) which counts as internal use and so the requirement doesn't apply.
It's a bit like the way the GPL has requirements that apply if you distribute the software (you have to release source code), but if you use your modified code internally the requirement doesn't apply. In this case "distributing" is equivalent to selling the CPUs as components, in contrast to using them in your own products (even ones you sell to customers) which counts as internal use and so the requirement doesn't apply.
I have no idea what Arm’s involvement is in all this, but from the outside it has consistently seemed like Apple launches chips that don’t follow what is documented and then the requirements invariably change soon after. Just off the top of my head, they’ve been shipping their own custom memory compression instructions for a while now, and I am pretty sure their chips dropped support for 4K granules much earlier than the rules claimed this was acceptable. AMX showed up in A13, I believe? Which would be just before the linked article is dated.
What is a memory compression instruction? You mean their CPUs have special machine code instructions as part of their ISA for compressing segments of memory? Or do you mean something more akin to the Thumb format but for ARM-64?
> You mean their CPUs have special machine code instructions as part of their ISA for compressing segments of memory?
Yes.
Yes.
> I have no idea what Arm’s involvement is in all this
But you implied that Apple flouted what was permitted by Arm.
> Just off the top of my head
That's not a reference.
As others have commented its much, much more likely that Apple and Arm have a very close relationship and that Apple are just at the leading edge.
But you implied that Apple flouted what was permitted by Arm.
> Just off the top of my head
That's not a reference.
As others have commented its much, much more likely that Apple and Arm have a very close relationship and that Apple are just at the leading edge.
I did not imply that. My comment was a response to the claim that Arm started allowing this in 2019 when Apple clearly was doing this earlier. For all I know the relationship is that Apple just writes the standard and tells Arm to publish it.
I have no idea what issues you had with my examples but if you describe them perhaps I can make them acceptable to you?
I have no idea what issues you had with my examples but if you describe them perhaps I can make them acceptable to you?
I'm genuinely interested in the point you make about the these instructions but it's not helpful to mention something like this without a link to more information.
On the first point you said 'Apple doesn't care' which I read as Apple doesn't care about the licensing rules - if you meant something else then fine but I think most people would read it that way.
On the first point you said 'Apple doesn't care' which I read as Apple doesn't care about the licensing rules - if you meant something else then fine but I think most people would read it that way.
I meant it in the way of "I don't think Apple has to care" since they do not follow them–what I do not know is if Apple has some other agreement in place, or they are violating something. For the instructions: the reason I didn't provide a link is because it would be very hard for me to do so, as these instructions are proprietary and undocumented, plus Apple strips out this code from their open source releases. The best I could find in the latest tarball is a definition for __ARM_WKDM_ISA_AVAILABLE__ for the Monsoon cores inside of A11: https://github.com/apple-open-source-mirror/xnu/blob/65d5be8...
> However more importantly, this is an advantage to Apple. By only letting their libraries use these special instructions Apple retains the freedom to radically change how this hardware works later. They could remove or add AMX instructions.
Depends on whether these instructions can be executed with unprivileged user-land code. Once the genie is out of the bottle, backward compatibility means imitating the old instruction set. Since a recompile is often cheap enough (and esp. if coprocessor calls can be remapped at the bitcode compilation time), this should be a non-issue.
Depends on whether these instructions can be executed with unprivileged user-land code. Once the genie is out of the bottle, backward compatibility means imitating the old instruction set. Since a recompile is often cheap enough (and esp. if coprocessor calls can be remapped at the bitcode compilation time), this should be a non-issue.
When Apple have decided not to document an instruction set to give them freedom to change it, I don't think they will cry about breaking your application in the next hardware version.
This is just like how (some) GPU vendors don't document their hardware's data structures and instruction set. You're living dangerously if you choose to rely on their exact behaviour. The vendors really only expect you to use their API (Vulkan/Metal/Direct3D/etc) because that abstraction layer allows them to radically and incompatibly change their hardware on a regular basis to improve performance or functionality.
This is just like how (some) GPU vendors don't document their hardware's data structures and instruction set. You're living dangerously if you choose to rely on their exact behaviour. The vendors really only expect you to use their API (Vulkan/Metal/Direct3D/etc) because that abstraction layer allows them to radically and incompatibly change their hardware on a regular basis to improve performance or functionality.
> This is just like how (some) GPU vendors don't document their hardware's data structures and instruction set. You're living dangerously if you choose to rely on their exact behaviour. The vendors really only expect you to use their API (Vulkan/Metal/Direct3D/etc) because that abstraction layer allows them to radically and incompatibly change their hardware on a regular basis to improve performance or functionality.
GPUs are external processors, that require DMA transport, and can perform useful jobs in parallel with the CPU, and are produced by multiple vendors. This almost certainly requires a HAL, and there's little impact of making dynamic libraries calls compared to the benefit received.
Compare this to a co-processor, that basically can be called directly from CPU instructions, without DMA and a remote instruction queue. Something like this gains great benefits from a compiler intrinsic, as a function call, particularly as a dynamic library is quite expensive. This compiler intrinsic can just map to bitcode, which can then map to whatever at compile-time on the app store deployment side.
Why would people not want this option? Why should this only be available to Apple's internal products? Why should Photoshop/Lightroom/Blender/physics simulation/video encoding/AAA Games etc. run slower?
There's no excuse for Apple not providing a way to use these instructions, either via bitcode, or providing a mechanism to detect relevant CPU metadata. At a minimum, I expect hackers to take advantage of this to speed up their own applications, as well as open source forks.
GPUs are external processors, that require DMA transport, and can perform useful jobs in parallel with the CPU, and are produced by multiple vendors. This almost certainly requires a HAL, and there's little impact of making dynamic libraries calls compared to the benefit received.
Compare this to a co-processor, that basically can be called directly from CPU instructions, without DMA and a remote instruction queue. Something like this gains great benefits from a compiler intrinsic, as a function call, particularly as a dynamic library is quite expensive. This compiler intrinsic can just map to bitcode, which can then map to whatever at compile-time on the app store deployment side.
Why would people not want this option? Why should this only be available to Apple's internal products? Why should Photoshop/Lightroom/Blender/physics simulation/video encoding/AAA Games etc. run slower?
There's no excuse for Apple not providing a way to use these instructions, either via bitcode, or providing a mechanism to detect relevant CPU metadata. At a minimum, I expect hackers to take advantage of this to speed up their own applications, as well as open source forks.
Exactly this. You buy hardware for the API that ships with the software on it these days.
Or if you do choose to rely on implementation details of specific hardware, you have to take on responsibility for managing breaking changes yourself. e.g. Implementing your own compatibility checks and fallback code paths, or knowing you will have to tweak and recompile for each hardware version.
Problem is that these matrix instructions are way more useful that the use cases they've catered for in their library. You could have a library that has a function call for each matrix instruction, but that's not worth the penalty of a function call. If your game, graphics software, physics simulation, render farm, video encoder etc can get a boost from these commands, it should be permitted, not just the software that Apple makes. Not sure why people are hostile to this.
But if it is undocumented Apple has no obligation to support these instructions in user code. In fact you will most likely get a fail trying to submit this to the app store.
Apple has always been merciless on this. If you use stuff they don't support and your software later breaks, that is your problem.
This is different from say Microsoft which has added support for quirks and bugs in software in their OS updates to keep old software running.
Apple doesn't do that, and I think it is the right decision. It makes developers used to following standards. But it is a tough call as it really pisses off people when their software breaks due to OS updates.
Apple has always been merciless on this. If you use stuff they don't support and your software later breaks, that is your problem.
This is different from say Microsoft which has added support for quirks and bugs in software in their OS updates to keep old software running.
Apple doesn't do that, and I think it is the right decision. It makes developers used to following standards. But it is a tough call as it really pisses off people when their software breaks due to OS updates.
I wonder if Apple would be as concerned with backward compatibility if they had Microsoft's reach in governments and enterprise. Enormous organizations are probably not receptive to "it's your own fault for using undocumented behavior" when negotiating contracts.
You don't negitiate contracts about OS promising backward compat. Not even MS, which has a quite good backward compat story for windows, does that. Plus pro apps are often specified and tested to run on a specific OS version.
> Apple doesn't do that, and I think it is the right decision. It makes developers used to following standards. But it is a tough call as it really pisses off people when their software breaks due to OS updates.
But why not support it at the bitcode level? Matrix operations have broad applicability, so they are just slowing down the software for customers using third party products that could use those instructions beyond what the library offers.
But why not support it at the bitcode level? Matrix operations have broad applicability, so they are just slowing down the software for customers using third party products that could use those instructions beyond what the library offers.
> Once the genie is out of the bottle, backward compatibility means imitating the old instruction set.
Since the instructions are completely unofficial and only available through what I assume is a dynamically linked system framework, there is 0 chance Apple will care.
For prior art, when Apple silently decided to change the ABI of the syscall underlying gettimeofday(2) in Sierra, projects who made raw syscalls despite that never being officially supported (cough cough golang) had to fix their stuff.
Since the instructions are completely unofficial and only available through what I assume is a dynamically linked system framework, there is 0 chance Apple will care.
For prior art, when Apple silently decided to change the ABI of the syscall underlying gettimeofday(2) in Sierra, projects who made raw syscalls despite that never being officially supported (cough cough golang) had to fix their stuff.
That's fair. But i could see these instructions being useful for photoshop, lightroom, image magick, rendering software, video encoders, games physics etc. way beyond what their library offers. And if wager that Apple will be using those instructions under the hood of their own products.
> But i could see these instructions being useful for photoshop, lightroom, image magick, rendering software, video encoders, games physics etc. way beyond what their library offers.
So?
> And if wager that Apple will be using those instructions under the hood of their own products.
Which they can, whether through the public library they offer or through one of their private frameworks.
So?
> And if wager that Apple will be using those instructions under the hood of their own products.
Which they can, whether through the public library they offer or through one of their private frameworks.
Apple doesn’t support macOS downgrades on their hardware, because macOS updates often incorporate firmware updates, and there’s no guarantee of backwards compatibility in those either.
This almost never comes up, because they typically don’t need to break things, but I can absolutely envision them performing microcode updates that make the CPU incompatible with previous releases.
They owe nothing to anyone regarding these instructions, and they will act accordingly. That’s what unsupported means here: not the PC or Android meanings where everyone has to kludge it in, but the Apple meaning where they will release without warning an OS point release version 15.2.7 that shatters an entire ecosystem built on unsupported APIs.
These instructions are just unsupported APIs in Silicon. Use them at your peril.
This almost never comes up, because they typically don’t need to break things, but I can absolutely envision them performing microcode updates that make the CPU incompatible with previous releases.
They owe nothing to anyone regarding these instructions, and they will act accordingly. That’s what unsupported means here: not the PC or Android meanings where everyone has to kludge it in, but the Apple meaning where they will release without warning an OS point release version 15.2.7 that shatters an entire ecosystem built on unsupported APIs.
These instructions are just unsupported APIs in Silicon. Use them at your peril.
Maybe but if your encoder runs twice as fast, or your game physics can handle twice the amount of objects, or your renders take half the time, you can bet vendors world at least provide unsupported patches of their software. If we don't see this in the wild, I'll be shocked.
The same vendors that can’t be bothered to port their software to a macOS released in the past three years, require us to run 32-bit apps for plugins and DRM dongles, and refuse to implement platform-native Metal in games because twice the performance of OpenGL isn’t worth it to them since they perceive Macs as being slow?
Somehow I doubt it. But hey. Maybe M1 will change their minds. Maybe something more powerful will. A 27” iMac that can play FPS games at 60fps/5K/HDR with virtual Atmos surround out of the box? Near-effortless console porting to a future Silicon AppleTV, with gameplay compatibility across both devices? They’ll realize what they’re missing out on eventually :) But they’re not going to code M1 assembler until they do, and it’s easier for them to use Apple’s frameworks than play games with the CPU.
(Yes, ffmpeg will probably do this. Good for them! And I’m sure Adobe will give it serious consideration - and then reject it, since it would get their apps rejected from the app stores and harm their relationship with Apple.)
Somehow I doubt it. But hey. Maybe M1 will change their minds. Maybe something more powerful will. A 27” iMac that can play FPS games at 60fps/5K/HDR with virtual Atmos surround out of the box? Near-effortless console porting to a future Silicon AppleTV, with gameplay compatibility across both devices? They’ll realize what they’re missing out on eventually :) But they’re not going to code M1 assembler until they do, and it’s easier for them to use Apple’s frameworks than play games with the CPU.
(Yes, ffmpeg will probably do this. Good for them! And I’m sure Adobe will give it serious consideration - and then reject it, since it would get their apps rejected from the app stores and harm their relationship with Apple.)
First, let's leave games aside, because Mac isn't a profitable target for game publishers. Same with engineering. As for 3d rendering, video editing, compositing though, there's some scope for Mac to reclaim lost market share.
> it’s easier for them to use Apple’s frameworks than play games with the CPU
The things I'm talking about are broader than what their accelerated libraries support. Matrix operations are ubiquitous across heavy-weight computation, including stress analysis, physics simulation, machine learning, video and sound processing, compression, ray-tracing.. why would Apple limit their market? There's a whole host of 3rd party applications that would be augmented with this co-processor. And it should be trivial to add a compiler intrinsic that compiles to some bitcode primitive that can ensure it works across all devices and OS's... or just provide a mechanism to detect the extension.
> And I’m sure Adobe will give it serious consideration - and then reject it, since it would get their apps rejected from the app stores and harm their relationship with Apple.
This is bad for Apple. And bad for the consumer. Final Cut Pro isn't the only tool that video professionals need to use, and you can bet FCP will be using the coprocessor directly under the hood.
> it’s easier for them to use Apple’s frameworks than play games with the CPU
The things I'm talking about are broader than what their accelerated libraries support. Matrix operations are ubiquitous across heavy-weight computation, including stress analysis, physics simulation, machine learning, video and sound processing, compression, ray-tracing.. why would Apple limit their market? There's a whole host of 3rd party applications that would be augmented with this co-processor. And it should be trivial to add a compiler intrinsic that compiles to some bitcode primitive that can ensure it works across all devices and OS's... or just provide a mechanism to detect the extension.
> And I’m sure Adobe will give it serious consideration - and then reject it, since it would get their apps rejected from the app stores and harm their relationship with Apple.
This is bad for Apple. And bad for the consumer. Final Cut Pro isn't the only tool that video professionals need to use, and you can bet FCP will be using the coprocessor directly under the hood.
> Matrix operations are ubiquitous across heavy-weight computation, including stress analysis, physics simulation, machine learning, video and sound processing, compression, ray-tracing.. why would Apple limit their market?
Apple offers frameworks that accelerate the relevant operations, and has over time shown an ongoing willingness to limit their market, even when that’s seen as unpalatable or unprofitable or unhelpful to themselves or their users. I assume that the frameworks offering acceleration will expand their uses somewhat over time. I expect those frameworks to be the only compelling solution for having portable matrix acceleration across the complete spectrum of Apple Silicon hardware revisions over time, especially if Apple revises their undocumented operations and updates their frameworks annually (which they can easily do, since each year’s hardware release already demands the latest macOS to boot at all).
Apple offers frameworks that accelerate the relevant operations, and has over time shown an ongoing willingness to limit their market, even when that’s seen as unpalatable or unprofitable or unhelpful to themselves or their users. I assume that the frameworks offering acceleration will expand their uses somewhat over time. I expect those frameworks to be the only compelling solution for having portable matrix acceleration across the complete spectrum of Apple Silicon hardware revisions over time, especially if Apple revises their undocumented operations and updates their frameworks annually (which they can easily do, since each year’s hardware release already demands the latest macOS to boot at all).
I admit it's been a while since I bought a new Mac but can you not boot older macOS from an external drive anymore?
You can, but the older version of macOS has to be new enough to support the hardware you're running, and it has to be compatible enough to allow booting.
In general, the oldest macOS you can boot on any given Mac is "the version the Mac shipped from the factory with". So, for example, the 2017 iMac Pro can't boot a 2016 or earlier macOS.
This is still unsupported, mind you — if your 2017 iMac Pro is firmware updated to 2020 and you're booting a macOS from 2018 on it, you may encounter random bugs and/or crashes and/or mysterious issues due to assumptions made by the older macOS that are no longer entirely valid on 2020 firmware — but in general, it should work as long as the macOS is newer than the hardware.
(You probably won't be able to boot an M1 macOS external drive on an Intel Mac, and vice versa, for other reasons unrelated to this. In that scenario, Migration Assistant is probably your only choice anyways.)
In general, the oldest macOS you can boot on any given Mac is "the version the Mac shipped from the factory with". So, for example, the 2017 iMac Pro can't boot a 2016 or earlier macOS.
This is still unsupported, mind you — if your 2017 iMac Pro is firmware updated to 2020 and you're booting a macOS from 2018 on it, you may encounter random bugs and/or crashes and/or mysterious issues due to assumptions made by the older macOS that are no longer entirely valid on 2020 firmware — but in general, it should work as long as the macOS is newer than the hardware.
(You probably won't be able to boot an M1 macOS external drive on an Intel Mac, and vice versa, for other reasons unrelated to this. In that scenario, Migration Assistant is probably your only choice anyways.)
I use Mavericks because I like Mavericks, and a few months ago I discovered something bizarre: the mere act of booting a Mavericks installation USB on my 2014 Macbook Air appears to wipe the hard drive. As in, I'm not running the installer and I'm not opening Disk Utility, I'm merely telling the computer to to boot from the USB key. It seems that by the time I've reached the language select screen, the internal hard drive's filesystem is toast.
I did this several times because I couldn't believe what I was seeing. I can't say for sure where the source of the problem lies—maybe something is weird with my installation media—but my theory is that a firmware and/or microcode update subtly broke something in Apple's old installers, possibly when they added APFS support. I looked into downgrading the firmware, but it seems to be impossible from what I can tell[1].
Mind you, outside of this one quirk the OS and installer run fine.
---
1: https://forums.macrumors.com/threads/guide-how-to-get-back-o...
I did this several times because I couldn't believe what I was seeing. I can't say for sure where the source of the problem lies—maybe something is weird with my installation media—but my theory is that a firmware and/or microcode update subtly broke something in Apple's old installers, possibly when they added APFS support. I looked into downgrading the firmware, but it seems to be impossible from what I can tell[1].
Mind you, outside of this one quirk the OS and installer run fine.
---
1: https://forums.macrumors.com/threads/guide-how-to-get-back-o...
Good to know, I hope you didn't lose any non-backed-up data!
Oh, yes, nothing important! Not only do I keep lots of backups, but the laptop is very much a secondary machine to my desktop, especially in the age of COVID. All I lost was the few hours it takes to reinstall my apps and such. :)
No M1 Mac can boot any OS prior to Big Sur - there simply isn't any binaries for it to run.
With intel macs you can boot and install older macOS's up to a certain point. Like the previous poster says, it's about what firmware versions you have.
For example, if your mac came with Sierra, and through the regular usage and update, you patched it up to Mojave, and installed (or was forced to install by the way, there is no way to reject certain types of updates) firmware updates, you find that all of a sudden you can't re-install Sierra anymore. You'd need to install whatever version your firmware now supports.
This behaviour is undocumented and it's case by case - for each mac hardware there is a firmware for it - there are lots of people online who have figured out patch levels are for each revision of hardware, etc - but it's more trial and error to see what older version of macOS you can downgrade to.
With intel macs you can boot and install older macOS's up to a certain point. Like the previous poster says, it's about what firmware versions you have.
For example, if your mac came with Sierra, and through the regular usage and update, you patched it up to Mojave, and installed (or was forced to install by the way, there is no way to reject certain types of updates) firmware updates, you find that all of a sudden you can't re-install Sierra anymore. You'd need to install whatever version your firmware now supports.
This behaviour is undocumented and it's case by case - for each mac hardware there is a firmware for it - there are lots of people online who have figured out patch levels are for each revision of hardware, etc - but it's more trial and error to see what older version of macOS you can downgrade to.
> For example, if your mac came with Sierra, and through the regular usage and update, you patched it up to Mojave, and installed (or was forced to install by the way, there is no way to reject certain types of updates) firmware updates, you find that all of a sudden you can't re-install Sierra anymore. You'd need to install whatever version your firmware now supports.
I have never seen any record of this happening. I'm really quite sure you've always been able to install the earliest OS that your model of Mac shipped with.
The odd situation I mentioned above is the closest I've ever seen to a firmware update causing problems (if that's even the cause), and even then the OS runs fine.
I have never seen any record of this happening. I'm really quite sure you've always been able to install the earliest OS that your model of Mac shipped with.
The odd situation I mentioned above is the closest I've ever seen to a firmware update causing problems (if that's even the cause), and even then the OS runs fine.
Yah even if it hasn't happened to you yet, be aware that this is something that happens.
There actually is a table of firmware versions, etc - once you upgrade your old Mac hardware past a certain firmware, you will no longer be able to run certain versions of macOS "before" the firware was supported.
This isn't just between "major" versions, also between "minor" versions of macOS
For example, between the current version of macOS Sierra, which is on 10.12.6 (6 being the minor version), if you try to install for example 10.12.x where x is below 6, etc - you might get the infamous ? or the circle with a slash through it on bootup.
There actually is a table of firmware versions, etc - once you upgrade your old Mac hardware past a certain firmware, you will no longer be able to run certain versions of macOS "before" the firware was supported.
This isn't just between "major" versions, also between "minor" versions of macOS
For example, between the current version of macOS Sierra, which is on 10.12.6 (6 being the minor version), if you try to install for example 10.12.x where x is below 6, etc - you might get the infamous ? or the circle with a slash through it on bootup.
But I have run 10.12.0 after installing 12.12.6. What models does this happen on?
...Wait, no, I can't say for sure if I'd ever run 10.12.6 specifically. But I ran stock Sierra (10.12.0) after running Mojave, and it's hard to imagine that would result in a more compatible firmware...
[deleted]
Apple isn't going to support your code using custom instructions.
I forget that this is how Apple's operates. I can just see a bunch of software that is really popular on Mac (e.g. image + video processing, 3d) that could get a massive speed boost with unfettered access to these instructions. Their library ain't going to cater for a fraction of the useful stuff that a matrix coprocessor can do.
The cited announcement here was for ARMv8-M only, the microcontroller variant of ARMv8 for embedded systems.
From the linked EETimes article:
>> Arm is opening up its instruction set to customers’ customized instructions for Cortex M cores
>> For Cortex-A cores, Arm is still a long way from offering any customizable instructions
From https://www.arm.com/why-arm/technologies/custom-instructions:
> Arm Custom Instructions are a standard feature of the Armv8-M architecture
Apple M1 is not an ARMv8-M core, it's an ARMv8-A core.
I'd very much like to understand the contractual terms under which Apple put custom instructions into their ARMv8-A core, but I don't think it is as simple as "ARM is totally fine with all licensees doing this now." AFAIK, the latest revisions of ARMv8-A still do not permit custom instructions.
Other reasons I don't think AMX is in the category of the publicly announced "custom instructions" part of the ISA:
- arm custom instruction are prohibited from accessing memory, but AMXLD* and AMXST* exist.
- arm custom instructions are prohibited from having their own register states, but AMX has amx0, amx1, amx2 registers.
https://developer.arm.com/architectures/instruction-sets/cus...