Tell HN: macOS Ventura broke the Raspberry Pi Pico ecosystem
12 comments
Clickable links
https://www.raspberrypi.com/news/the-ventura-problem/
https://www.zdnet.com/article/raspberry-pi-pico-has-a-proble...
https://www.tomshardware.com/news/mac-ventura-raspberry-pi-p...
https://mjtsai.com/blog/2022/11/02/the-raspberry-pi-ventura-...
https://blog.adafruit.com/2022/10/31/uploading-uf2-files-wit...
https://www.raspberrypi.com/news/the-ventura-problem/
https://www.zdnet.com/article/raspberry-pi-pico-has-a-proble...
https://www.tomshardware.com/news/mac-ventura-raspberry-pi-p...
https://mjtsai.com/blog/2022/11/02/the-raspberry-pi-ventura-...
https://blog.adafruit.com/2022/10/31/uploading-uf2-files-wit...
> "So while additional writes to the volume will succeed, reading back the data will give the wrong contents," explains Allan.
That's ... not a correct implementation of a USB mass storage device. I'm not surprised it broke; I'm surprised it ever worked at all.
Looking into this a bit more, the RPi bootloader emulates block-level access to a FAT filesystem, but throws away writes to anything other than the contents of a few emulated files.
Filesystem implementations and OS page caches are not written to treat the physical block storage as authoritative and capable of modifying their writes out from under them. To a filesystem or the OS, that just looks like a failing drive.
That's ... not a correct implementation of a USB mass storage device. I'm not surprised it broke; I'm surprised it ever worked at all.
Looking into this a bit more, the RPi bootloader emulates block-level access to a FAT filesystem, but throws away writes to anything other than the contents of a few emulated files.
Filesystem implementations and OS page caches are not written to treat the physical block storage as authoritative and capable of modifying their writes out from under them. To a filesystem or the OS, that just looks like a failing drive.
Agreed.
Reading TFA, I couldn’t help but be bewildered that anyone at the Raspberry Pi foundation ever thought this was a sane move. It’s truly one of the worst design decisions they could have made. It’s hard to express my surprise at how sophomoric this is. Blaming _any_ OS vendor for such a basic architectural deformity seems lame.
It’s a wonder it hasn’t broken before now.
Reading TFA, I couldn’t help but be bewildered that anyone at the Raspberry Pi foundation ever thought this was a sane move. It’s truly one of the worst design decisions they could have made. It’s hard to express my surprise at how sophomoric this is. Blaming _any_ OS vendor for such a basic architectural deformity seems lame.
It’s a wonder it hasn’t broken before now.
I’m not happy that this change has happened; however, since learning about how the BOOTSEL mode works under the hood I’ve been expecting news like this from one of the big two OS vendors.
While a sleek looking use case on the surface, mounting the PI as a fake, non-standards compliant, mass storage device is just a hack.
While a sleek looking use case on the surface, mounting the PI as a fake, non-standards compliant, mass storage device is just a hack.
I'm not sure which part of the standard you think they're violating but the Pi is presenting itself as a USB device. One that accepts a (limited) number of blocks of data, and can respond with blocks of data.
Drag and drop to program isn't the same thing as flashing an EPROM on a chip burner like we're still back in the 80's, but that's because time's moved on since then.
Drag and drop to program isn't the same thing as flashing an EPROM on a chip burner like we're still back in the 80's, but that's because time's moved on since then.
> I'm not sure which part of the standard you think they're violating
The part where you're not supposed return a successful Command Status claiming a write succeeded, when actually, you're dropping writes — to anything but a few specific blocks, and only if you see those writes in the order you expect — on the floor.
The part where you're not supposed return a successful Command Status claiming a write succeeded, when actually, you're dropping writes — to anything but a few specific blocks, and only if you see those writes in the order you expect — on the floor.
Oh whew, that's just some layer 7 concerns. I'd be interested in seeing where that's (that it must actually write those blocks) actually defined by in the spec and what language they use. I was worried my Pico was sending invalid blocks that didn't properly parse and was relying on undefined behavior to implement flashing.
It may be a hack but it isn’t a new idea. MBed DAPlink has had drag and drop programming for a long time. The STlink programmer also does this, so do the Segger J-links.
At this point you must know that if you buy a mac you are surrendering any claim on your third party apps surviving between updates. Irritating, yes, but not surprising.
Apple isn't remotely to blame for this; the RPi bootloader exposes a broken, incorrect emulation of the USB mass storage protocol.
The only way this ever worked was pure luck, relying on the OS:
- Writing file blocks in the desired order — which is not guaranteed.
- Writing filesystem metadata in the desired order — also not guaranteed.
- Not caching any important pages, so that the VM page cache isn't out-of-sync with the block device after it silently discards writes.
- Not caching any important filesystem metadata, so that the filesystem implementation isn't out-of-sync.
I'm surprised (1) anyone thought this was anything other than a risky hack, and (2) not only did RPi actually ship this broken design, but did so in ROM, and thus cannot patch it.
[edit]
Surprised at the downvotes for posting a technical rebuttal to an inflammatory comment.
As someone with a few decades in OS/kernel development, this doesn't seem like a controversial or even difficult question. The implementation is broken and cannot work reliably.
If you disagree, then please explain why you believe the USB mass storage emulation is not broken.
The only way this ever worked was pure luck, relying on the OS:
- Writing file blocks in the desired order — which is not guaranteed.
- Writing filesystem metadata in the desired order — also not guaranteed.
- Not caching any important pages, so that the VM page cache isn't out-of-sync with the block device after it silently discards writes.
- Not caching any important filesystem metadata, so that the filesystem implementation isn't out-of-sync.
I'm surprised (1) anyone thought this was anything other than a risky hack, and (2) not only did RPi actually ship this broken design, but did so in ROM, and thus cannot patch it.
[edit]
Surprised at the downvotes for posting a technical rebuttal to an inflammatory comment.
As someone with a few decades in OS/kernel development, this doesn't seem like a controversial or even difficult question. The implementation is broken and cannot work reliably.
If you disagree, then please explain why you believe the USB mass storage emulation is not broken.
I don’t understand any of the technical details involved here, but reading your comments makes me seriously wondering, does RPi rely on these everywhere, or just when plugged to a Mac? Because all three things seem like pretty reasonable optimisation opportunities for a file manager to implement, and given how USB storage devices have been around I’m very surprised if they managed to get away without issues until now.
In case you haven't seen the UF2 file format, it's specifically designed to be robust w.r.t ensuring that the content makes it to the target intact despite common OS quirks.
https://github.com/microsoft/uf2
https://github.com/microsoft/uf2
- Data is divided into 512 byte blocks, which lines up with the mass storage transfer size.
- Each block has an identifying magic sequence, a payload address, and size, which can be used to identify the destination.
- This allows the emulated storage to ignore metadata blocks and to handle out-of-order and repeated blocks.The UF2 format is a hack that can be made to generally work (e.g. it relies on things like per-block beginning and ending magic numbers to identify blocks that should be written to flash, which means isn't dependent on seeing FAT table writes in a particular order).
UF2 is just the file format, though. The mechanism for reading/writing UF2 files is via the emulated USB mass storage, which is fundamentally broken.
The USB storage device responds to SCSI write commands with a success status, but actually throws away data, such that a successive read does not return the previously written blocks.
To an OS or filesystem, that's just a failing drive. You might get lucky and things might work.
UF2 is just the file format, though. The mechanism for reading/writing UF2 files is via the emulated USB mass storage, which is fundamentally broken.
The USB storage device responds to SCSI write commands with a success status, but actually throws away data, such that a successive read does not return the previously written blocks.
To an OS or filesystem, that's just a failing drive. You might get lucky and things might work.
Are there any workarounds?
There are at least two workarounds to the problem. But essentially, they both do the same thing: take the macOS Finder out of the picture, and copy the UF2 to the virtual volume without involving those pesky extended attributes and additional resource forks.I tried this.
I still had constant problems including junk displaying as the list of files on the device.
I still had constant problems including junk displaying as the list of files on the device.
The Finder on macOS Ventura can't be used to flash firmware onto a Raspberry Pi Pico anymore. The ecosystem is fine.
> If you pass the -X flag to the cp command you can successfully copy a UF2 onto your board
> Alternatively, if the errors bother you, you can use the rsync command rsync blink.uf2 /Volumes/RPI-RP2/ which will also copy your UF2 to the board.
> we offer something called picotool (...) it can be used to load binaries onto your Raspberry Pi Pico
I was expecting some integral part of the Raspberry Pi software ecosystem to be dependent on the current version of macOS because that's what the title implies. We regularly read stories about crazy dependencies here. "macOS Ventura is incompatible with the Raspberry Pi Pico" would not imply the crazy dependency I expected.
But then after reading the submission I think "Oh no, the Raspberry Pi Pico can't currently be used with macOS - will they have to patch the bootloader? Do mac users need to use special programmers now? Will they sell a new version that's compatible again?"
Nope, wrong again. It looks like the Finder on macOS can't be used for flashing firmware anymore. The workaround is almost trivial. Sure, it's a bit annoying and should somehow be fixed, but it's not the end of the world at all. The ecosystem hasn't been broken.
> If you pass the -X flag to the cp command you can successfully copy a UF2 onto your board
> Alternatively, if the errors bother you, you can use the rsync command rsync blink.uf2 /Volumes/RPI-RP2/ which will also copy your UF2 to the board.
> we offer something called picotool (...) it can be used to load binaries onto your Raspberry Pi Pico
I was expecting some integral part of the Raspberry Pi software ecosystem to be dependent on the current version of macOS because that's what the title implies. We regularly read stories about crazy dependencies here. "macOS Ventura is incompatible with the Raspberry Pi Pico" would not imply the crazy dependency I expected.
But then after reading the submission I think "Oh no, the Raspberry Pi Pico can't currently be used with macOS - will they have to patch the bootloader? Do mac users need to use special programmers now? Will they sell a new version that's compatible again?"
Nope, wrong again. It looks like the Finder on macOS can't be used for flashing firmware anymore. The workaround is almost trivial. Sure, it's a bit annoying and should somehow be fixed, but it's not the end of the world at all. The ecosystem hasn't been broken.
[deleted]
It seems a little hyperbolic to suggest that a Ventura Finder bug "broke the entire Raspberry Pi Pico ecosystem." Works on my machine on Monterey, and I suspect there are a lot of users that aren't on macOS at all that are running fine today.
Honestly, Ventura is the worst MacOS update in a long time. For example, I’ve experienced constant WindowServer crashes until I stopped using full screen for apps.
https://developer.apple.com/forums/thread/719033
There are also other just stupid things, such as if you use a rotating photo album for desktop backgrounds, you need to select a single image to set ‘fit to screen’ then deselect the album. There seems to be little to know QA on this release.
There are also other just stupid things, such as if you use a rotating photo album for desktop backgrounds, you need to select a single image to set ‘fit to screen’ then deselect the album. There seems to be little to know QA on this release.
I wonder why I haven't heard about it, but damn, the stage manager implementation is awful. I don't mean the design, that's a matter of taste, but there are constant glitches, windows jumping away from the mouse (sometimes outside the screen) and, my absolute favorite, window management becoming more buggy after turning stage manager off: The window movement would lag behind the mouse and when attempting to resize the window would violently wobble and scale to wildly different sizes.
I'm on a dual monitor, dual resolution setup with the monitors not perfectly aligning, I guess one or more of these things could have something to do with my experience. But I'm pretty sure that's a setup they support and should've tested and more than 0 people should have run into these issues during the beta phase. So what's the excuse? If stage manager is a preview of the quality of new features the future looks bad.
I'm on a dual monitor, dual resolution setup with the monitors not perfectly aligning, I guess one or more of these things could have something to do with my experience. But I'm pretty sure that's a setup they support and should've tested and more than 0 people should have run into these issues during the beta phase. So what's the excuse? If stage manager is a preview of the quality of new features the future looks bad.
Apple hater here. They didn't break the whole ecosystem. They just broke the easiest workflow for using it.
The whole uf2 thing feels like a big hack.
Yep. It's essentially trying to use POSIX API to do firmware updates. This will come in handy if you have a remote server to manage a fleet of OTA firmware updates of course. That's why it should have had been FUSE based in the first place.
Since when Apple cares of non Apple things?
I dunno, maybe when you become is big/complex you stop prioritizing supporting niche fake non-standards compliant mass storage devices? You'd never do that though I'm sure.
Microsoft did everything possible to keep backward compatibility for years and years even for niche software.
Someone says only stable linux ABI is win32...
https://news.ycombinator.com/item?id=32471624
https://news.ycombinator.com/item?id=32471624
it also broke jedi on my neovim setup…
That means micropython and circuitpython don't work on Macos Ventura.
Apparently the Raspberry Pi Foundation have not even succeeded in contacting whoever it is inside Apple who works on Finder - which is what has done the breaking.
Six weeks down the track and Apple has just left it all as busted. Thanks Apple.
https://www.raspberrypi.com/news/the-ventura-problem/
https://www.zdnet.com/article/raspberry-pi-pico-has-a-problem-with-macs-using-macos-ventura/
https://www.tomshardware.com/news/mac-ventura-raspberry-pi-pico-problem
https://mjtsai.com/blog/2022/11/02/the-raspberry-pi-ventura-problem/
https://blog.adafruit.com/2022/10/31/uploading-uf2-files-with-macos-13-0-ventura-apple-microbit_edu-raspberry_pi-circuitpython/