Linux's SystemV Filesystem Support Being Orphaned(phoronix.com)
phoronix.com
Linux's SystemV Filesystem Support Being Orphaned
https://www.phoronix.com/news/Linux-SysV-File-System-Orphan
8 comments
Just to prevent inevitable confusion, SystemV filesystem is different than SystemV init which you may be thinking of. SysV init isn't part of the kernel and while it's been replaced with systemd in many distributions, it's not impacted by this change.
The SysV name is also attached to a set of old and strange interprocess communication primitives:
https://man7.org/linux/man-pages/man7/sysvipc.7.html
Unix has a bigger range of IPC mechanisms than i think most people realise. Everyone knows about internet sockets and pipes. Most people know about shared memory, named pipes, and unix domain stream sockets. But how many people know about unix domain datagram sockets, or POSIX message queues?
https://man7.org/linux/man-pages/man7/sysvipc.7.html
Unix has a bigger range of IPC mechanisms than i think most people realise. Everyone knows about internet sockets and pipes. Most people know about shared memory, named pipes, and unix domain stream sockets. But how many people know about unix domain datagram sockets, or POSIX message queues?
In a university class, we had an assignment that asked us to write a program that would communicate with its copies over POSIX message queues. I remember grepping through the source of all the software I had on my Gentoo installation, and couldn’t find a single one that had any sort of usage or support for POSIX message queues. Does anyone know of even a single app that uses them?
I don't know of anything other than one I wrote back in the late 2000s that let me coordinate a web service with process pool of ERP instances. When I wrote it I remember thinking that ftok could have real issues on a large filesystem. I never ran into any problems so I sort of put the issue out of my mind.
There are two things: SysV IPC and POSIX IPC, both of which have message queues. In the SysV case this feature probably made sense because it was designed at a time when BSD sockets were not yet the universaly accepted API. POSIX equivalent probably exists mostly for completeness.
One might say that the whole of POSIX IPC is simply SysV IPC with sane and "unix-like" API (ie. names instead of "IPC keys").
One of the problems with POSIX IPC is that you cannot really use it if you want to be portable, because on BSD and derived systems the implementation is best described as also existing only for completeness.
One might say that the whole of POSIX IPC is simply SysV IPC with sane and "unix-like" API (ie. names instead of "IPC keys").
One of the problems with POSIX IPC is that you cannot really use it if you want to be portable, because on BSD and derived systems the implementation is best described as also existing only for completeness.
They tend to be used more in real time systems I think.
Too bad, I had a use for this and will be sad to see it go. I used this file system many times to copy data from diskettes created and formatted for CoherentOS. Will I need it in the future ? If I can find a nice ISA 486 maybe :)
https://www.autometer.de/unix4fun/coherent/
https://www.autometer.de/unix4fun/coherent/
Needing a utility to manage ~2MB of data for transport via legacy storage devices isn't the same thing as "I need a kernel filesystem driver", though.
A python wrapper for simple formats like this around a block interface (fuse mount, etc...) is a few afternoons of work for someone. I bet anything someone's already written this.
A python wrapper for simple formats like this around a block interface (fuse mount, etc...) is a few afternoons of work for someone. I bet anything someone's already written this.
Being orphaned doesn’t mean it’s being removed. Some subsystems will eventually get a new maintainer such as the floppy driver.
Hopefully someone reintroduces it as a loadable FUSE module for cases such as this.
Plenty of fish in the sea, really...
FWIW, I've been running Linux since 1993 or 94, and have never felt a need for this filesystem. Compatibility with other UNIX systems always meant FTP or NFS and then application support for whatever file formats were in use.
I used it all the time in the 90s and early 2000s to mount disk created on System V based systems, most notably SCO Unix, which was very popular at the time. Links were slow, NFS was sorta unreliable and had limitations, and FTP doesn't preserve filesystem attributes among other issues, and some use cases just require the actual filesystem, so it was often easier just to move/send a physical disk. I think I've even used it in the last 5 years to do some data recovery on an old SVR4 system for a friend. I won't be sad if it goes away, tho...these days I'd probably just image the disk and mount it on a VM running SCO or older Linux or some such. Time marches on...
Same. I used it periodically to recover data from expired hardware. Kids today don't have any idea that you couldn't just take a drive from an installed sco system and just boot it up on any other motherboard and hard drive controller. (well, there are wizard incantations that you won't know and won't be able to figure out in time) So couldn't boot the old system on the old dead hardware or on new hardware, couldn't buy a new copy of the old hardware... but you could mount the disk and get everything that matters.
A fuse implementation would be good enough today though. You're not doing actual production work on htfs today no matter how special the circumstances.
...and did anyone ever automate parsing the divvy partition table?
A fuse implementation would be good enough today though. You're not doing actual production work on htfs today no matter how special the circumstances.
...and did anyone ever automate parsing the divvy partition table?
You're not doing actual production work on htfs today no matter how special the circumstances.
Someone is, apparently. Some shop called Xinuos apparently bought part of the corpse (the not lawsuit obsessed, not fraud committing part, presumably) and is still selling SCO OpenServer and Unixware. That's some serious dead-ender computing. But hey...people still run OS/2 in production.
Divvy...haven't thought about that in not long enough.
Someone is, apparently. Some shop called Xinuos apparently bought part of the corpse (the not lawsuit obsessed, not fraud committing part, presumably) and is still selling SCO OpenServer and Unixware. That's some serious dead-ender computing. But hey...people still run OS/2 in production.
Divvy...haven't thought about that in not long enough.
Imagine telling them we could boot the machine from a SCSI tape…
Or manually repair the borked inode table with adb/fsdb.
You have to do it manually. You don't dare let fsck -y do it.
I confess I have never productively used either but have destructively used fsck.
I confess I have never productively used either but have destructively used fsck.
There are many distributions based on long-term stable kernels that will have this driver. RedHat 8 and friends are supported until 2029 on a v4 kernel.
In the (far) future, just use a past distro to see them. Removal now wipes functionality from the market very, very slowly.
In the (far) future, just use a past distro to see them. Removal now wipes functionality from the market very, very slowly.
I understand it'd be more important early on (as in "1990's"), but things like NFS abstracted away the underlying filesystem (at least where the machine that has the disk is alive). I mentioned the historical preservation aspect, and, following a comment, I'd say it belongs more under FUSE than inside the mainline kernel.
> NFS belongs more under FUSE
besides any philosophical argument, doesn't it still crap up your whole system whenever the network flakes out? (unlike, i imagine, any FUSE driver)
besides any philosophical argument, doesn't it still crap up your whole system whenever the network flakes out? (unlike, i imagine, any FUSE driver)
Not sure what you might mean by 'crap out'. If you have soft-mounted a remote export and the network goes away, then your applications experience i/o errors which they either handle one way or another or not, which yields silent data corruption, but you have been warned about that. If you have hard-mount'ed an remote export and the network connection to that server goes dark, then any access to files residing on that will stall ('hang') until the network issue is resolved.
What exactly do you want the client to do, when it relies on remote resources and the network isn't reliable? This fortunately is these days much less a problem than in the days of cheapernet (10BASE2).
What exactly do you want the client to do, when it relies on remote resources and the network isn't reliable? This fortunately is these days much less a problem than in the days of cheapernet (10BASE2).
> unlike, i imagine, any FUSE driver
FUSE drivers aren't exempt from hanging syscalls, I've experienced them even with local mounts let alone network ones. I don't know why people keep repeating this.
FUSE drivers aren't exempt from hanging syscalls, I've experienced them even with local mounts let alone network ones. I don't know why people keep repeating this.
Antiquated unixes weren't always connectivity monsters -- some POS with a bunch of serial ports for instance.
FTP and NFS are good, I'd also want ckermit / tip / tar in my toolbag...
Probably not an issue these decades, though.
FTP and NFS are good, I'd also want ckermit / tip / tar in my toolbag...
Probably not an issue these decades, though.
Eh, FTP's port requirements are pretty onerous in today's NAT based world we live in.
Something this old and unchanged should be considered done and perhaps even bug-free. Unfortunately the state of software seems to be pursuing constant change instead of stability.
May be a good in between is somehow hosting these as separate drivers that can be modprobed into modern kernels, and removing them from the normal linux kernel tree. Then, if someone somewhere wants to develop them, or if they are really "done" and "bug-free" then people who want them can still use them while not I guess making maintainers feel upset.
That would mean a stable API for filesystem drivers, which apparently Linux doesn't believe in.
“Old and unchanged” has historically meant relatively little in Linux[1], in part due to a lack of a stable API/ABI. These components are very possibly more buggy than those under continuing development.
[1]: https://www.phoronix.com/news/Linux-Floppy-2021-Regression
[1]: https://www.phoronix.com/news/Linux-Floppy-2021-Regression
The filesystem code itself might be done, but the internal Linux APIs that code interacts with are not and change over time. And every time those APIs are changed, the SysV fs code might also have to be updated and tested, except apparently you can't even properly test it anymore.
That's a lot of work for something pretty much nobody uses. And in the end the SysV FS support might not even work anymore because of oversights during the API updates.
That's a lot of work for something pretty much nobody uses. And in the end the SysV FS support might not even work anymore because of oversights during the API updates.
When you test a file system are you testing it via the libc APIs, creating, deleting, checking filesystem objects and so on? Or is it done at a lower level API (that I’d hope is the same across filesystems, different features excepted)
For filesystems the former. Having test code running in kernel space is relatively uncommon, though some kernel subsystems do that.
"can't even properly test it anymore" almost certainly means not that it is impossible for the maintainer, but more like they have better things to spend their time on.
"can't even properly test it anymore" almost certainly means not that it is impossible for the maintainer, but more like they have better things to spend their time on.
> For filesystems the former.
Well... Then, if it builds, it can at least be tested with the same tools that test every other filesystem.
Well... Then, if it builds, it can at least be tested with the same tools that test every other filesystem.
And it's not as if older versions of the kernel cease to exist when a new one lands, too. Worst case, it'll always be there in source control, just a checkout and a build away.
I didn't even know there was a sysv file system... when I first started using Linux, I think my choices were ext2, fat, or ufs.
I remember using the Minix filesystem on floppies. ext{,2} was a bit too heavyweight for a floppy. And UMSDOS on my MS-DOS6.22/Wfw3.11 FAT16 partition. And HPFS so I could mount my OS/2 2.0 partition (EMX on the Hobbes OS/2 CD-ROM was my gateway drug to Linux.)
xiafs sounded intriguing, but I never actually used it.
Of all the filesystems currently in the mainline Linux kernel, the most useless (surely) must be BFS (not BeFS) – https://www.kernel.org/doc/html/next/filesystems/bfs.html
xiafs sounded intriguing, but I never actually used it.
Of all the filesystems currently in the mainline Linux kernel, the most useless (surely) must be BFS (not BeFS) – https://www.kernel.org/doc/html/next/filesystems/bfs.html
Hey, I support a system that 4 complete rewrites ago used BFS!
Even today, Open BSD use FFS.
You can mount ext2/3/4 and FAT but pretty sure only FFS is supported for the root file system.
There was also Minix, used in the very early days.
My undergrad operating systems class used Minix
Linux foundation has BILLIONS of dollars and hundreds of global companies supporting it. You're telling me there's no way they can even see if they can mount a DISK or even DISK IMAGE in a specific format? I simply don't buy it.
I could buy "It's a waste of time, nobody's using the code, we are pruning legacy code for the Rust rewrite, etc." But not that it's impossible to test. That doesn't seem truthful.
I could buy "It's a waste of time, nobody's using the code, we are pruning legacy code for the Rust rewrite, etc." But not that it's impossible to test. That doesn't seem truthful.
This is a good thing. There's a constant struggle between people who want Linux to support their ancient hardware, and people who have to work with the kernel source regularly, and would like to speed up compile times and reduce git clone times. But it's undeniable that the balance needs to be struck, and old hardware needs to be deprecated at some point. I think the tipping point should be when 99% of people who need some feature will be perfectly fine with an outdated kernel. And, I think this case qualifies.
From an interoperability and preservation perspective, would it be viable to keep it, provided there is a comprehensive set of tests to make sure it remains safe to use?
Yes, but something like this should be moved to FUSE.
The tests would certainly help that migration.
The tests would certainly help that migration.
Maybe one of these days all filesystems will be moved to FUSE, and Linux will have evolved from a monolithic kernel into a microkernel
Yes, yes, FUSE performance is nothing to write home about – but some people seem to think that io_uring might solve that. L4 is evidence that, in principle, the right microkernel design can perform about as well as monolithic kernels can
Yes, yes, FUSE performance is nothing to write home about – but some people seem to think that io_uring might solve that. L4 is evidence that, in principle, the right microkernel design can perform about as well as monolithic kernels can
Ironically with cloud workloads it already kind of is, specially when it doesn't run bare metal, and has a type 1 hypervisor underneath it.
Or for container based distributions where it is basically there to provide the minimal set of services to run containers, and any other kernel would do, it is only a matter of convenience to ping back on the Linux kernel.
The monolithic kernel folks just haven't noticed it.
Or for container based distributions where it is basically there to provide the minimal set of services to run containers, and any other kernel would do, it is only a matter of convenience to ping back on the Linux kernel.
The monolithic kernel folks just haven't noticed it.
>The monolithic kernel folks just haven't noticed it.
I know may be by sheer number of users most people use them in the cloud or for development but some of us do actually use linux on our everyday machines.
I know may be by sheer number of users most people use them in the cloud or for development but some of us do actually use linux on our everyday machines.
With docker?
> docker?
> GP: development
Probably yes?
Even foregoing development scenarios, containers can be used for everyday Linux tasks with things like toolbox/distrobox (which do support GUIs).
> GP: development
Probably yes?
Even foregoing development scenarios, containers can be used for everyday Linux tasks with things like toolbox/distrobox (which do support GUIs).
So playing microkernels anyway, with worse performance due to the extra monolithic layers.
How are namespaces similar to microkernels? I see them a lot more as glorified chroot; it's not like there's a modprobe namespace or anything.
Namespaces are a tiny part of what means a complete container infrastructure, across all OS layers.
Then there is the whole IPC taking place across containers with much slower APIs and heavier context switch states.
Then there is the whole IPC taking place across containers with much slower APIs and heavier context switch states.
> Ironically with cloud workloads it already kind of is, specially when it doesn't run bare metal, and has a type 1 hypervisor underneath it.
AWS overwhelmingly uses Linux/KVM, not any type 1 hypervisor. There's a few leftover Xen systems, but those are legacy; you get KVM if you provision a new instance.
AWS overwhelmingly uses Linux/KVM, not any type 1 hypervisor. There's a few leftover Xen systems, but those are legacy; you get KVM if you provision a new instance.
Hypervisors and microkernels are meaningfully different, both in terms of history and technology: Microkernels implement one OS by running different userspace components on a minimal kernel, whereas hypervisors run multiple OSes, each with their own kernel, on a minimal hardware multiplexer which ideally provides no abstractions, as opposed to a kernel, which provides many abstractions. Conflating them is both historically wrong and confusing.
"An operating system is a collection of things that don't fit into a language. There shouldn't be one." – Dan Ingalls
That is the beauty of hypervisors, just enough abstraction for a portable language runtime.
That is the beauty of hypervisors, just enough abstraction for a portable language runtime.
FUSE works well enough for Gluster to benefit from RDMA over TCP, it's already pretty good imo
Not as cheap but that's the price of interfaces
Not as cheap but that's the price of interfaces
Well yes and no, SystemV FS support isn't there for performance.
Old and generally unused filesystems shouldn't be living untested in the kernel.
Old and generally unused filesystems shouldn't be living untested in the kernel.
For something like SystemV filesystem support to be truly useful, won't we need a way to format drives using the original implementations?
If we just create the filesystem using the Linux driver, and validate it using the same driver, then what is achieved exactly? For preservation we'd still need access to Xenix systems and a drive of some sort that can be moved between the old system and a more modern computer.
I suppose disk images could be used, but that still seems like only a partial solution.
If we just create the filesystem using the Linux driver, and validate it using the same driver, then what is achieved exactly? For preservation we'd still need access to Xenix systems and a drive of some sort that can be moved between the old system and a more modern computer.
I suppose disk images could be used, but that still seems like only a partial solution.
We should have reference filesystems generated on actual System V machines that are mounted and used in tests.
I highly doubt you're going to be running a modern Linux on any hardware connected to a SysV filesystem. You'll want to use an older Linux that can run on 386's with 8MB of RAM, or similar vintage. And those older Linux's aren't going away.
> For now this file-system support is staying put in the kernel -- albeit stale -- but without someone willing to maintain it runs the risk of further code rot and removal in the future.
Eventually it'll start breaking builds and failing tests. Some code cleanup can be done as a last step before offloading it to another project that focuses on preserving software like this rather than providing a nice OS for usable computers.