Re: [Git Pull] Bcachefs(lwn.net)
lwn.net
Re: [Git Pull] Bcachefs
https://lwn.net/ml/linux-kernel/[email protected]/
6 comments
This is a bit stale, bcachefs is now in the linux-next tree.
I'm aware of most of the problems with the behaviour of Linus in the past, but reading over this post on the mailing list actually comes over pretty civilized. No name calling, no big drama (compared to the past), just giving some clear boundaries to someone who is not doing things like they should be done.
As someone said here, it comes of a bit infantile to expect special treatment for your project, IMO.
It's been a bit difficult to watch the Bcachefs upstreaming saga unfold.
I keep an eye on Kernel developments, and have a particular interest in filesystems. Bcachefs has a lot to bring to the table and Kent is a brilliant person, but I can't recall the last time I've seen so much pushback and conflict.
Here's to hoping the adults can set aside whatever differences they have and come together in the interest of the community.
I keep an eye on Kernel developments, and have a particular interest in filesystems. Bcachefs has a lot to bring to the table and Kent is a brilliant person, but I can't recall the last time I've seen so much pushback and conflict.
Here's to hoping the adults can set aside whatever differences they have and come together in the interest of the community.
Kent expecting special treatment and not having to submit code via the regular, normal process that every other kernel developer uses just comes off as infantile.
Really looking forward to bcachefs, but honestly, what a prima-donna. Just follow the process like everyone else.
If this means we're going to have to continue the next decade with btrfs I may scream. I already use ZFS everywhere, but it's ease of use has been distro dependent.
Linus looks like he's back to his old ways. I should take a closer look at illumos.
Linus looks like he's back to his old ways. I should take a closer look at illumos.
Linus isn't behaving greatly here, but having a read through of the whole mailing list thread I'm actually fairly understanding of his reaction for once.
Here's this big project that Kent's been wanting to merge for ages and very basic steps that any Kernel developer would be aware of (like sending it to linux-next first) have not been done.
Here's this big project that Kent's been wanting to merge for ages and very basic steps that any Kernel developer would be aware of (like sending it to linux-next first) have not been done.
Linus is the king of not behaving well, and his position of "owning" the most popular kernel allows him to get away with it.
Yes Kent isn't following the right process but he made a more fundamental mistake: getting involved with Linux development at all.
Yes Kent isn't following the right process but he made a more fundamental mistake: getting involved with Linux development at all.
What's wrong with btrfs? I've been using it for a decade, both as a daily driver and on a NAS. I think it's great.
I hate building external modules, ZFS has never seemed worth the trouble to me. Why do you care so much?
I hate building external modules, ZFS has never seemed worth the trouble to me. Why do you care so much?
Basically everything related to multiple disks is super basic and too simple.
1. If you are writing with the `single` profile. It always writes to the one with the most space. This means that adding a new disk to a system will hotspot writes to that disk. Alternatively you can rebalance and it will make it so that all disks have the same space available which will basically hotspot reads. Most sane filesystems will do some sort of balancing.
2. If you are writing to the `RAID0` (striping with no redundancy) it will write across all disks and fill them up equally. When one disk fills it will be dropped out and the other disks will be hotspotted.
3. If you read from a replicated profile it picks the disk based on your process ID rather than any sort of intelligent metric like queue length or typical latency.
4. Replication profile can only be set on the filesystem level. bcachefs can set replication profile on the per-file or recursively at the directory level.
I mean nothing is absolutely terrible. But it is just shockingly basic for many things.
1. If you are writing with the `single` profile. It always writes to the one with the most space. This means that adding a new disk to a system will hotspot writes to that disk. Alternatively you can rebalance and it will make it so that all disks have the same space available which will basically hotspot reads. Most sane filesystems will do some sort of balancing.
2. If you are writing to the `RAID0` (striping with no redundancy) it will write across all disks and fill them up equally. When one disk fills it will be dropped out and the other disks will be hotspotted.
3. If you read from a replicated profile it picks the disk based on your process ID rather than any sort of intelligent metric like queue length or typical latency.
4. Replication profile can only be set on the filesystem level. bcachefs can set replication profile on the per-file or recursively at the directory level.
I mean nothing is absolutely terrible. But it is just shockingly basic for many things.
Interesting, thanks. I've honestly never had a reason to play with the multiple disk stuff.
How long have you got?
https://arstechnica.com/gadgets/2021/09/examining-btrfs-linu...
I have had Btrfs collapse and corrupt itself more in the last half a decade than all other Linux filesystems put together this century.
If you fill the volume, it will fail, and its own repair tools do not work and will damage it further. I would not and do not trust it.
https://arstechnica.com/gadgets/2021/09/examining-btrfs-linu...
I have had Btrfs collapse and corrupt itself more in the last half a decade than all other Linux filesystems put together this century.
If you fill the volume, it will fail, and its own repair tools do not work and will damage it further. I would not and do not trust it.
I don't care about the raid features personally, that's irrelevant to me. I care about snapshotting and send streams more than anything else.
Your anecdotal evidence about data loss is meaningless. I can counter with my own anecdote: I've had zero data loss with btrfs running pre-release RC kernels on my laptops for over a decade. I fill up disks a lot.
Your anecdotal evidence about data loss is meaningless. I can counter with my own anecdote: I've had zero data loss with btrfs running pre-release RC kernels on my laptops for over a decade. I fill up disks a lot.
Good for you.
One failure report is worth 1,000 success reports. Add as many orders of magnitude as you wish, and the statement remains true. This is not merely an axiom of reviewing and assessment, it's a joke:
https://xkcd.com/937/
I've been reviewing tech for a living for nearly 30 years now, and evaluating tech for a living -- as well as using it and fixing it and deploying it -- since the decade before that.
The point of tech assessment and tech reviewing is to balance the good features against the bad features. Too many people get dazzled by the good stuff so that they don't notice the bad stuff.
As the late great Douglas Adams put it:
« In other words - and this is the rock-solid principle on which the whole of the Corporation's Galaxywide success is founded - their fundamental design flaws are completely hidden by their superficial design flaws.” »
I don't care how good the features of a filesystem are if I can't trust it. Either it needs to be very solid, and have good documented battle-tested tools for fixing it and repairing it, such as XFS or JFS... which probably means it is also conservative on the features.
Or, alternatively, absolutely blasted bulletproof, to the extend that a multi-billion-dollar corporation sees fit to launch it without a repair tool because it doesn't need one.
There is one such system, and it is ZFS.
Btrfs is neither. It is loaded with features, some half implemented if that, it is fragile, and it does not have good repair features.
If I had seen it fail once, I would be dubious and sceptical.
If I had seen it fail twice, I would no longer trust it.
But I have not. I have seen it fail half a dozen times in 4 years.
And it is not just me.
Here is the documentation on the repair tool:
« Warning
Do not use `--repair` unless you are advised to do so by a developer or an experienced user, and then only after having accepted that no fsck successfully repair all types of filesystem corruption. E.g. some other software or hardware bugs can fatally damage a volume. »
Source: https://btrfs.readthedocs.io/en/latest/btrfs-check.html
Here is #1 corporate user SUSE's:
« WARNING: Using '--repair' can further damage a filesystem instead of helping if it can't fix your particular issue. »
Source: https://www.suse.com/support/kb/doc/?id=000018769
In corporate terms, this is an admission.
This tool cannot be trusted and that it is present means that the FS cannot be relied upon.
This is a giant red flashing light and eardrum-bashing warning siren.
It doesn't matter how many people like it and have had no problems. THERE ARE PROBLEMS. It doesn't matter how many corporates use it. A broken tool may still be usable.
Meta may deploy Btrfs across racks of kit but they don't care about the data on those racks and it can be replaced.
Fedora doesn't use snapshot support because it's bodged that functionality into OStree so it doesn't care if it's dangerous. It just wants to describe how horrendously inefficient Flatpak is.
One failure report is worth 1,000 success reports. Add as many orders of magnitude as you wish, and the statement remains true. This is not merely an axiom of reviewing and assessment, it's a joke:
https://xkcd.com/937/
I've been reviewing tech for a living for nearly 30 years now, and evaluating tech for a living -- as well as using it and fixing it and deploying it -- since the decade before that.
The point of tech assessment and tech reviewing is to balance the good features against the bad features. Too many people get dazzled by the good stuff so that they don't notice the bad stuff.
As the late great Douglas Adams put it:
« In other words - and this is the rock-solid principle on which the whole of the Corporation's Galaxywide success is founded - their fundamental design flaws are completely hidden by their superficial design flaws.” »
I don't care how good the features of a filesystem are if I can't trust it. Either it needs to be very solid, and have good documented battle-tested tools for fixing it and repairing it, such as XFS or JFS... which probably means it is also conservative on the features.
Or, alternatively, absolutely blasted bulletproof, to the extend that a multi-billion-dollar corporation sees fit to launch it without a repair tool because it doesn't need one.
There is one such system, and it is ZFS.
Btrfs is neither. It is loaded with features, some half implemented if that, it is fragile, and it does not have good repair features.
If I had seen it fail once, I would be dubious and sceptical.
If I had seen it fail twice, I would no longer trust it.
But I have not. I have seen it fail half a dozen times in 4 years.
And it is not just me.
Here is the documentation on the repair tool:
« Warning
Do not use `--repair` unless you are advised to do so by a developer or an experienced user, and then only after having accepted that no fsck successfully repair all types of filesystem corruption. E.g. some other software or hardware bugs can fatally damage a volume. »
Source: https://btrfs.readthedocs.io/en/latest/btrfs-check.html
Here is #1 corporate user SUSE's:
« WARNING: Using '--repair' can further damage a filesystem instead of helping if it can't fix your particular issue. »
Source: https://www.suse.com/support/kb/doc/?id=000018769
In corporate terms, this is an admission.
This tool cannot be trusted and that it is present means that the FS cannot be relied upon.
This is a giant red flashing light and eardrum-bashing warning siren.
It doesn't matter how many people like it and have had no problems. THERE ARE PROBLEMS. It doesn't matter how many corporates use it. A broken tool may still be usable.
Meta may deploy Btrfs across racks of kit but they don't care about the data on those racks and it can be replaced.
Fedora doesn't use snapshot support because it's bodged that functionality into OStree so it doesn't care if it's dangerous. It just wants to describe how horrendously inefficient Flatpak is.
jcalvinowens(2)
I've used btrfs for about two years and in that time it completely destroyed my filesystem and almost lost data. Not trusting it again.
Realistically, that's a basic requirement for any filesystem. I've not had a filesystem do that in my two plus decades of linux usage. That means I'm just not using it. Restored to ZFS from backup and couldn't be happier. ZFS works; is extraordinarily stable; and hasn't lost data for me yet.
Realistically, that's a basic requirement for any filesystem. I've not had a filesystem do that in my two plus decades of linux usage. That means I'm just not using it. Restored to ZFS from backup and couldn't be happier. ZFS works; is extraordinarily stable; and hasn't lost data for me yet.
You can sometimes get a narcissist to behave superficially like a person without a personality disorder by pushing them hard, but they always relapse.
whats this got to do with ZFS?
Given bcachefs' cheeky strapline "The COW filesystem for Linux that won't eat your data" (clearly referring to btrfs' spotty record of reliability and data loss), it's understandable the btrfs people are feeling triggered, that doesn't excuse the passive-aggressive obstacles those who are gatekeepers for kernel FS code have been using to stall adoption of bcachefs in the kernel.
ZFS is the only other alternative to bcachefs (a rock-solid one, albeit not license-compatible), which is why it's mentioned.
ZFS is the only other alternative to bcachefs (a rock-solid one, albeit not license-compatible), which is why it's mentioned.
Omg how after all this time are people still fucking repeating this 'not license-compatible' nonsense. This is one of this myths that at some point has infected the OpenSource community and its so totally wrong and damaging.
Its not license incompatible, many, many, many organizations have shipped ZFS as part of the distribution and absolutely nobody ever sued or even threatened sue over the license.
Linus didn't want to merge it because of some vague fear of Oracle and how they sued over Java, but that was about API not license.
What Linus should have done is to simple upstream it. If some big company (IBM or whoever) wants to not 'risk' ZFS then they should damn well do their own work and remove it. There is no reason that the waste majority of the Linux community should be denied great features (ZFS, DTrace and friends) over some vague fear of Oracle.
Its not license incompatible, many, many, many organizations have shipped ZFS as part of the distribution and absolutely nobody ever sued or even threatened sue over the license.
Linus didn't want to merge it because of some vague fear of Oracle and how they sued over Java, but that was about API not license.
What Linus should have done is to simple upstream it. If some big company (IBM or whoever) wants to not 'risk' ZFS then they should damn well do their own work and remove it. There is no reason that the waste majority of the Linux community should be denied great features (ZFS, DTrace and friends) over some vague fear of Oracle.
CDDL means ZFS can't be incorporated into the Linux kernel as opposed to a distribution. That does have impacts as in when the kernel makes breaking changes in the internal vfs structures used by ZoL. Oracle could change that at the stoke of a pen, but chose not to, for whatever reason.
I don't think fear of Oracle is the issue, more likely just an irrational rejection of everything Sun and Solaris. After all btrfs also originated in Oracle (what's worse, despicable Oracle itself, not via the Sun acquisition).
Perhaps Oracle's unwillingness to relicense is simply that Oracle knows how inferior btrfs and thus doesn't mind giving it away by licensing it GPLv2, but ZFS is much more valuable and they want to keep the rights to it as potential competitive advantage. If Sun hadn't licensed ZFS CDDL before the acquisition, I'm pretty sure Oracle would be happy to keep it closed-source.
I hope bcachefs makes it into the kernel quickly, that distros drop btrfs and that its infiltrators are turfed out of kernel-land, but that's probably too much to hope for. In the meantime, I am happy to keep running ZFS for all my critical data on Illumos, Alpine and Ubuntu.
I don't think fear of Oracle is the issue, more likely just an irrational rejection of everything Sun and Solaris. After all btrfs also originated in Oracle (what's worse, despicable Oracle itself, not via the Sun acquisition).
Perhaps Oracle's unwillingness to relicense is simply that Oracle knows how inferior btrfs and thus doesn't mind giving it away by licensing it GPLv2, but ZFS is much more valuable and they want to keep the rights to it as potential competitive advantage. If Sun hadn't licensed ZFS CDDL before the acquisition, I'm pretty sure Oracle would be happy to keep it closed-source.
I hope bcachefs makes it into the kernel quickly, that distros drop btrfs and that its infiltrators are turfed out of kernel-land, but that's probably too much to hope for. In the meantime, I am happy to keep running ZFS for all my critical data on Illumos, Alpine and Ubuntu.
If you can legally distributed then there is no reason it can't be linked into the main kernel.
> I don't think fear of Oracle is the issue
Why then did Linus say exactly that?
> After all btrfs also originated in Oracle (what's worse, despicable Oracle itself, not via the Sun acquisition).
Wrong. It was simply a guy that worked for Oracle, it wasn't an Oracle project.
> Perhaps Oracle's unwillingness to relicense is simply that
It doesn't matter what Oracle once, of course they don't want anything to do with ti. The community has forked ZFS long ago and Oracle has no power what so ever to reliance most of that code.
Literally nobody wants the Oracle proprietary ZFS anyway.
> I don't think fear of Oracle is the issue
Why then did Linus say exactly that?
> After all btrfs also originated in Oracle (what's worse, despicable Oracle itself, not via the Sun acquisition).
Wrong. It was simply a guy that worked for Oracle, it wasn't an Oracle project.
> Perhaps Oracle's unwillingness to relicense is simply that
It doesn't matter what Oracle once, of course they don't want anything to do with ti. The community has forked ZFS long ago and Oracle has no power what so ever to reliance most of that code.
Literally nobody wants the Oracle proprietary ZFS anyway.
I obviously share your frustration! I have long viewed these bogus licensing claims as the open source variant of NIH syndrome,[0] complete with the conjuring of both fear and grievance that always makes for particularly virulent thinking among the orthodox.
[0] https://en.wikipedia.org/wiki/Not_invented_here
[0] https://en.wikipedia.org/wiki/Not_invented_here
But the license claims aren't bogus – or at least not all of them? For example the CDDL states that you can't add additional restrictions, but if you combine it with the GPL then the GPL also applies, so that's an "additional restriction", no?
Of course, all of this is highly legalistic and in spirit the CDDL and GPL are essentially identical, and from that perspective any lawsuit would be complete bollocks, but ... bollocks lawsuits do happen.
There's lots of claims about GPL and CDDL compatibility floating around and much of it sounds at least plausible and it's all a bit controversial. Clearing up this sort of confusion is exactly the sort of scenario where courts come in to play. So it seems to me that fears of a lawsuit are not entirely unreasonable.
Do you want to put all your money and the future development of Linux at risk? The nature of lawsuits in the US means that as soon as the lawsuit is filed you've lost already, and the ruling just decides if you've lost even harder.
Not denying Linux can have its share of NIH, but I see this mostly as a matter of risk management. e.g. Canonical has decided to take that risk, which is fair and reasonable, but Linus decided to not take that risk, which is also fair and reasonable.
Of course the entire situation is ridiculous; btrfs is partly sponsored by Oracle, and Oracle is also sitting on a perfectly functional filesystem they can just relicense like they did with dtrace. I don't know what's preventing Oracle from doing that.
Of course, all of this is highly legalistic and in spirit the CDDL and GPL are essentially identical, and from that perspective any lawsuit would be complete bollocks, but ... bollocks lawsuits do happen.
There's lots of claims about GPL and CDDL compatibility floating around and much of it sounds at least plausible and it's all a bit controversial. Clearing up this sort of confusion is exactly the sort of scenario where courts come in to play. So it seems to me that fears of a lawsuit are not entirely unreasonable.
Do you want to put all your money and the future development of Linux at risk? The nature of lawsuits in the US means that as soon as the lawsuit is filed you've lost already, and the ruling just decides if you've lost even harder.
Not denying Linux can have its share of NIH, but I see this mostly as a matter of risk management. e.g. Canonical has decided to take that risk, which is fair and reasonable, but Linus decided to not take that risk, which is also fair and reasonable.
Of course the entire situation is ridiculous; btrfs is partly sponsored by Oracle, and Oracle is also sitting on a perfectly functional filesystem they can just relicense like they did with dtrace. I don't know what's preventing Oracle from doing that.
Why then does Canonical and many other places link it with the kernel and distribute it. Canonical stated there is no issue.
If you really believe there is a license issue go ahead and start suing people and see how far you get with that.
> So it seems to me that fears of a lawsuit are not entirely unreasonable.
And yet lots cooperation shipped code and products with ZFS and Linux with no issue what so ever for many, many years now.
> Do you want to put all your money and the future development of Linux at risk?
Its not a risk for Linux at all. Its a risk for a company that distributes Linux with ZFS in it. If a company for some reason believes that there might be an issue and they don't want it, then they can feel free to remove it.
My guess is that 99.9% of companies will simply use Linux with ZFS in it.
> Linus decided to not take that risk, which is also fair and reasonable
He could just say he would merge it, and unless half the large cooperations in the world jump on him to stop him there is no reason to not merge it. But he didn't do it, he just reject it out of hand based on some vague 'maybe oracle' something. Did he consult a lawyers?
If you really believe there is a license issue go ahead and start suing people and see how far you get with that.
> So it seems to me that fears of a lawsuit are not entirely unreasonable.
And yet lots cooperation shipped code and products with ZFS and Linux with no issue what so ever for many, many years now.
> Do you want to put all your money and the future development of Linux at risk?
Its not a risk for Linux at all. Its a risk for a company that distributes Linux with ZFS in it. If a company for some reason believes that there might be an issue and they don't want it, then they can feel free to remove it.
My guess is that 99.9% of companies will simply use Linux with ZFS in it.
> Linus decided to not take that risk, which is also fair and reasonable
He could just say he would merge it, and unless half the large cooperations in the world jump on him to stop him there is no reason to not merge it. But he didn't do it, he just reject it out of hand based on some vague 'maybe oracle' something. Did he consult a lawyers?
Lots of lawyers have said it's probably incompatible. Is it? Who knows. You're saying it's all nonsense, which seems quite a far-fetched claim. I find it hard to see how anyone could defend any other answer than "it's presently unclear", possibly followed by "but it's probably (in)?compatible".
That distros haven't been sued yet seems meaningless. Granted, Oracle has been a bit better in recent years, but people haven't forgotten Oracle v. Google, SCO v. {everyone}, USL v. BSDi, etc. One of the lessons is that you can never know who will own some piece of copyright or IP in the future and what they will do with it. Oracle could off-load all of the Solaris stuff to some copyright troll tomorrow.
But it doesn't really matter who is right or wrong here or if it is or isn't compatible: there's enough of a dispute for lawsuits, and they're going to be costly and a headache anyway.
That distros haven't been sued yet seems meaningless. Granted, Oracle has been a bit better in recent years, but people haven't forgotten Oracle v. Google, SCO v. {everyone}, USL v. BSDi, etc. One of the lessons is that you can never know who will own some piece of copyright or IP in the future and what they will do with it. Oracle could off-load all of the Solaris stuff to some copyright troll tomorrow.
But it doesn't really matter who is right or wrong here or if it is or isn't compatible: there's enough of a dispute for lawsuits, and they're going to be costly and a headache anyway.
> That distros haven't been sued yet seems meaningless.
I don't think if meaningless. And its not just distros, there are whole companies whos buissness model depends on linux with zfs. And its also all the companies who use the distros.
Literally 100s and 100s of company are running Ubuntu. All of those companies have decided that ZFS isn't a problem of them. All of them could have ask to exclude ZFS but non have to my knowlage.
Its also not really about Oracle. Sun did the legwork when they open-sourced their stuff. Those licenses in itself are not in question. And those licenses certainty don't care about being linked with GPL. The only question is if GPL is incomparable with the CDDL.
So the fear of Oracle in this case that Oracle would put itself forward as defender of the GPL? That makes no sense. But anybody could sue anybody about that at any time.
If Linux had included ZFS the ZFS code would already exists on most of the worlds computers by now. And the waste, waste majority of companies would not remove ZFS over some open source inside baseball.
I don't think if meaningless. And its not just distros, there are whole companies whos buissness model depends on linux with zfs. And its also all the companies who use the distros.
Literally 100s and 100s of company are running Ubuntu. All of those companies have decided that ZFS isn't a problem of them. All of them could have ask to exclude ZFS but non have to my knowlage.
Its also not really about Oracle. Sun did the legwork when they open-sourced their stuff. Those licenses in itself are not in question. And those licenses certainty don't care about being linked with GPL. The only question is if GPL is incomparable with the CDDL.
So the fear of Oracle in this case that Oracle would put itself forward as defender of the GPL? That makes no sense. But anybody could sue anybody about that at any time.
If Linux had included ZFS the ZFS code would already exists on most of the worlds computers by now. And the waste, waste majority of companies would not remove ZFS over some open source inside baseball.
I'm not disagreeing with you. I fully understand why the code can't be incorporated as is. Then again, workarounds were found for things like the proprietary nVidia drivers, and the unwillingness to do so for an arguably far superior filesystem reeks of NIH.
Hi Bryan! Yes, NIH is the most likely explanation, since btrfs was accepted depite originating from deepest, darkest Oracle (and I happen to share your dim opinion of them).
Ah right, I hadn't really looked at bcachefs since about 2016 when it was still a specialist object store
ZFS is the far more solid big sibling of Btrfs, but its license is not compatible with the GPL so most distros won't use it.
This licensing issue is the reason bcachefs exists.
This licensing issue is the reason bcachefs exists.