I would assume because then the shape of the data would be too different? SOAs is super effective when it suits the shape of the data. Here, the difference would be the difference between an OLTP and OLAP DB. And you wouldn't use an OLAP for an OLTP workload?
Appreciate this rincebrain. Know that you know better than most and this certainly covers my 2nd point. I don't imagine these cases cover my first point though? These are not bugs of the type a fsck would catch?
I think you may be misreading my point above. I am not arguing ZFS doesn't have bugs. That's nuts. I am arguing that the bug the parent says he has would be an extraordinary bug.
This is not just a bug that a scrub wouldn't find, but also it is a bug which an fsck would find. And it is not just a bug in the spacemaps or other metadata, but the parent's claim is this is a bug which a scrub, which is just a read, wouldn't see, but a subsequent read would reveal.
I think you're missing the 2nd feature to the parent's point that I take issue with, which is this is not just a bug that a scrub wouldn't find, but it must also be a bug which an fsck would find.
The parent's point is -- ZFS should have an fsck tool because an fsck does something ZFS cannot do by other means. I disagree. Yes, ZFS has bugs like any filesystem. However, I'm not sure an fsck tool would make that situation better?
How is the structure not valid here? Can you explain to us how an fsck would discover this bug (show an example where an fsck fixed a similar bug) but ZFS could never? The point I take contention with is that missing an fsck is a problem for ZFS, so more specifically can you answer my 4th Q:
>> 4) If so, wouldn't this just be a bug, like (a bug in) fsck, not some fundamental limitation of the system?
So -- is it possible an fsck might discover an inconsistency ZFS couldn't? Sure. Would this be a fundamental flaw of ZFS, which requires an fsck, instead of merely a bug? I'm less sure.
You do seem to at least understand my general contention with the parent's point. However, the parent is also making a specific claim about a bug which would be extraordinary. Parent's claim is this is a bug which a scrub, which is just a read, wouldn't see, but a subsequent read would reveal.
So -- is it possible an fsck might discover this specific kind of extraordinary bug in ZFS, after a scrub had already read back the data? Of that I'm highly dubious.
> Imagine that a directory ZAP has an entry that points to a bogus object ID. That would be an example. The ZAP block is intact but its content is inconsistent.
The above is interesting and fair enough, but a few points:
First, I'm not sure that makes what seems to be the parent's point -- that scrub is an inadequate replacement for an fsck.
Second, I'm really unsure if your case is the situation the parent is referring to. Parent seems to be indicating actual data loss is occurring. Not leaking objects or space or bogus object IDs. Parent seems to be saying she/he scrubs with no errors and then when she/he tries to read back a file, oops, ZFS can't.
> Imagine a race condition that writes a file node where a directory node should be. You have a valid object with a valid checksum, but it's hooked into the wrong place in your data structure.
A few things: 1) Is this an actual ZFS issue you encountered or is this a hypothetical? 2) And -- you don't imagine this would be discovered during a scrub? Why not? 3) But -- you do imagine it would be discovered and repaired by an fsck instead? Why so? 4) If so, wouldn't this just be a bug, like a fsck, not some fundamental limitation of the system?
FWIW I've never seen anything like this. I have seen Linux plus a flaky ALPM implementation drop reads and writes. I have seen ZFS notice at the very same moment when the power dropped via errors in `zpool status`. I do wonder if ext4's fsck or XFS's fsck does the same when someone who didn't know any better (like me!) sets the power management policy to "min_power" or "med_power_with_dipm".
"Scrubs differ significantly from traditional filesystem checks. Tools such as fsck or chkdsk examine logical structures and attempt to repair inconsistencies related to directory trees, allocation maps, reference counts, and other metadata relationships. ZFS does not need to perform these operations during normal scrubs because its transactional design ensures metadata consistency. Every transaction group moves the filesystem from one valid state to another. The scrub verifies the correctness of the data and metadata at the block level, not logical relationships."
> ZFS scrubs do not check filesystem objects for correctness and consistency; it only checks that they have the expected checksum and so have not become corrupted due to disk errors or other problems
A scrub literally reads the object from disk. And, for each block, the checksums are read up the tree. The object is therefore guaranteed to be correct and consistent at least re: the tree of blocks written.
> Unfortunately it's possible for ZFS bugs and issues to give you filesystem objects that have problems
Can you give a more concrete example of what you mean? It sounds like you have some experience with ZFS, but "ZFS doesn't have an fsck" is also some truly ancient FUD, so you will forgive my skepticism.
I'm willing to believe that you request an object and ZFS cannot return that object because of ... a checksum error or a read error in a single disk configuration, but what I have never seen is a scrub that indicates everything is fine, and then reads which don't return an object (because scrubs are just reads themselves?).
Now, are things like pool metadata corruption possible in ZFS? Yes, certainly. I'm just not sure fsck would or could help you out of the same jam if you were using XFS or ext4. AFAIK fsck may repair inconsistencies but I'm not sure it can repair metadata any better than ZFS can?
> That will also put it on the unfortunate position of being the place that breaks every time somebody adds a bug to the C code.
Can someone explain charitably what the poster is getting at? To me, the above makes zero sense. If the Rust code is what is implemented correctly, and has the well-defined semantics, then, when the C code breaks, it's obviously the C code's problem?
You have it the wrong way around. Here, ZFS uses many small S3 objects as the storage substrate, rather than physical disks. The value proposition is that this should be definitely cheaper and perhaps more durable than EBS.
> I think that companies like Oracle, SAP, and Broadcom begin to resemble specialized private equity firms
This is an entirely fair/accurate. I suppose what I am getting at is that these are just 2 different business models, and, the world can sustain a multitude of business models. There need not be only one (har har).
It's also fair to believe there is a moral dimension to one's own model which doesn't extract maximum value from the customer. Because IMHO "let's kick them in the dicks again" isn't an especially likable model, even if it is successful, and it's fair to avoid doing business with such people.
Imagine trying to sell your partners on doing business with Broadcom. If your core principle is "Broadcom needs to be around in 10 years", maybe the persistence/"kick them in the dicks" model is appealing, but otherwise, its fair for their competitors/Oxide to point out how awful dealing with a corporate sociopath might be.
>> Certainly, these companies not endure as innovators: when coercion is your business model, innovation is not merely unnecessary but actively antithetical.
Oracle and VMware do seem like just rent seekers. I'm sure those rents do pay for plenty of nice things, but it's really hard for me to ever understand Oracle or VMware as an "innovator", beyond their initial innovations (their flagship DB, x86 virtualization).
> Oracle has endured nearly 50 years. Sun did not endure.
IMHO it's perfectly fine for companies to live well, and then be sold. AFAIAC persistence is only proof of persistence. Sun created plenty of wealth/millionaires too. And, by Bryan's lights, it did so mostly ethically. That's a good life.
> Page management isn't really a thing we can do well "in user space".
But it is the thing most high performance OLTP DBMSs, most of us are aware of, do? I'm also not sure your cite is relevant here. Or it is at least niche. The comparison is made to LeanStore, which is AFAICT is not feature complete, and a research prototype?
Your cite does not describe a unikernel use case, but instead in kernel helper modules. Your cite is about leveraging in kernel virtual memory for the DB buffer cache, and thus one wonders how sophisticated the VM subsystem is in most unikernels? That is -- how is this argument for unikernels? Seems as though your cite is making the opposite argument -- for a more complex relationship between the DB and the kernel, not a pared down one.
> And then, further, a DB goes and basically implements its own equivalent of a filesystem, managing its own storage. Often fighting with the OS about the semantics of fsync/durability, etc.
The fights you're describing what have thus far been the problems of ceding control of the buffer cache to the kernel, via mmap, especially re: transactional safety.
If your argument is kernels may need a bottom up redesign to make ideas like this work, I suppose that makes sense. However, again, I'm not sure that makes unikernels more of an answer here than anywhere else, though.
> I don't think it's an unreasonable mental leap for people to start thinking: "I'm by necessity [cuz cloud] in a VM. Now I'm inside an OS in a VM, and the OS is sometimes getting in my way, and I'm doing things to get around the OS... Why?"
I think that's a fair thought to have, but the problem is how it actually works in practice. As in, less code seems really enticing, the problem is what abstractions are you throwing away. If the abstraction is less memory protection, maybe this is not a good tradeoff in practice.
> On performance: ... In particular I am thinking how there's a whole body of research of database buffer pool management
Why? The solution thus far has been to turn off what the kernel does, and, do those things in userspace, not move everything in the kernel? Where are these performance gains to be had?
> The Linux kernel is a general purpose utility optimizing for the entire range of "normal things" people do with their Linux machines.
Yeah, like logging and debugging. Perhaps you say: "Oh we just add that logging and debugging to the blob we run". Well isn't that now another thing that can take down the system, when before it was a separate process?
> That and startup times, big world of difference.
Perhaps in this very narrow instance, this is useful, but what is it useful for? Can't Linux or another OS be optimized for this use case without having to throw the baby out with the bathwater? Can't one snapshot a Firecracker VM and reach even faster startup times?
> On security, I don't think it's unreasonable or pure "security theatre" to go removing an attack surface entirely
Isn't perhaps the most serious problem removing any and all protection domains? Like between apps and the kernel and between the apps themselves?
I mean -- sure maybe remove the filesystem, but isn't no memory protection what makes it a unikernel? And, even then, a filesystem is usually a useful abstraction! When have I found myself wanting less filesystem? Usually, I want more -- like ZFS.
This is all just to say -- you're right -- there may be a use case for such systems, but no one has really adequately described what that actually is, and therefore this feels like systems autoeroticism.
> I think Rust is more unsafe than C due to supply chain issues in the Rust ecosystem
This is such an incredibly cheap shot. First, the supply chain issues referenced have nothing to do with Rust, the language, itself. Second, Rust's build system, cargo, may have these issues, but cargo's web fetch features simply aren't used by the Linux kernel.
So -- we can have a debate about which is a better a world to live in, one with or without cargo, but it really has nothing to do with the Linux kernel security.
> I'm more interested in the % of rust code that is marked unsafe.
I think you should less interested in % unsafe as what the unsafe is used to do, that is, it's likelihood to cause UB, etc. If it's unsafe to interface with C code, or unsafe to do a completely safe transmute, I'm not sure one should care.