At least FreeBSD's syscall ABI is guaranteed to be stable, one can run ancient binaries on a modern kernel. I believe the same is not true of OpenBSD and maybe NetBSD however.
The pf maintainer in FreeBSD has been doing a ton of work to bring more recent improvements over from OpenBSD, trying to bring them in sync as much as possible without breaking compatibility:
https://cgit.freebsd.org/src/log/sys/netpfil/pf
The state of affairs you described is much less the case now than in the past.
A design that works by default isn't automatically better either though. You have to look at the details.
> I guess one might be tempted to forgive a few warts in the interface layers
... well, yeah, that's exactly my sentiment about kqueue here. What you're talking about is basically a small wart that no one's bothered to address because it's inconsequential.
The article clearly isn't talking about technical debt within the kernel implementations of epoll and kqueue, and if one wanted, it'd be easy to define fallback EVFILT_READ/WRITE filters using a device's poll implementation.
I don't really understand what argument you're making. Is io_uring also a bad design because it requires new file_operations?
Well, no, it's "this interface works fine for you if you implement it."
The kernel doesn't magically know whether your device file has data available to read, your device file has to define what that means. That's all I'm referring to. Hooking that up to kqueue usually involves writing 5-10 lines of code.
I'm not sure. Maybe it's "wait for events that aren't tied to an fd."
For instance, FreeBSD (and I think other BSDs) also have EVFILT_PROC, which lets you monitor a PID (not an fd) for events. One such event is NOTE_FORK, i.e., the monitored process just forked. Can you wait for such events with epoll? I'm not sure.
More generally, suppose you wanted to automatically start watching all descendants of the process for events as well. If I was required to use a separate fd to monitor that child process, then upon getting the fork event I'd have to somehow obtain an fd for that child process and then tell epoll about it, and in that window I may have missed the child process forking off a grandchild.
I'm not sure how to solve this kind of problem in the epoll world. I guess you could introduce a new fd type which represents a process and all of its descendants, and define some semantics for how epoll reports events on that fd type. In FreeBSD we can just have a dedicated EVFILT_PROC filter, no need for a new fd type. I'm not really sure whether that's better or worse.
I don't think the article does a good job of arguing its premise, which I think is that kqueue is a less general interface than epoll.
When adding a new descriptor type, one can define semantics for existing filters (e.g., EVFILT_READ) as one pleases.
To give an example, FreeBSD has EVFILT_PROCDESC to watch for events on process descriptors, which are basically analogous to pidfds. Right now, using that filter kevent() can tell you that the process referenced by a procdesc has exited. That could have been defined using the EVFILT_READ filter instead of or in addition to adding EVFILT_PROCDESC. There was no specific need to introduce EVFILT_PROCDESC, except that the implementor presumably wanted to leave space to add additional event types, and it seemed cleaner to introduce a new EVFILT_PROCDESC filter. Process descriptors don't implement EVFILT_READ today, but there's no reason they can't.
So if one wants to define a new event type using kevent(), one has the option of adding a new definition (new filter, new note type for an existing filter, etc.), or adding a new type of file descriptor which implements EVFILT_READ and other "standard" filters. kqueue doesn't really constrain you either way.
In FreeBSD, most of the filter types correspond to non-fd-based events. But nothing stops one from adding new fd types for similar purposes. For instance, we have both EVFILT_TIMER (a non-fd event filter) and timerfd (which implements EVFILT_READ and in particular didn't need a new filter). Both are roughly equivalent; the latter behaves more like a regular file descriptor from kqueue's perspective, which might be better, but it'd be nice to see an example illustrating how.
One could argue that the simultaneous existence of timerfds and EVFILT_TIMER is technical debt, but that's not really kqueue's fault. EVFILT_TIMER has been around forever, and timerfd was added to improve Linux compatibility.
So, I think the article is misguided. In particular, the claim that "any time you want kqueue to do something new, you have to add a new type of event filter" is just wrong. I'm not arguing that there isn't technical debt here, but it's not really because of kqueue's design.
The same is true of kqueue/kevent though... the driver just needs to decide which filters it wants to implement. There's no need to extend kqueue when adding some custom driver or fd type. One just needs to define some semantics for the existing filters.
For what it's worth, the default root shell is now /bin/sh instead of csh. I think that's true as of 14.0. /bin/sh is also a better interactive shell than it used to be, though yeah, I don't use it to do anything other than install my preferred shell.
CHERI does more than help eliminate security vulnerabilities. Consider that today we rely on the MMU to provide memory isolation between Unix processes; CHERI enables isolation without switching page tables, at a smaller hardware cost (though it's not like you can drop unmodified software into such an architecture). So I don't think it's correct to consider this yet another layer of complexity. If anything it has the potential to lead to simpler system designs.
CHERI does permit tricks like storing flags in the low bits of a pointer, at least to some extent. Quite a lot of low level C code (including some in the CheriBSD kernel) needs that to work.
I use "ktrace -t f" once in a while for debugging and it's really handy. Output looks like
78436 cat PFLT 0x6c71f99cda8 0x2<VM_PROT_WRITE>
78436 cat PRET KERN_SUCCESS
78436 cat PFLT 0x3c6efd36c280 0x2<VM_PROT_WRITE>
78436 cat PRET KERN_SUCCESS
78436 cat PFLT 0x3c6efd36e158 0x2<VM_PROT_WRITE>
78436 cat PRET KERN_SUCCESS
...
Obviously not nearly as flexible as ebpf though. For instance it'll log all page faults happening in the context of the process, and so includes page faults that happen in the kernel due to copyin()/copyout() etc. Sometimes it's helpful and other times confusing.
How much data ends up being served from RAM? I had the impression that it was negligible and that the page cache was mostly used for file metadata and infrequently accessed data.
This is mostly true on FreeBSD as well. The real problem is that capability mode also disallows openat(AT_FDCWD) - there has to be an explicit directory descriptor.
It depends on the bug. syzkaller does an excellent job finding race conditions, but it can be difficult to generate a reliable reproducer for them. It often succeeds nonetheless. In other cases there can be a wide gap between the proximate and root causes of a crash. For instance some system call bug might corrupt memory in a way that only results in a crash some time later, when some asynchronous task runs, in which case it's also difficult to find a reproducer. Sanitizers can help identify such bugs earlier and so reduce the amount of manual analysis needed in the absence of a reproducer.
I'm not sure what happened in this case. The linked report does indeed have an associated reproducer.
That's fair. At the time, though, it wasn't clear that the author's angle was that FreeBSD doesn't have a culture of doing code reviews. We do, and I don't think one has to look very hard to see that.
There's some discussion happening now and I do expect to see some process changes coming out of this. It's tricky. The review you link does nominally follow the process of creating a review and having some discussion, but there isn't much actual code review happening there.
It's not. We do a lot of code review, and it's done publicly. It's easy to look at the commit logs. I find it telling that the article doesn't spend even one word trying to delve into our code review practices. This case was an aberration.