Let me give you an analogy: Volvo is known to manufacture very safe cars. Now let's say I drive a Volvo car with a box of dynamite on the passenger seat. I stop at a red light but hit the brake a bit too hard and the box of dynamite falls and causes an explosion, disintegrating everything in a 20-foot radius. So whose fault was it? Volvo?
> Missing a check (or precondition, or validation) is squarely on Microsoft.
Missing a check for presence of dynamite before allowing me to start the car is squarely on Volvo!
You see how silly that sounds?
Now, back to being serious: MS cannot possibly control and validate everything you decide to install and run on your system, specially if the things you install are kernel drivers. It is simply impossible. If you install a kernel driver developed by a 3rd party company, and that driver crashes your system because the devs at that company forgot to perform proper validation of data, well... that's on them. Even if MS wanted, they wouldn't be able to verify the soundness of any piece of code that is installed as a driver and runs with kernel level privileges. That'd require solving the halting problem.
> for the case where you want to work with semi-structured IR (GIMPLE) rather than machine code
Most of the code for syscall handlers is carefully hand-crafted assembly, so probably not GIMPLE. Maybe something like Valgrind's VEX IR. I see what you mean though.
> How does that even work, libc-wise?
It works as you say, the "return an error like a stub" logic is the same as the "this syscall isn't implemented logic". AFAIK libc will provide the wrappers regardless (if there are wrappers, not all syscalls have them) and the kernel will just return -ENOSYS, like it would do for any invalid syscall number.
> If so, maybe consider that a bug? Submit a patch to have an arch's stubbed optional syscalls return a different error than for syscalls that don't exist for that arch
I am 99.9% sure that'd be impossible. The "stubbed optional syscalls" return -ENOSYS (as if they did not exist) by design. Although annoying, it's not really a bug, it's the way it's intended to work. I doubt such a patch would such an API-breaking change would be accepted, as a lot of existing code relies on this behavior. I don't think there even is an appropriate errno number to return in such case. It's unfortunate, but it is what it is.
> To be clear, I wasn't talking about compile-time code inclusion; I was talking about runtime
Yeah, it was clear that you meant runtime but less clear what you exactly meant with "all kernel services were already initialized". Now I see what you mean. Yes, what you describe definitely seems doable from a theoretical point of view for some architectures, but I struggle to think about such a solution given its complexity. It would still require manual recognition of interesting source code files and syscall handler code, plus a significant amount of scripting/patching/compiling to get it to work. Not to talk about emulation since this would need to be done for different archs. That's why even though it'd be nice in theory, it practically seems like a borderline unapproachable problem to me, from multiple sides.
I appreciate all the input anyway, this is definitely an interesting topic.
Yeah the few "esoteric" syscalls that I hardcode in my tool are historical ones. That's pretty much the only reason why I bothered hardcoding them. I don't assume any new syscall will ever be implemented like that nowadays. Such insane implementations would be rejected unless there is a very specific compelling reason.
> finds the intermediate build artifacts pertaining to the code unit that contains the syscall handler
Hmm I think this is unneeded, vmlinux already has all the code. Also things move around too much across kernel versions and archs so can't easily pinpoint which object files to choose. Additionally, you would need an entire built kernel source tree, which is a lot more than simply a built vmlinux plus an optional non-built kernel source dir (that is what I use right now). Just as an example: currently I have some 600 kernel images with debug info that I keep for reference, which requires around 76 Gigabytes of space on my disk. Having 600 built kernel trees would require a lot more space, in the order of Terabytes.
> taking the resulting IR data-structure / AST / whatever, and doing some symbolic interpretation of it
I have been thinking about this a lot. I do a simplified version of this for x86 >= v6.9 because the syscall table was removed and turned into a giant switch case, which I symbolically emulate to extract syscall numbers, but that's pretty simple and definitely not an exhaustive analysis (some other stuff could be happening before reaching the handler). The problem is that this kind of solution is very hard to implement and I think would be way too slow on a general case. There also aren't even decent symbolic execution engines to do this for some archs. You are right when you say "insane amounts of up-front work" - that is definitely too much for me for a hobby project like this :').
The first main problem however is that all of this starts from the assumption that you already have built a kernel with all the syscalls available. This is not the case unless you meticulously configure it accordingly, which is not so simple and requires constant manual (sigh) updates to the build configuration each kernel release. There isn't a way to e.g. pretend that "all kernel services were already initialized" as you say in point #2. If a kernel is built w/o a certain syscall, the code will simply not be there. Kernel configuration remains a problem also for your point #1. The only real solution I see would be submitting kernel patch to add a target in the root Makefile that enables all syscalls with their related configs, and hope kernel devs like it (doubt it).
Thank you very much :). I am using static analysis of kernel images (vmlinux ELF) that are built with debug information. Each table you see was extracted from a kernel built by my tool, Systrack, that can configure and build kernels that have all the syscalls available. The code is heavily commented and available on GitHub if you are interested: https://github.com/mebeim/systrack
I realized soon in the process that simply looking at kernel sources was not enough to extract everything accurately, specially definition locations. I also wanted this to be a tool to extract syscalls actually implemented from a given kernel image, so that's what it does.
Your approach should be fine, that is what any other language does basically: rely on uapi headers provided by the kernel (just beware that some may be generated at build time inside e.g. include/asm/generated/xxx). You should rely on the headers that are exported when you do `make headers_install`. Also, make sure to have a generic syscall() function that takes an arbitrary syscall number and an arbitrary amount of args to make raw syscalls for the weird ones you don't easily find in uapi headers and you should be good. After all, even in the C library headers some of the "weird" syscalls aren't present sometimes.
Nope, not even that, because believe it or not, sometimes not even the .tbl files have all of them :'). In fact, the only arch where IMHO syscalls make sense and are organized in a sane way is arm64 that doesn't even have a .tbl file. And not even the table in kernel memory is enough sometimes! Some are special handlers in syscall entry code (like the one I mentioned above). It's just a mess, hence why I sort of gave up at some point and for some "esoteric" syscall I just hardcode them.
Fun fact, some weird syscalls don't even appear under /sys/kernel/debug/tracing because they lack ftrace metadata. It was pretty fun (read: a nightmare) to deal with some of those in my tool. You can grep -R -F "NOT found in ftrace metadata" in the logs in my db (https://github.com/mebeim/linux-syscalls/tree/master/db) to see which ones.
Not really. Even though section 2 should be for "syscalls", it is really only for libc syscall wrappers. Very few pages in the section 2 document the raw syscalls, and those that do say it specifically at the beginning. OTOH, in section 3 you won't find any syscall at all (wrapper or not).
The homepage loads exactly the most common x86-64 ABI, which is x64 (according to kernel naming). It's the one for 64-bit syscalls made by 64-bit code. On x86-64 you also have IA32 (32-bit syscalls made by 32-bit code) and x32 (64-bit syscalls made by 64-bit code specifically built to only use 32-bit pointers).
I'm the author and I can confirm. The website will not work with JS disabled simply because it's a static HTML "skeleton" page loading JSON tables with JS and populating a <table> element. If it's working then it means you must have JS enabled. I don't plan to add support for browsers without JS, but the JSON tables have all the information you need anyway, and those are just static files (e.g. https://syscalls.mebeim.net/db/x86/64/x64/latest/table.json).
> Missing a check (or precondition, or validation) is squarely on Microsoft.
Missing a check for presence of dynamite before allowing me to start the car is squarely on Volvo!
You see how silly that sounds?
Now, back to being serious: MS cannot possibly control and validate everything you decide to install and run on your system, specially if the things you install are kernel drivers. It is simply impossible. If you install a kernel driver developed by a 3rd party company, and that driver crashes your system because the devs at that company forgot to perform proper validation of data, well... that's on them. Even if MS wanted, they wouldn't be able to verify the soundness of any piece of code that is installed as a driver and runs with kernel level privileges. That'd require solving the halting problem.