OS X BSD System Calls Reference(dyjakan.sigsegv.pl)
dyjakan.sigsegv.pl
OS X BSD System Calls Reference
http://dyjakan.sigsegv.pl/osx-bsd-syscalls/
5 comments
This makes me wonder, what would this look like for plan9?
http://aiju.de/misc/plan9-syscalls
http://aiju.de/misc/plan9-syscalls
i like how the plan9 system calls list can fit on an index card.
I guess that i'm then wondering, to what extent are the profusion of api calls on bsd/linux (by comparison) due to which of the following: a) legacy of evolving richer apis while supporting older ones b) poorer factoring of abstractions c) different special casing for various performance or security related primitives?
I guess that i'm then wondering, to what extent are the profusion of api calls on bsd/linux (by comparison) due to which of the following: a) legacy of evolving richer apis while supporting older ones b) poorer factoring of abstractions c) different special casing for various performance or security related primitives?
Note that the BSD system calls do not cover the full functionality of OSX, and the mach interfaces are necessary. For example, sleep() needs to use the Mach message interfaces.
Why can't sleep() be implemented on top of select(0, 0, 0, 0, timeout)?
Embrace and extend? How is this different to Android vs. the Linux kernel?
"The kernel of NeXTSTEP is based upon the Mach kernel, which was originally developed at Carnegie Mellon University, with additional kernel layers and low-level user space code derived from select parts of BSD." [1]
NeXTSTEP became the basis for OS X. It meets the Single UNIX Specification [2].
[1] https://en.wikipedia.org/wiki/OS_X#History
[2] https://en.wikipedia.org/wiki/Single_UNIX_Specification#OS_X
NeXTSTEP became the basis for OS X. It meets the Single UNIX Specification [2].
[1] https://en.wikipedia.org/wiki/OS_X#History
[2] https://en.wikipedia.org/wiki/Single_UNIX_Specification#OS_X
That doesn't answer my question.
Both Android and OSX are listed under https://en.wikipedia.org/wiki/POSIX#Mostly_POSIX-compliant
What I would like to know is, is there any fundamental difference in how the two have derived from their predecessors, ie. Android kernel from the Linux kernel and OSX's XNU kernel from the NexTSTEP/BSD kernels, specifically in terms of compatibility concerns.
(Edit reply to below: Yes but my understanding is that the Android kernel changed fundamental system calls, such as those used for memory allocation, forcing everything userland to be rewritten. Is the 'two systems' of XNU you describe not, therefore, similar in its effect on compatibility?)
Both Android and OSX are listed under https://en.wikipedia.org/wiki/POSIX#Mostly_POSIX-compliant
What I would like to know is, is there any fundamental difference in how the two have derived from their predecessors, ie. Android kernel from the Linux kernel and OSX's XNU kernel from the NexTSTEP/BSD kernels, specifically in terms of compatibility concerns.
(Edit reply to below: Yes but my understanding is that the Android kernel changed fundamental system calls, such as those used for memory allocation, forcing everything userland to be rewritten. Is the 'two systems' of XNU you describe not, therefore, similar in its effect on compatibility?)
The Linux kernel has only one set of system calls. Everything you can do with Linux, you can do through it's syscall interface. Android doesn't change this, although it ships a different, more minimal libc.
The OSX kernel has two, due to it's bolt-on nature. Mach message passing and BSD system calls. As far as I am aware, the BSD system calls is an emulation layer, and is not entirely complete.
As a user, you don't care; the quirks of both systems are papered over by libc, which does the appropriate syscall dance to create a posix compliant system. (For an example of a Linux quirk, 'exit()' on Linux doesn't actually exit the process, it only exits the current thread).
The OSX kernel has two, due to it's bolt-on nature. Mach message passing and BSD system calls. As far as I am aware, the BSD system calls is an emulation layer, and is not entirely complete.
As a user, you don't care; the quirks of both systems are papered over by libc, which does the appropriate syscall dance to create a posix compliant system. (For an example of a Linux quirk, 'exit()' on Linux doesn't actually exit the process, it only exits the current thread).
For a bit of history regarding OS-X which may answer some of your questions, Amit Singh wrote about the history of NEXTSTEP[1] and also about XNU[2].
BTW, Singh's writings regarding OS-X are exceptionally well researched, detailed, and thorough. I highly recommend them for anyone interested in OS-X/XNU internals.
HTH
1 - http://www.kernelthread.com/publications/appleoshistory/7.ht...
2 - http://www.osxbook.com/book/bonus/ancient/whatismacosx/arch_...
BTW, Singh's writings regarding OS-X are exceptionally well researched, detailed, and thorough. I highly recommend them for anyone interested in OS-X/XNU internals.
HTH
1 - http://www.kernelthread.com/publications/appleoshistory/7.ht...
2 - http://www.osxbook.com/book/bonus/ancient/whatismacosx/arch_...
I don't know anyone involved, however, one would have to guess that it's a simple matter of people. Apple had people from NeXT. NeXT had people from CMU who worked on Mach. People work with what they are familiar with. Therefore, Apple has Mach.
They could have started from some other kernel. They didn't happen to do that. I don't think it means much.
They could have started from some other kernel. They didn't happen to do that. I don't think it means much.
I heard from somewhere that technically, OS X does not preserve backward compatibility for system calls, so you should use libc function's instead, otherwise it is an undefined behavior. Whereas Linux keeps a hard compatibility guarantee. Is that true? I guess Apple will really hesitate before commiting breaking changes, but I'm genuinely curious if they reserve the rights in thoery.
This is true - the compatibility boundary for OS X is at the link-against-libSystem layer. The actual interface between libSystem and the kernel is private to Apple and is not guaranteed to stay argument or ABI compatible. (this includes the syscall() function)
Several xnu syscalls are actually paired with a userspace wrapper function that does some extra stuff. Existing syscalls may even become wrapped in a subsequent release if needed.
Several xnu syscalls are actually paired with a userspace wrapper function that does some extra stuff. Existing syscalls may even become wrapped in a subsequent release if needed.
I don't know about OS X. You're right about Linux, though, and Torvalds will descend into an angry screed if you try to break the ABI. Windows, on the other hand, does not save the numbers between releases. http://j00ru.vexillium.org/ntapi_64/ is a table showing the system call table for each given release, and how sometimes it even changes between service packs of the same released OS.
> I heard from somewhere that technically, OS X does not preserve backward compatibility for system calls, so you should use libc function's instead, otherwise it is an undefined behavior.
I believe this article[1] definitively answers this as being the case.
> Whereas Linux keeps a hard compatibility guarantee.
Not being a troll here... Why would anyone care unless they are writing a libc replacement? Before anyone objects regarding compatibility with programs written for older versions of OS-X, there have been compatibility libraries distributed for every version I can recall. To wit, I run the Pages app from iWork '09 regularly.
1 - https://developer.apple.com/library/mac/qa/qa1118/_index.htm...
I believe this article[1] definitively answers this as being the case.
> Whereas Linux keeps a hard compatibility guarantee.
Not being a troll here... Why would anyone care unless they are writing a libc replacement? Before anyone objects regarding compatibility with programs written for older versions of OS-X, there have been compatibility libraries distributed for every version I can recall. To wit, I run the Pages app from iWork '09 regularly.
1 - https://developer.apple.com/library/mac/qa/qa1118/_index.htm...
> Why would anyone care unless they are writing a libc replacement?
- So that you can feel safe upgrading your kernel without upgrading your whole userspace. (Linus harps on this one a lot.)
- So that there can be multiple competing libc's. Linux is used in a lot of diverse environments, and the tradeoffs made by glibc (in terms of bloat, license, etc.) are not necessarily the best for everyone. Android does not use glibc, for example.
- So that you can isolate apps from each other using containers (hermetic chroot environments), where different containers may have entirely different libc. Docker (and my own Sandstorm.io) relies very heavily on Linux's syscall ABI compatibility promises.
- Because the syscall ABI is frankly a much clearer boundary than the libc ABI, and arguably easier to keep backwards-compatible. An app literally _can't_ break this boundary and access the private interfaces beneath (assuming a secure implementation, lol).
- So that you can feel safe upgrading your kernel without upgrading your whole userspace. (Linus harps on this one a lot.)
- So that there can be multiple competing libc's. Linux is used in a lot of diverse environments, and the tradeoffs made by glibc (in terms of bloat, license, etc.) are not necessarily the best for everyone. Android does not use glibc, for example.
- So that you can isolate apps from each other using containers (hermetic chroot environments), where different containers may have entirely different libc. Docker (and my own Sandstorm.io) relies very heavily on Linux's syscall ABI compatibility promises.
- Because the syscall ABI is frankly a much clearer boundary than the libc ABI, and arguably easier to keep backwards-compatible. An app literally _can't_ break this boundary and access the private interfaces beneath (assuming a secure implementation, lol).
> Not being a troll here... Why would anyone care unless they are writing a libc replacement?
Statically compiling the libc is common with proprietary Linux software, if I remember correctly.
Statically compiling the libc is common with proprietary Linux software, if I remember correctly.
Gotcha. I guess it's an OS expectation thing then. In OS-X, it's pretty much an assumption that dynamic linking is what's going to happen (good or bad).
Thanks for clarifying this point for me.
Thanks for clarifying this point for me.
Linux userland has various libcs available (klibc, glibc, eglibc, uclibc, dietlibc, musl, …), with distributions occasionally switching between them, and ABI-incompatible changes happen every once in a while. Source-distributed software does not have to care usually, but for binary-only distribution, statically compiling the libc is the easiest bet. Thus, the kernel has to maintain compatibility.
More than that, static linking is very difficult indeed as there is not even a crt for static linking let alone libc.a, you would have to write your own.
> Not being a troll here... Why would anyone care unless they are writing a libc replacement?
Linux doesn't have any single fixed libc, the system call interface is the public API. Indeed there are many different libc's used in different linux applications - glibc on many desktop distros, bionic libc on android, uclibc, musl and others on various embedded/specialized setups.
Linux doesn't have any single fixed libc, the system call interface is the public API. Indeed there are many different libc's used in different linux applications - glibc on many desktop distros, bionic libc on android, uclibc, musl and others on various embedded/specialized setups.
Linux's syscalls: http://man7.org/linux/man-pages/dir_section_2.html
(I have learned a lot by just going to that page and randomly clicking on things.)
(I have learned a lot by just going to that page and randomly clicking on things.)
This is documentation. It's not an ABI reference the same way the article is. The article is about the nuts and bolts of how user mode makes calls into the kernel, which manpages don't bother with.
If you want to learn the syscall ABI on Linux you need to look at <asm/unistd.h>. Seems like this file has gone through some refactorings since the last time I looked, but here's an older version: http://lxr.free-electrons.com/source/include/asm-generic/uni... [edit: ia32 here http://lxr.free-electrons.com/source/arch/x86/include/asm/un...] -- the constants in the __NR_* macros define the table that this article is attempting to build. [Basically, the interrupt handler for a syscall needs to know where to jump based on that index.]
If you want to learn the syscall ABI on Linux you need to look at <asm/unistd.h>. Seems like this file has gone through some refactorings since the last time I looked, but here's an older version: http://lxr.free-electrons.com/source/include/asm-generic/uni... [edit: ia32 here http://lxr.free-electrons.com/source/arch/x86/include/asm/un...] -- the constants in the __NR_* macros define the table that this article is attempting to build. [Basically, the interrupt handler for a syscall needs to know where to jump based on that index.]
Why so many duplicates?
For expample int nosys() is listed 185 times, each pointing to bsd/kern/subr_xxx.c
Because when you remove items from that list or otherwise change the order, you break binary compatibility.
The C library specifies the index into this table when making a syscall. You don't want a situation where the C library and the kernel are mismatched and disagree about what the syscalls are.
The safe way to remove a syscall is to change it to return ENOSYS. All the syscalls that come after it in the table therefore retain the same index.
The C library specifies the index into this table when making a syscall. You don't want a situation where the C library and the kernel are mismatched and disagree about what the syscalls are.
The safe way to remove a syscall is to change it to return ENOSYS. All the syscalls that come after it in the table therefore retain the same index.
nosys() is a stub which simply [EDIT: raises SIGSYS then] fails with ENOSYS, i.e. "syscall not implemented".
This makes me wonder, what would this look like for plan9? Just FS-related syscalls, and that's it? Curious...