I am not a Windows guy but I (with help) managed to get IOCP working for this in a basic prototype. Will share publicly soon. I also sketched out an IoRing version (if you are interested in helping debug and flesh that out let me know!).
Main learnings: the IOCP version can't do asynchronous flush! Which we want. The IoRing version can! But it can't do scatter/gather AKA vector I/O yet! Which is an essential feature for buffer pool implementation. So actually I am basically waiting for IoRing to add support for that before taking it too seriously (I can see they are working on it because the ops are present in an enum, it's just that the build functions are missing).
So my guess is that in a year or so we should be able to run all PostgreSQL disk I/O through IoRing on Windows. Maybe?
Another complications is that it really wants to be multithreaded (consuming completions for IOs started in another process requires a lot of hoop jumping, I made it work but...) This will resolve itself naturally with ongoing work to make PostgreSQL multithreaded.
The next problem is that IoRing doesn't support sockets! So in future work on async networking (early prototypes exist) we will likely also need IOCP for that part.
BTW I have patches for PostgreSQL AIO on FreeBSD, which I will propose for v19. It works pretty well! I was trying to keep out of Andres's way for the core architectural stuff and basic features ie didn't want to overload the pipes with confusing new topics for v18 :-)
I suppose this means that they still have a firm intention to produce eventually, hopefully even with an updated chipset given that the original parts are lost ("ODM is effectively unwilling to release remaining produced stock and the pre-purchased chipsets"!) Obviously you know all that, fellow backer, but just in case anyone else is interested in this debacle:
I had various Psion models in the 90s. Several of the people involved in the design of those things went to do modern-ish devices with Psion series 5-style keyboards, including the Cosmo Communicator. I was curious enough to back the Astro Slide 5G project, which attempted to "reverse the clamshell" to make a device with Psion series 5-style keyboard that closes to resemble a modern standard all-screen rectangular slab. It appears to have gone horribly wrong during final manufacturing at a Chinese factory that closed down during COVID, with thousands of IndieGoGo backers not having received their Astro Slide devices after several years (I am one). A shame. Anyway this article shows the Psion 5 and the Communicator side-by-side. That's a way out of date device now running ancient Android on a slow CPU, and sadly the refresh project seems to have died on the vine :-(
After it was dropped from PostgreSQL, a team from IBM showed up on the mailing list (see one of my other answers for link), so perhaps that is now going to happen! I think it's a case that they were using it, but didn't realise that the project was on the verge of dropping it for years due to lack of interested maintainer & resources. Open source is funny like that. Deleting it was certainly one way to get their attention.
I've been able to debug PostgreSQL issues reported by [closed] Solaris users by booting illumos inside a virtual machine on my laptop in minutes...
I want to keep AIX support, for, well mainly irrational nostalgic reasons -- PostgreSQL used to run on pretty much the entire Unix family tree, and I wrote lots of code on AIX for a decade. But they make it hard. I don't know why an OS vendor wouldn't make an image of an OS available to developers as conveniently as possible (it is even possible to boot recent AIX versions on QEMU if you have the patience... the hypothesis is that they might have done work to make that possible, 'cause it didn't work in earlier versions; but you can't get an image of OS, so shrug).
We (the denizens of [email protected]) didn't see any AIX users or developers or other interested parties in our community... There was a group in France, but they disbanded... However, after AIX was dropped in the PostgreSQL 17 development cycle, suddenly we heard from IBM who run products on it. So perhaps it will make a comeback in 18? That was the latest idea, anyway, let's see.
POSTGRES was developed on SunOS (and its close cousin 4.3BSD or 4.2?). It's not so hard to support Solaris, as it was so influential that Linux uses a lot of similar things, including for example the ELF format and linker details, the sort of thing that developers have to maintain. So the burden is lower than (say) AIX. There is also someone from the Solaris community who feeds and waters a modern Solaris build farm animal (buildfarm.postgresql.org). Same goes for illumos (= forked from OpenSolaris, like Solaris 10), people run build farm animals and help test occasionally as required etc, which makes a big difference. That's my take, anyway (speaking as someone who works on PostgreSQL portability).
This is fantastic work, thanks. Hmm, what else... let's see... Xenix also really, really wants to be free! What a magnificent piece of forgotten computer history it is. https://en.wikipedia.org/wiki/Xenix
... but the BSDs did have support for SystemV binaries, through modules compat_svr4 and compat_ibcs2, so the grand parent comment made perfect sense to me. Not sure why there were two and I don't plan to go down that rabbit hole today, but perhaps due to SVR4 vs SVR3 differences, and ELF vs COFF, shrug.
Fun read. I'm pretty sure there were ways to run SCO SysV binaries unmodified on Linux (and I think FreeBSD and others), back in the day. Maybe something to do with ibcs2?
One technical problem is the WAL records changes to "shared" relations (stuff like the catalogues of databases and users that are global to the whole "cluster", not just one database), and those are mixed up with changes to per-database objects (ie most WAL activity).
FWIW I just removed pg_attribute_always_inline from those three comparator functions in tuplesort.c and it made no difference to the generated code under GCC 10 at -O2. If we can show that's true for Clang and MSVC too, maybe we should just take them out. Edit: But I don't really mind them myself, they do document the intention of the programmer in instantiating all these specialisations...
Speaking as the author of that change, and also as a long time user of C++ (in other projects), I hear ya! But I don't have the power to make PostgreSQL switch to C++. Years ago, std::sort() vs qsort() was almost a standard example of how C++ can be faster than C.
src/lib/sort_template.h is an example of a pseudo template function, but we also have at least one pseudo template container in src/lib/simplehash.h.
And if you want to see some "policy based design" (C++ community term) in PostgreSQL C code, check out src/backend/exec/nodeHashjoin.c. It uses forced inlining to "instantiate" two different versions of the hash join executor code, one with parallelism and one without, so that we don't have all the "am I in parallel mode?" branching/code size in the generated code.
Main learnings: the IOCP version can't do asynchronous flush! Which we want. The IoRing version can! But it can't do scatter/gather AKA vector I/O yet! Which is an essential feature for buffer pool implementation. So actually I am basically waiting for IoRing to add support for that before taking it too seriously (I can see they are working on it because the ops are present in an enum, it's just that the build functions are missing).
So my guess is that in a year or so we should be able to run all PostgreSQL disk I/O through IoRing on Windows. Maybe?
Another complications is that it really wants to be multithreaded (consuming completions for IOs started in another process requires a lot of hoop jumping, I made it work but...) This will resolve itself naturally with ongoing work to make PostgreSQL multithreaded.
The next problem is that IoRing doesn't support sockets! So in future work on async networking (early prototypes exist) we will likely also need IOCP for that part.