This is a great response. The shady operators can ruin simple services like these :( I’m now curious how you’d ever practically deal with this. How does iCloud deal with encrypted illegal content? Surely they can’t penalize Apple in these situations?
Exactly. The code is adjusting for responsiveness. With less cpus you need a smaller minimum slice. As you have more cpus you can increase the slice and still schedule the same number of processes per second.
E.g. 1 ms slice with 1 core = 1000 process switches per second. With 2 cores you can increase the slice to 2 ms and still maintain the same number of switches per second for the system, but reducing the switches per second on each core to 500. This reduces the overhead for the scheduler.
It seems like at around 8 times the slice efficiency starts to go the other way, so they’ve limited it. Seems reasonable, but scheduler math is crazy.
Note, that this has nothing to do with the scheduler assignments per core which have clearly been working or people would’ve noticed!
I believe it just emits at least one packet on each system 'write' call. As long as your 'write' invocations are larger blocks then I'd expect you'd see very little difference with O_NDELAY enabled or disabled. I've always assumed you want to limit system calls so I'd always assumed it to be better practice to encode to a buffer and invoke 'write' on larger blocks. So this feels like a combination of issues.
Regardless, overriding a socket parameter like this should be well documented by Golang if that's the desired intent.