Benchmark of Asynchronous Servers in Python(nichol.as)
nichol.as
Benchmark of Asynchronous Servers in Python
http://nichol.as/asynchronous-servers-in-python
2 comments
I don't know that this was an intentional design decision but being able to fail fast and return may enable more aggressive load balancing decisions. Ideally that should be tunable for different architectures of course but the sooner you can get an overloaded node out of the pool the better.
Naively, you take N cpu-seconds to handle one request (say N=0.01).
That means you can't handle more than 1/N requests/second before you start to slow down all requests.
So if you don't have a bound of 1/N requests/second on your server, you'll have a latency increase if the connection rate goes above that. At some pt that latency increase will be problematic.
It's also may be worse than the naive assessment above, since resources required to process one request might be flat but might go up as total #conns goes up (you'll hit a wall if your memory usage is enough to get into swap, you may be ok with an O(1) lookup from socket->connection state tho).
A decision by the server to refuse a TCP connect when it's at a configured limit (or dynamically determined limit) can be detected by a load balancer more easily than the latency on what might be varied requests.
Realworld example: people like to set MaxClients on apache, and for good reasons.
That means you can't handle more than 1/N requests/second before you start to slow down all requests.
So if you don't have a bound of 1/N requests/second on your server, you'll have a latency increase if the connection rate goes above that. At some pt that latency increase will be problematic.
It's also may be worse than the naive assessment above, since resources required to process one request might be flat but might go up as total #conns goes up (you'll hit a wall if your memory usage is enough to get into swap, you may be ok with an O(1) lookup from socket->connection state tho).
A decision by the server to refuse a TCP connect when it's at a configured limit (or dynamically determined limit) can be detected by a load balancer more easily than the latency on what might be varied requests.
Realworld example: people like to set MaxClients on apache, and for good reasons.
FWIW terrible example, that's why Apache is always blowing up when it gets much load.
I'd much rather a server service all requests properly than fail. Then you can measure the load yourself, and load balance appropriately, knowing that all requests are being serviced properly.
I'd much rather a server service all requests properly than fail. Then you can measure the load yourself, and load balance appropriately, knowing that all requests are being serviced properly.
> I'd much rather a server service all requests properly than fail.
Depends what you mean by "properly". If you mean "within N seconds", then when overload takes you over the point at which reqs take N seconds, all your reqs are now failing to be served properly.
Whereas if you have a cap at M concurrent reqs (which you calculate so that your platform can service each of those M reqs within N secs), you get to handle M concurrent reqs properly under overload.
So, not under overload - same behaviour. Under overload, a cap allows M concurrent reqs to be handled properly, no cap allows none.
What alternative are you suggesting? Let the platform run slower as load increases until your alarm system pings you at a threshold and you then provision more servers?
Depends what you mean by "properly". If you mean "within N seconds", then when overload takes you over the point at which reqs take N seconds, all your reqs are now failing to be served properly.
Whereas if you have a cap at M concurrent reqs (which you calculate so that your platform can service each of those M reqs within N secs), you get to handle M concurrent reqs properly under overload.
So, not under overload - same behaviour. Under overload, a cap allows M concurrent reqs to be handled properly, no cap allows none.
What alternative are you suggesting? Let the platform run slower as load increases until your alarm system pings you at a threshold and you then provision more servers?
Isn't that a double edged sword considering DOS attacks? We might be able to learn more about it by looking at memory usage as well.
I don't see why it's related to DOS attacks. You would usually deal with DOS in other ways - limit req/s on a source IP basis, etc, or with iptables rules.
Failure at all, seems an extremely undesirable situation. Surely. surely it's better to be slow than to fail? eg if the server is under extreme load, just means requests take a bit longer to get processed, rather than dropping them.
Failure at all, seems an extremely undesirable situation. Surely. surely it's better to be slow than to fail? eg if the server is under extreme load, just means requests take a bit longer to get processed, rather than dropping them.
I don't claim to be particularly competent at dealing with DOS attacks, I'm really just asking questions here. What you say seems right to me up to a point.
But once the server gets so overwhelmed that it has no chance to ever process those requests, it makes little sense to queue them all up for delivery in 20 minutes, does it?
Also, I wonder whether queuing them all up uses scarce resources. That's why I was asking about memory usage.
But once the server gets so overwhelmed that it has no chance to ever process those requests, it makes little sense to queue them all up for delivery in 20 minutes, does it?
Also, I wonder whether queuing them all up uses scarce resources. That's why I was asking about memory usage.
A couple of possibilities:
1. Having too many open file handles
2. HTTP timeouts.
1. Having too many open file handles
2. HTTP timeouts.
This may be a dumb question, but I really can't imagine. What reasonable real-world process would need more than 1400 HTTP connections per second (which is the worst performer) on a single process and single server?..
If you're doing Comet type things for a start.
Mibbit does around 2.5k/sec, but that's split over a few servers for memory/failover reasons.
I agree though, probably a niche use case, until more 'real time' stuff happens.
Mibbit does around 2.5k/sec, but that's split over a few servers for memory/failover reasons.
I agree though, probably a niche use case, until more 'real time' stuff happens.
Give you extra time to warn about an DDOS.
"The performing servers return less overall errors. There is however, one exception. Cogen was able to return ALL its requests successfully no matter how hard it was hammered."
Why would the others decide to drop connections or return errors? Surely that makes them pretty unusable?
I wonder why they start dropping/erroring, and in what form?