Ask HN: Fastest concurrent HTTP server library for Python?
For my project, I am running a high-performance web service with really long response times (~30 seconds) and high concurrency (able to do multiple requests at once without significant slowdown in response time). I am using the built-in BaseHTTPService with ThreadingMixIn right now for the web service, but it is choking with a load of even 3-4 requests per second. Apparently, the high response time is a difficult requirement, since it rules out libraries like Tornado who do a serial FIFO "one at a time" processing of requests despite being asynchronous. I looked at several libraries, and I am leaning towards syncless right now. There are many of these libraries though, and really, it comes down to the "tyranny of choice." The important thing is actually owning the server code in Python rather than having say nginx call my code for each job since I am interfacing with a device driver that needs to have an open context the entire time the server is running. This is another requirement that is pretty unique to my project. What do you suggest I use?
7 comments
I vote Gevent because of its speed, small resource footprint, and its simple synchronous-looking API, but be aware of its limitations. The greenlets (threads) spawned by gevent run in a single OS thread, so you won't be able to take full advantage of all the cores on your server with Gevent alone. Also, Greenlets run cooperatively meaning each one gains control of the CPU until it's done with its computation or hits an IO call. This is great for IO bound computation, not so much for CPU bound work. Even with these limitations, I still think it'll work well with in your application.
Check out this comparison of Python servers: http://nichol.as/benchmark-of-python-web-servers
Check out this comparison of Python servers: http://nichol.as/benchmark-of-python-web-servers
BaseHTTPServer.serve_forever() polls every 0.5s by default, so you might want to use BaseHTTPServer.handle_request() instead in a "while True:" loop. This might get you out of your immediate pickle.
This question is more suitable for SO than HN.
Given a four core CPU, if each request is lasting 30 seconds without I/O, then each request is going to use 1 core. On a 4-core CPU, you would only be able to serve 4 requests within a 30 second span. What is the request doing for 30 seconds? How much of it is actual CPU-bound work and what amount is I/O.
Given a four core CPU, if each request is lasting 30 seconds without I/O, then each request is going to use 1 core. On a 4-core CPU, you would only be able to serve 4 requests within a 30 second span. What is the request doing for 30 seconds? How much of it is actual CPU-bound work and what amount is I/O.
The request is harvesting data from Facebook. For users with a tremendous amount of data, batch requests must be made, so it can take a while sometimes.
I can't say authoritatively for Python, but Twisted is probably what you need here (I have used both Node and EventMachine to do things like this, and I'm pretty sure Twisted is similar). The event reactor model will let you defer the request while it's doing things like performing external IO, so your application is freed up to handle other requests in the meantime.
Gevent will handle requests with high response times.
Seconded. I also recommend using gunicorn in conjunction with gevent (if you are doing http servers) to easily get multiple service "child/worker" processes in order to make use of multi-cores/cpus.