600K concurrent HTTP connections with Clojure and http-kit(http-kit.org)
http-kit.org
600K concurrent HTTP connections with Clojure and http-kit
http://http-kit.org/600k-concurrent-connection-http-kit.html
27 comments
I'm pretty impressed, and especially by the apachebench test -- to serve a spike of 30K reqs/s while having 600K idle connections is quite good.
Does anyone know how they managed to not only bypass the C10k limit, but bypass it by a factor of 60?
From what I understand C10K was mostly a RAM limitation. As system ram got bigger the limit has naturally risen. Here are some guys hitting a million in Erlang.
http://blog.whatsapp.com/index.php/2011/09/one-million/
HN discussion for the above link. https://news.ycombinator.com/item?id=3028547
http://blog.whatsapp.com/index.php/2011/09/one-million/
HN discussion for the above link. https://news.ycombinator.com/item?id=3028547
From my understanding, the C10K concept is more a thread problem than a RAM limitation. It happens when each request create its own thread on server (as in Apache or IIS). Computer have problem to run 10,000 threads in same times: too much time is consumed by the OS to find the one thread that is not blocked/locked or waiting for datadabase results.
Node.JS solve this problem by doing all in Asynchronous way (in only one thread). GO solve this pb by doing clever threads (goroutine). Functional programming seems to be very good on this issue (no need of lock mecanism and light thread).
Node.JS solve this problem by doing all in Asynchronous way (in only one thread). GO solve this pb by doing clever threads (goroutine). Functional programming seems to be very good on this issue (no need of lock mecanism and light thread).
[deleted]
Its based on netty which surpassed c10k long ago. More generally modern socket concurrency is pretty high using epoll
It's not based on netty:
> {:dependencies [[org.clojure/clojure "1.5.1"]]}
netty is in the dev dependencies, but for benchmarking only I would guess.
[edit] This project is one of the best thing that happened in the web area in clojure recently imho. Not only it is a game changer in performance/resource use, but it makes websocket and async in general trivial to use and actually production ready (same goes for its client). Some would mention Pedestal but it forces you to learn quite a few abstractions and is totally alien compared to the rest of the clojure web ecosystem and also only supports a single async "protocol" (SSE). But pedestal is about a full stack experience, integration with the client etc, so it certainly will attract other users.
> {:dependencies [[org.clojure/clojure "1.5.1"]]}
netty is in the dev dependencies, but for benchmarking only I would guess.
[edit] This project is one of the best thing that happened in the web area in clojure recently imho. Not only it is a game changer in performance/resource use, but it makes websocket and async in general trivial to use and actually production ready (same goes for its client). Some would mention Pedestal but it forces you to learn quite a few abstractions and is totally alien compared to the rest of the clojure web ecosystem and also only supports a single async "protocol" (SSE). But pedestal is about a full stack experience, integration with the client etc, so it certainly will attract other users.
Ah, interesting, I should have looked deeper. Well they're both almost certainly based on java NIO then, which once again leads us to epoll and friends.
Well, aleph already exists, but candidly http-kit has a simpler API for most tasks
edit upon review, not a huge diff between the two actually.
edit upon review, not a huge diff between the two actually.
Some would also mention Aleph, too.
https://github.com/ztellman/aleph
https://github.com/ztellman/aleph
10k isnt really a limit. In linux you should consider it being about 100 for every 1MB of ram (10%) of fs.file-max. 1 million isn't really hard nowadays
C10k was a relevant limit 10 years ago perhaps. It's only of historical interest now.
Java NIO has amazing performance. I'm surprised not more JVM languages taking advantage of it.
Netty has become the standard library for NIO on the JVM and is used widely by programmers in various JVM languages. It's also refreshing to see high performance NIO systems written without using Netty for a change.
Earlier discussion, primarily about benchmarking: https://news.ycombinator.com/item?id=5127251
So? Not really impressed. WhatsApp are getting over 2M concurrent HTTP connections with Erlang in a production system. And this was 1 year ago. http://blog.whatsapp.com/index.php/2012/01/1-million-is-so-2...
The Whatsapp 2 million connection result was on a 24 core box with a ton of RAM. The Clojure benchmark was pulled off on a 4-core desktop. Also, Whatsapp connections are mostly idle, only a small fraction of users would be sending a message in any given second.
Edit: It looks like the test was run only on localhost, which also means that it artificially removes much of the processing and actual network I/O factors from the equation.
Edit: It looks like the test was run only on localhost, which also means that it artificially removes much of the processing and actual network I/O factors from the equation.
This is a good point, the exact hardware comparison is as follows:
| Clojure | WhatsApp
---------------------------
CPU | i7-2600 | Xeon X5675
Clock| 3.4GHz | 3.07GHz
Cores| 4 | 24
RAM | 16G | 96G
Conns| 600,000 | 2,000,000
Then what are we really measuring? Is it how many connections we get can get on as small a machine as possible? If so then I am sure I can get 600K connections with Erlang in a smaller system. At least running a benchmark.
There is one very significant difference and that is that the Clojure benchmark was just a benchmark while for WhatsApp it was their actual system doing something real. They have mentioned to me that they have managed to push their system to 3M connections but not in production.
There is one very significant difference and that is that the Clojure benchmark was just a benchmark while for WhatsApp it was their actual system doing something real. They have mentioned to me that they have managed to push their system to 3M connections but not in production.
This does not measure 600k concurrent active connections because each client only sends a request once every 5-45s. To see how many concurrent active connections you can support you need to understand how much RAM each connection consumes and how much RAM is available to your application.
Are they using the same box to run server and clients? I can't figure that out from TFA.
Wouldn't it be better if clients are running on different machine(s)?
Wouldn't it be better if clients are running on different machine(s)?
Does this process also have 600K threads?
No, it's evented, I think it's setup to use 4 worker threads by default.
[edit]
https://github.com/http-kit/http-kit/blob/master/src/org/htt...
https://github.com/http-kit/scale-clojure-web-app/blob/maste...
[edit]
https://github.com/http-kit/http-kit/blob/master/src/org/htt...
https://github.com/http-kit/scale-clojure-web-app/blob/maste...
Old news [1].
[1] https://news.ycombinator.com/item?id=5127251
[1] https://news.ycombinator.com/item?id=5127251
Y'know... that little "K" abbreviation on the end is a pretty important detail. I skimmed the article about three times, thinking to myself: "600 users... Really? Big deal."
It wouldn't hurt to actually emphasize "600,000" by explicitly expressing all of the zeroes.
It wouldn't hurt to actually emphasize "600,000" by explicitly expressing all of the zeroes.
"That little K abbreviation" is kind of a common thing... I'm not sure what your experience is, but in order to understand a lot of what gets discussed on the web in terms of performance, you'll need to improve your attention to detail.