Ask HN: Event loop vs. Threads
19 comments
Take a look at the ring buffer data structure from the LMAX architecture (my talk on LMAX http://codemonkeyism.com/lmax-architecture-high-performance-... or http://martinfowler.com/articles/lmax.html from Martin Fowler).
They have incoming work, I/O for incoming and outgoing work is done multi threaded while work on events is single threaded.
The architecture is quite clever as more than one event processor can work on the data structure and event processors can have dependencies on each other. Independent processors can race past each other.
It was written for trading and might be portable to C++, but if you consider switching to JS it might also be ok for you to switch to Java. Their framework is called Disruptor and open source
http://code.google.com/p/disruptor/
"LMAX aims to be the fastest trading platform in the world. Clearly, in order to achieve this we needed to do something special to achieve very low-latency and high-throughput with our Java platform. Performance testing showed that using queues to pass data between stages of the system was introducing latency, so we focused on optimising this area."
They have incoming work, I/O for incoming and outgoing work is done multi threaded while work on events is single threaded.
The architecture is quite clever as more than one event processor can work on the data structure and event processors can have dependencies on each other. Independent processors can race past each other.
It was written for trading and might be portable to C++, but if you consider switching to JS it might also be ok for you to switch to Java. Their framework is called Disruptor and open source
http://code.google.com/p/disruptor/
"LMAX aims to be the fastest trading platform in the world. Clearly, in order to achieve this we needed to do something special to achieve very low-latency and high-throughput with our Java platform. Performance testing showed that using queues to pass data between stages of the system was introducing latency, so we focused on optimising this area."
I think a lot has been written on why threads, locks etc. are hard to program (= i.e. hard to maintain) and are considered the "assembly of parallel programming".
As for efficiency: Threads and Processes are what the OS offers. Any parallel programming model will have to use these in one way or another, so does Node.js. The discussion is kind of similar to assembly vs. high level programming languages. You can always write an assembly program that is as fast as the program compiled from a high level language. However, it will take you a lot more time to write it.
In the end, it comes down to choosing the right tool for the job. Therefore you should try find out what people with a similar job choose and what their experiences are; and also for which job a framework has been created.
E.g. Node.js is good at a job where there are lots of events and I/O is involved. If your job is number crunching, then Node.js is the wrong tool.
As for efficiency: Threads and Processes are what the OS offers. Any parallel programming model will have to use these in one way or another, so does Node.js. The discussion is kind of similar to assembly vs. high level programming languages. You can always write an assembly program that is as fast as the program compiled from a high level language. However, it will take you a lot more time to write it.
In the end, it comes down to choosing the right tool for the job. Therefore you should try find out what people with a similar job choose and what their experiences are; and also for which job a framework has been created.
E.g. Node.js is good at a job where there are lots of events and I/O is involved. If your job is number crunching, then Node.js is the wrong tool.
+1, if your job is number crunching, stay the hell away from JavaScript as it will bite you in the ass. http://stackoverflow.com/questions/307179/what-is-javascript...
You will find these problems have been solved: https://github.com/substack/node-bigint
I'm kind of playing in the same space, although I tend to be more a hardware/assembler guy. Im currently prototyping with nodejs and C/C++ with inline assembler for AVX crunching.
I would view Node.js as an effective tool for leveraging js/v8. What's attractive for me when prototyping is throwing js/HTML UI over the top without having to switch gears. Look at raw node as an IO multiplexer and dispatcher but not a compute capable platform. The GC is weak for large data sets and the CPU efficiency is extremely poor for any heavy processing - very hard to manage your cache lines efficiently. Node sweet spot is packet and stream switching in webby stacks where the solid http, ssl and so forth are invaluable. Check out fabric (I forget the name) if you want to look at extracting more from JavaScript.
Suggest you check out LMAX and kx systems. And think how close you can get to a pure event sourced or stream processing model.
You will need the equivalent of one thread per (hyper)core to maximise effective instructions per clock - whether you need to do that will depend on your sustained memory bandwidth. So there a few C/C++ threads is not necessarily a bad thing.
Its possible to build a world class system in erlang with custom DSP, FPGA logic or using a GPU farm if you have the budget. This is the approach I would use if you have millions of decision sources.
If you want to maximise performance on x86 with a simple code base - and leverage SIMD it's hard to beat a combination of intel fortran and intel C/C++ compiler. You can roll your own messaging layer and put the compute node code in fortran - where you'll get great AVX throughput out of ifort.
I would view Node.js as an effective tool for leveraging js/v8. What's attractive for me when prototyping is throwing js/HTML UI over the top without having to switch gears. Look at raw node as an IO multiplexer and dispatcher but not a compute capable platform. The GC is weak for large data sets and the CPU efficiency is extremely poor for any heavy processing - very hard to manage your cache lines efficiently. Node sweet spot is packet and stream switching in webby stacks where the solid http, ssl and so forth are invaluable. Check out fabric (I forget the name) if you want to look at extracting more from JavaScript.
Suggest you check out LMAX and kx systems. And think how close you can get to a pure event sourced or stream processing model.
You will need the equivalent of one thread per (hyper)core to maximise effective instructions per clock - whether you need to do that will depend on your sustained memory bandwidth. So there a few C/C++ threads is not necessarily a bad thing.
Its possible to build a world class system in erlang with custom DSP, FPGA logic or using a GPU farm if you have the budget. This is the approach I would use if you have millions of decision sources.
If you want to maximise performance on x86 with a simple code base - and leverage SIMD it's hard to beat a combination of intel fortran and intel C/C++ compiler. You can roll your own messaging layer and put the compute node code in fortran - where you'll get great AVX throughput out of ifort.
Hi, I work at a company that makes heavy use of event loops. I'll try to accurately convey what I know:
From an efficiency standpoint, using event loops requires much less memory, but marginally more CPU time than threaded approaches.
In terms of maintenance, you're essentially writing your programs in continuation passing style which means error handling is explicit everywhere. There are tools that allow you to hide this like streamlinejs, Haskell's continuation monad transformer and to a lesser degree, promises/futures/deferrables. If you decide to stick with callback passing, then a good knowledge of functional programming is useful as all loops require recursion. If your application is single threaded, that tends to make finding race conditions easier, but on the whole I'd say it's harder to write good asynchronous code than it is to write good synchronous code.
From an efficiency standpoint, using event loops requires much less memory, but marginally more CPU time than threaded approaches.
In terms of maintenance, you're essentially writing your programs in continuation passing style which means error handling is explicit everywhere. There are tools that allow you to hide this like streamlinejs, Haskell's continuation monad transformer and to a lesser degree, promises/futures/deferrables. If you decide to stick with callback passing, then a good knowledge of functional programming is useful as all loops require recursion. If your application is single threaded, that tends to make finding race conditions easier, but on the whole I'd say it's harder to write good asynchronous code than it is to write good synchronous code.
At high concurrency, I'd argue you will probably end up being more cpu efficient as well. The cost of context switching effectively larger frames and getting into and out of privileged mode can get expensive.
I don't know what exactly you are referring to with high concurrency, but if you mean "high number of tasks to work on in parallel" then a ThreadPool will eliminate the problems you are describing.
It probably depends, but the cost of creating a heap allocated closure and then running it and deallocating it can be quite high against the cost of 2 context switches.
Thought this was the definitive blog post on the issue:
http://sheddingbikes.com/posts/1280829388.html
"epoll is faster than poll when the active/total FD ratio is < 0.6, but poll is faster than epoll when the active/total ratio is > 0.6."
http://sheddingbikes.com/posts/1280829388.html
"epoll is faster than poll when the active/total FD ratio is < 0.6, but poll is faster than epoll when the active/total ratio is > 0.6."
Really not sure, why someone voted this down? The only open experiments on the issue of threads vs. events - at least I don't know of any other.
That has nothing to do with threads vs events at all. It is Zed going "I am going to make up some imaginary fallacies about poll and epoll and then prove them wrong, by proving epoll does exactly what it says it does". These are both ways of checking file descriptors for activity, which says nothing about threads vs event loops vs cps.
Events can either mean single-thread/event queue (like the LMAX architecture), or async/sleeping with wake up on event trigger. Or both.
Poll usually is run with threads on incoming connections/work, while epoll is keeping conections sleeping and wake up on incoming work. So in which way does this have "nothing to do with threads vs events at all"?
For sure the ratio of sleeping vs. active connections is important in the discussion on node.js model vs. a thread pool with a thread-per-connection setup. If you have very few active connections, poll and a thread-per-connection setup is more efficient (which does say nothing about the ratio of threads per CPU, context switches etc.), if you have a high number of inactive connections, say server-push or chat scenarios, event based connections are more efficient. The OP asked for efficiency.
I don't think the node.js is in any way relevant or efficient for event queues / single thread setups in high throughput scenarios like LMAX where poll/epoll is not relevant. But for node.js, poll vs. epoll setups are highly relevant as they show e.g. node.js vs. standard thread based behaviour.
Poll usually is run with threads on incoming connections/work, while epoll is keeping conections sleeping and wake up on incoming work. So in which way does this have "nothing to do with threads vs events at all"?
For sure the ratio of sleeping vs. active connections is important in the discussion on node.js model vs. a thread pool with a thread-per-connection setup. If you have very few active connections, poll and a thread-per-connection setup is more efficient (which does say nothing about the ratio of threads per CPU, context switches etc.), if you have a high number of inactive connections, say server-push or chat scenarios, event based connections are more efficient. The OP asked for efficiency.
I don't think the node.js is in any way relevant or efficient for event queues / single thread setups in high throughput scenarios like LMAX where poll/epoll is not relevant. But for node.js, poll vs. epoll setups are highly relevant as they show e.g. node.js vs. standard thread based behaviour.
>Poll usually is run with threads on incoming connections/work, while epoll is keeping conections sleeping and wake up on incoming work. So in which way does this have "nothing to do with threads vs events at all"?
Poll and epoll are trivial implementation details in both cases. Nobody is going to be choosing threads vs event loops based on poll vs epoll, you are using one or the other in both cases, and it is no more relevant to the decision than what you had for breakfast in either case.
Poll and epoll are trivial implementation details in both cases. Nobody is going to be choosing threads vs event loops based on poll vs epoll, you are using one or the other in both cases, and it is no more relevant to the decision than what you had for breakfast in either case.
Current implementation of node.js does not ensure in any way a parallelism mechanism like the system threads. Please do note that node.js event loop works within a context of a single event que.
What does that result in?
- If you have lots of code that is blocking on IO operations (like file/socket) then you will see some improvements in performance. - If your code utilizes your cpu, then node.js will be slower then your current thread implementation.
What does that result in?
- If you have lots of code that is blocking on IO operations (like file/socket) then you will see some improvements in performance. - If your code utilizes your cpu, then node.js will be slower then your current thread implementation.
Hi villagefool,
so you've lots of legacy C++, then I'd suggest you better stay there and rewrite performance critical parts effectively with ANSI C code.
Honestly I think StackOverflow is a better Platform for questions like this. You'll see that the answer to your question is: "Yes, use both." source: http://stackoverflow.com/questions/953428/event-loop-vs-mult... I think YCombinator is more of a community of entrepreneurs and investors, who can give you concrete advice on questions regarding technology decisions, but even though you'll also get quality answers to to CompSci questions, those type of questions are betters answered over there at SO.
Coding-Standards exist to allow "easier maintenance" of your application. You better check if there's an ISO-Standard for your branch, that defines the best practices in your business.
Here are "Google's C++ Guidelines" for example: http://google-styleguide.googlecode.com/svn/trunk/cppguide.x... but you'll need Coding Guidelines that fit better to your branch. Maybe you'll find some guidelines on the pages of the SEC http://www.sec.gov
I think there are people who automatically build up prejudices when you say "App" to something large like a Trading-Platform, but that mustn't be the case. I believe there are also people who'll think that you know what they want when you talk about complex things in the form of an "App" (something they know). Just be aware of it.
Can you answer me a question? How is Cuppertino, CA for a Software-Developer? (I won't work for Apple, just about to stay there for a while)
so you've lots of legacy C++, then I'd suggest you better stay there and rewrite performance critical parts effectively with ANSI C code.
Honestly I think StackOverflow is a better Platform for questions like this. You'll see that the answer to your question is: "Yes, use both." source: http://stackoverflow.com/questions/953428/event-loop-vs-mult... I think YCombinator is more of a community of entrepreneurs and investors, who can give you concrete advice on questions regarding technology decisions, but even though you'll also get quality answers to to CompSci questions, those type of questions are betters answered over there at SO.
Coding-Standards exist to allow "easier maintenance" of your application. You better check if there's an ISO-Standard for your branch, that defines the best practices in your business.
Here are "Google's C++ Guidelines" for example: http://google-styleguide.googlecode.com/svn/trunk/cppguide.x... but you'll need Coding Guidelines that fit better to your branch. Maybe you'll find some guidelines on the pages of the SEC http://www.sec.gov
I think there are people who automatically build up prejudices when you say "App" to something large like a Trading-Platform, but that mustn't be the case. I believe there are also people who'll think that you know what they want when you talk about complex things in the form of an "App" (something they know). Just be aware of it.
Can you answer me a question? How is Cuppertino, CA for a Software-Developer? (I won't work for Apple, just about to stay there for a while)
One year ago, I wrote an implementation of a concept I had. This implementation is a system daemon, which manages a lot of I/O work. The prototype was in bash, the actual system daemon was in C.
I am extremely proud of my recent decision to rewrite it in JavaScript and use Node.js. In the C implementation, I used standard FS functions (e.g. unlink, fwrite, fread) and of course, they are synchronous. In order to optimize the I/O, the entire daemon needed to be rewritten to use threads for the basic worker units, which were at least 5-6, each doing a very simple job. The alternative was to use async FS library in C, but I had to name every single callback.
So I scratched the C code and rewrote the daemon in Node.js. It is much faster because it manages to utilize the system resource much more efficiently, and the code is 3-4 times smaller.
The point is. If you have to do a lot of computation, DO NOT switch to Node or DO switch to Haskell. If you simply have to manage I/O operations, writing in Node might actually decrease complexity.
I am extremely proud of my recent decision to rewrite it in JavaScript and use Node.js. In the C implementation, I used standard FS functions (e.g. unlink, fwrite, fread) and of course, they are synchronous. In order to optimize the I/O, the entire daemon needed to be rewritten to use threads for the basic worker units, which were at least 5-6, each doing a very simple job. The alternative was to use async FS library in C, but I had to name every single callback.
So I scratched the C code and rewrote the daemon in Node.js. It is much faster because it manages to utilize the system resource much more efficiently, and the code is 3-4 times smaller.
The point is. If you have to do a lot of computation, DO NOT switch to Node or DO switch to Haskell. If you simply have to manage I/O operations, writing in Node might actually decrease complexity.
You might consider the Actor model. Event loop code can be a mess and threaded code while simpler because it appears linear eventually has such complicated runtime behaviours it also becomes a dangerous mess. And Actor model combines the best of both worlds. Messages are queued to an actor which combines a state machine plus its own thread or a slice of a thread. The state machine aspect and the centralization of the code around an actor object and the reliance only messages make it conceptually easy to understand and program. Actors can act as endpoints in protocols, services, publish/subscribe, timers etc so they offer a high level of abstraction away from lower level frameworks.
If one of my developers posted something like this, I would fire them on the spot.
You've got a "lot of legacy C++ code", and it seems like you're just randomly deciding whether or not to port it. You're basing this decision not on measurement, team considerations, or the needs of your project. Instead you're soliciting opinions from random people on the internet on what is a very religious issue.
If you have problems with your code (performance, maintainability, debugging, whatever), then go fix that. Maybe switching to Node is the right thing to do. But you'd be a fool to make that decision based on something someone said on Hacker News.
You've got a "lot of legacy C++ code", and it seems like you're just randomly deciding whether or not to port it. You're basing this decision not on measurement, team considerations, or the needs of your project. Instead you're soliciting opinions from random people on the internet on what is a very religious issue.
If you have problems with your code (performance, maintainability, debugging, whatever), then go fix that. Maybe switching to Node is the right thing to do. But you'd be a fool to make that decision based on something someone said on Hacker News.
Why be so negative? At least he or his team is taking some initiatives.
Funny, that almost all the programmers I know posted questions at one time or other on stackoverflow or codeproject. Do you mean to say all these guys taking opinions from strangers need to be fired ?
That's a bit unfair.
The OP uses the word "we" implying some sort of team decision. And asks for references, implying this is still under research.
The OP uses the word "we" implying some sort of team decision. And asks for references, implying this is still under research.
To be fair, he did ask for "references to good constructive material" rather than just "something someone said on Hacker News". They were looking for sources that discuss the issue, rather than someone on HN to give them the answer straight out.
Erlang is something else you might want to look at, but you really do have to go into more detail... Node.js and Erlang are both potentially a lot slower than C++.
While not C++ related, this category page on the Tcl'ers wiki contains links to several articles discussing event loops: http://wiki.tcl.tk/_/ref?N=8558
Tk has had an event loop based paradigm from its start, and straight Tcl also has the ability to explicitly enter an event loop.
Tk has had an event loop based paradigm from its start, and straight Tcl also has the ability to explicitly enter an event loop.
It depends on many variables, including but not limited to anticipated workload, the nature of the hardware, and parallelism of the underlying workload. It would help if you gave more information about the task at hand.
You cannot expect reasonable answers without being more specific.
Neither? _Bell Labs and CSP Threads_ by Russ Cox, http://swtch.com/~rsc/thread/, as used in #golang.
Most developers with no background in electroncis do not understand asynchronuous paradigm : Transitions = factorial(state).
In best case if they don't confuse states and transitions (wich is common) you'll end up with a spaghetti code where goto are replaced with callbacks on events. In common case they will make intricated state models without making the docs (state transition diagrams are a MUST have (like RFCs on network protocol)).
In the multi-threading context, most devs don’t fully grasp the concurrency problem (which is still an asynchronous problem).
So if you want to stay safe, use multithreading with disjoint data. Map Reduce is actually a pretty idiot proof paradigm for multi-threading. It only requires your data to be smartly shardable.
Executive note : - if event model : have all state transition models DOCUMENTED ; - if multi-threading : once you have shared a context (config), uncouple all the data passed to your thread (and handle a SIGHUP to reload the conf safely).
Event model done wrong will cry havoc on your code maintability the same as multitreading done wrong.
Executive note : - if event model : have all state transition models DOCUMENTED ; - if multi-threading : once you have shared a context (config), uncouple all the data passed to your thread (and handle a SIGHUP to reload the conf safely).
Event model done wrong will cry havoc on your code maintability the same as multitreading done wrong.
> Transitions = factorial(state). In best case if they don't confuse states and transitions (wich is common) you'll end up with a spaghetti code where goto are replaced with callbacks on events.
Hey could you expand on this? I don't understand what you mean by transitions = factorial(state) and why this leads to spaghetti code when you confuse state and transitions.
Hey could you expand on this? I don't understand what you mean by transitions = factorial(state) and why this leads to spaghetti code when you confuse state and transitions.
In a state/transition model states are the node of an oriented graph .
When you reach a state you have a well defined condition (ex connexion established).
If you have an oriented graph with N nodes, and the possibility to boucle up on yourself, than you may compute the number of available paths, (edges, transitions, ...) according to this it is [ N possibilties X N possibilieties ].
So I may have a little over stated the number of transitions. :/ (what an idiot)
* 7 states <=> 49 possible transitions, * 8 states <=> 64 possible transitions.
Possibilities are increasing in a more than polynomial way, with a brain that can remember at most 7 items in memory.
A good example of a state transition diagram is in section 7.2.2 http://www.ietf.org/rfc/rfc3720.txt
Soz, for miscalculating.
So I may have a little over stated the number of transitions. :/ (what an idiot)
* 7 states <=> 49 possible transitions, * 8 states <=> 64 possible transitions.
Possibilities are increasing in a more than polynomial way, with a brain that can remember at most 7 items in memory.
A good example of a state transition diagram is in section 7.2.2 http://www.ietf.org/rfc/rfc3720.txt
Soz, for miscalculating.
"I was thinking about switching to fad software X" is generally not a good line of thought. Why do you want to switch? Node.js doesn't offer parallelism, so it is obviously not a good choice for your app.
And "threads" is two things. There's programming in a threaded style, and there's using native operating system threads. They don't have to go together. For example, if you use a green thread library, you end up with the exact same benefits and limitations as an event loop, but with simpler, easier to understand code. A really good thread library could then handle multiplexing green threads over OS threads to get you parallelism too (see haskell).
And "threads" is two things. There's programming in a threaded style, and there's using native operating system threads. They don't have to go together. For example, if you use a green thread library, you end up with the exact same benefits and limitations as an event loop, but with simpler, easier to understand code. A really good thread library could then handle multiplexing green threads over OS threads to get you parallelism too (see haskell).
WebWorkers are testament to the lack of mutual exclusivity between event loops and threads.
http://en.wikipedia.org/wiki/Web_worker
https://developer.mozilla.org/En/Using_web_workers
Once you start doing any heavy lifting or large amounts of parallelizable tasks you will find yourself drifting towards the threading model from within your event loop.
http://en.wikipedia.org/wiki/Web_worker
https://developer.mozilla.org/En/Using_web_workers
Once you start doing any heavy lifting or large amounts of parallelizable tasks you will find yourself drifting towards the threading model from within your event loop.
Unfortunately Node's implementation of web workers uses external processes and serializes all the data as JSON. This makes them considerably slower than Chrome's implementation of web workers, which use threads.
In general if you're doing something compute-bound, Node's single threaded event loop isn't going to help you.
Depending on your performance requirements you might take a look at LMAX and Disruptor. At the very least you could be writing Java instead of C++
In general if you're doing something compute-bound, Node's single threaded event loop isn't going to help you.
Depending on your performance requirements you might take a look at LMAX and Disruptor. At the very least you could be writing Java instead of C++
How about switching to Go and using goroutines?
EDIT: more details about the application - it is a trading app that communicates with multiple sources at high rates to gather information and send commands, but it also does quite alot of number crunching.