Node.js on mozilla spidermonkey(blog.zpao.com)
blog.zpao.com
Node.js on mozilla spidermonkey
http://blog.zpao.com/post/4620873765/about-that-hybrid-v8monkey-engine/
3 comments
Are people doing this just for fun or are there advantages in switching to Spidermonkey?
I don't know if this is their motivation for it, but SpiderMonkey implements many useful non-standard features, including generators and iterators.
JS 1.5: https://developer.mozilla.org/en/JavaScript/New_in_JavaScrip...
JS 1.6: https://developer.mozilla.org/en/JavaScript/New_in_JavaScrip...
JS 1.7: https://developer.mozilla.org/en/JavaScript/New_in_JavaScrip... <- generators, iterators, and more
JS 1.8: https://developer.mozilla.org/en/JavaScript/New_in_JavaScrip...
(As a side note, I once implemented Twisted's inlineCallbacks in JS 1.7; this worked because it had those Python-like generators. Too bad it doesn't work anywhere but SpiderMonkey.)
JS 1.5: https://developer.mozilla.org/en/JavaScript/New_in_JavaScrip...
JS 1.6: https://developer.mozilla.org/en/JavaScript/New_in_JavaScrip...
JS 1.7: https://developer.mozilla.org/en/JavaScript/New_in_JavaScrip... <- generators, iterators, and more
JS 1.8: https://developer.mozilla.org/en/JavaScript/New_in_JavaScrip...
(As a side note, I once implemented Twisted's inlineCallbacks in JS 1.7; this worked because it had those Python-like generators. Too bad it doesn't work anywhere but SpiderMonkey.)
Those features are a lot of the reason for it, yes. As a cool example of what can be done with generators, check out my colleague Dave Herman's task.js library: https://github.com/dherman/taskjs
That library gives you coroutine-style I/O on top of node.js primitives, which is, as far as I can tell, the #1 feature people have been asking for all along in Node. It only works if node.js implements coroutines (which Ryan has so far refused to do) or if the language implements generators (which you get with SpiderMonkey).
That library gives you coroutine-style I/O on top of node.js primitives, which is, as far as I can tell, the #1 feature people have been asking for all along in Node. It only works if node.js implements coroutines (which Ryan has so far refused to do) or if the language implements generators (which you get with SpiderMonkey).
V8 is a single-threaded VM and I don't see that changing any time soon, it's very much married to Chrome's per-process model. SpiderMonkey doesn't have that drawback, it supports threads just fine.
These guys have their work cut out for them if they want to support the V8 API from within SM. It's a well designed API but large and fast moving.
Still, an interesting project. I'm going to watch it, maybe submit a few patches.
These guys have their work cut out for them if they want to support the V8 API from within SM. It's a well designed API but large and fast moving.
Still, an interesting project. I'm going to watch it, maybe submit a few patches.
I had some problems with SpiderMonkey's thread safe mode. If you have your own lock system, and you lock a resource while running JS, SM might do garbage collection at any time, for which it has its own lock. This causes deadlock very quickly indeed.
I had to resort to having a pool of JS runtimes from which each thread would borrow a runtime when it wanted to execute code, and return it when it was finished. This got messy quickly when I found out that script objects can't be shared, and I needed to load an individual copy of each script for each runtime.
Furthermore, JS objects weren't able to be shared between threads even when I didn't have separate runtimes, and I needed to (this was actually the recommended approach) write them to a binary format using JS_WriteStructuredClone when one thread was finished with them, and reassemble them with JS_ReadStructuredClone on the other end.
In the end, I gave up trying to do multithreaded Javascript and went for a nice, single threaded event driven model. I don't regret it. Heck, since it's lock-free - and structured clone free - it's probably more scalable!
I had to resort to having a pool of JS runtimes from which each thread would borrow a runtime when it wanted to execute code, and return it when it was finished. This got messy quickly when I found out that script objects can't be shared, and I needed to load an individual copy of each script for each runtime.
Furthermore, JS objects weren't able to be shared between threads even when I didn't have separate runtimes, and I needed to (this was actually the recommended approach) write them to a binary format using JS_WriteStructuredClone when one thread was finished with them, and reassemble them with JS_ReadStructuredClone on the other end.
In the end, I gave up trying to do multithreaded Javascript and went for a nice, single threaded event driven model. I don't regret it. Heck, since it's lock-free - and structured clone free - it's probably more scalable!
I hear you, threading in SM is indeed fraught with dangers.
On the off chance that you're going to revisit it, the way to go is to compile with -DJS_THREADSAFE and register a callback with JS_SetGCCallback() that tells the VM when it's safe to run the garbage collector.
Actually, the approach I took was to always tell the VM 'no' and invoke JS_GC() manually from time to time. Conceptually easy and performance is mostly amortized.
PS: Feel free to contact me if you have follow-up questions, my email is in my profile.
On the off chance that you're going to revisit it, the way to go is to compile with -DJS_THREADSAFE and register a callback with JS_SetGCCallback() that tells the VM when it's safe to run the garbage collector.
Actually, the approach I took was to always tell the VM 'no' and invoke JS_GC() manually from time to time. Conceptually easy and performance is mostly amortized.
PS: Feel free to contact me if you have follow-up questions, my email is in my profile.
I think Mozilla is also making SpiderMonkey threading model more strict. https://developer.mozilla.org/en/JS_THREADSAFE says:
JS_THREADSAFE is now permanently on.
We have recently made major changes to this feature. Until recently, sharing objects among threads would mostly work, although scripts could easily make it crash. We have now completely removed that feature. Each thread that uses the JavaScript engine must essentially operate in a totally separate region of memory.
------
So both VMs provide essentially the same multithreading capabilities.
JS_THREADSAFE is now permanently on.
We have recently made major changes to this feature. Until recently, sharing objects among threads would mostly work, although scripts could easily make it crash. We have now completely removed that feature. Each thread that uses the JavaScript engine must essentially operate in a totally separate region of memory.
------
So both VMs provide essentially the same multithreading capabilities.
V8 supported multiple threads since long ago using a big VM lock approach (v8::Locker). Since 3.2.4 it's possible to run independent V8 code in parallel using the new v8::Isolates API: http://code.google.com/p/v8/source/browse/trunk/include/v8.h...
If you're interested in helping out, best thing to do is stop by the IRC channel and see what we need the most help with (or if you want to start more simple, we can point you at that too). We've been moving pretty quickly in the week since that went up.
I always wondered why Node.js is based on V8 in the first place; to my (limited) experience, SpiderMonkey's API is both simpler in use and better documented. Was it a speed thing in Node's early days?
Cool move, nevertheless. Sounds like a pain to maintain, though.
Cool move, nevertheless. Sounds like a pain to maintain, though.
> I always wondered why Node.js is based on V8 in the first place
Probably just a case of that engine being the one he was playing with at the time (or had some/more experience with) so it was the one his experiment that became node.js got implemented in.
A lot of implementation decisions get made this waym, especially with projects that start out as an experiment or a proof of concept for a thought experiment: if the list of reasons why not to use the tool you currently have in your hand are insignificant at the time, why mess around looking for other tools to try as well?
Sometimes this means a project gets tied to a particular tool even though another tool might offer some useful features, which is the case with the current node.js implementation. In the grand scheme of things it may not matter: is node.js suffering (or likely to suffer in future) because it is tied V8? Would making node.js flexible about its JS engine add enough overhead that it is undesirable? It is certainly worth experimenting with other JS engines though, so they have answers to those questions (maybe is isn't worth it, but now that is known for sure, and maybe it is worth it and the benefits of a change (or generalisation) can be demonstrated with real examples).
Probably just a case of that engine being the one he was playing with at the time (or had some/more experience with) so it was the one his experiment that became node.js got implemented in.
A lot of implementation decisions get made this waym, especially with projects that start out as an experiment or a proof of concept for a thought experiment: if the list of reasons why not to use the tool you currently have in your hand are insignificant at the time, why mess around looking for other tools to try as well?
Sometimes this means a project gets tied to a particular tool even though another tool might offer some useful features, which is the case with the current node.js implementation. In the grand scheme of things it may not matter: is node.js suffering (or likely to suffer in future) because it is tied V8? Would making node.js flexible about its JS engine add enough overhead that it is undesirable? It is certainly worth experimenting with other JS engines though, so they have answers to those questions (maybe is isn't worth it, but now that is known for sure, and maybe it is worth it and the benefits of a change (or generalisation) can be demonstrated with real examples).
I'm willing to bet speed was a factor, but the other thing Ryan noted was the API difference. The article on the Register about SpiderNode (http://www.theregister.co.uk/2011/04/20/mozilla_moves_node_d...) mentions that:
> Dahl originally set out to build Node atop Mozilla's SpiderMonkey – the JavaScript engine included with Firefox browser – but this effort didn't last long. After about two days, he switched to Google's V8. "V8 is just a nice, clean library," Dahl told us earlier this year. "It's compact and extracted away from Chrome. It's distributed as its own package, and it's easy to build and it's got a nice header file with nice documentation. It's kind of constrained. It doesn't have dependencies on other things. It seemed much more modern than the Mozilla stuff."
I mostly agree about the clean API. It's compartmentalized and makes more sense to me overall. Documentation though...
As for maintenance, I'm really hoping it won't be too bad. Once V8Monkey is standing, it shouldn't be terrible to keep up. And so long as Node doesn't start using new or modified APIs, then SpiderNode should be fine.
> Dahl originally set out to build Node atop Mozilla's SpiderMonkey – the JavaScript engine included with Firefox browser – but this effort didn't last long. After about two days, he switched to Google's V8. "V8 is just a nice, clean library," Dahl told us earlier this year. "It's compact and extracted away from Chrome. It's distributed as its own package, and it's easy to build and it's got a nice header file with nice documentation. It's kind of constrained. It doesn't have dependencies on other things. It seemed much more modern than the Mozilla stuff."
I mostly agree about the clean API. It's compartmentalized and makes more sense to me overall. Documentation though...
As for maintenance, I'm really hoping it won't be too bad. Once V8Monkey is standing, it shouldn't be terrible to keep up. And so long as Node doesn't start using new or modified APIs, then SpiderNode should be fine.
SpiderMonkey used to be very slow. Wikipedia says Node.js was released in 2010, I'm not sure when development started, but if it was when V8 was significantly faster, it seems like an obvious choice.
http://arewefastyet.com/old-awfy.php
http://arewefastyet.com/old-awfy.php
As far as I know they used V8 for speed, yes.
I was wondering a few months ago why there wasn’t a Spidermonkey implementation for Node; didn’t realise it was tied so tightly into V8. This should be good.