Sayonara, V8(groups.google.com)
groups.google.com
Sayonara, V8
https://groups.google.com/forum/#!topic/v8-users/MUq5WrC2kcE
89 comments
I appreciate his attitude about this - he doesn't come across as bitchy at all (at least not to me). He's essentially saying "you choose not to run the project a certain way, and that's fine, but therefore I can no longer afford to run mine at all, and that's the consequence".
Thank you, TallGuyShort :). i think i did come across as a bit bitchy, but i tried to stay on topic and removed lots of stuff i wrote simply out of primal frustration :/. i'm only human ;). i thought about it for a couple months before writing it, and spent a couple days rewriting it, to make sure i wasn't just sending flame bait.
A better headline than either the original or the reversion-to-page-title would be:
Maintainer of v8-juice/cvv8: "Sayonara, V8"
Maintainer of v8-juice/cvv8: "Sayonara, V8"
I didn't wanted to make sensational title that says nothing about content. I hate this on HN. Such attention hunting is an act without honor.
Article is about V8-juice author abandoning his library for reasons that I don't personally consider right or wrong. I wanted to share this because there is a slight chance that state of V8 documentation (which I've never seen or used myself) is a political decision on a higher level (too little outcome from too much effort) and some discussion around the topic may change that.
Article is about V8-juice author abandoning his library for reasons that I don't personally consider right or wrong. I wanted to share this because there is a slight chance that state of V8 documentation (which I've never seen or used myself) is a political decision on a higher level (too little outcome from too much effort) and some discussion around the topic may change that.
Wow, what a poor choice on the part of the mod who decided to change this headline.
This is a huge loss. cvv8 has made writing nodejs native addons for large APIs much easier than the standard v8 interface. The entire nodejs community will be pretty significantly impacted - the v8 changes break all native addons for node 0.12.
i'm happy to hand over admin access to the repo as soon as someone asks for it. i would really hate to see cvv8 go to waste (IMH (and biased) opinion it's the best binding/conversion lib for v8), but i can't deal with them pulling the rug out from under me again (this is not the first time they've done it, only the most invasive, and it IS the first time they've warned us in advance).
I would love to but also don't have the bandwidth.
By the way, thank you for all your work. It took me quite a while to wrap my head around the template craziness you use in cvv8 but it turns out being pretty awesome. We have a huge API surface we're using cvv8 on and so far it's working great.
IMHO probably the best approach that I haven't seen anyone but the chromium project use would be a bindings compiler that takes WebIDL and generates wrapper code. I started down that path before diving into cvv8, but didn't understand well enough what my compiler should output. May have to revisit that project at some point if we want to upgrade beyond node 0.10.
By the way, thank you for all your work. It took me quite a while to wrap my head around the template craziness you use in cvv8 but it turns out being pretty awesome. We have a huge API surface we're using cvv8 on and so far it's working great.
IMHO probably the best approach that I haven't seen anyone but the chromium project use would be a bindings compiler that takes WebIDL and generates wrapper code. I started down that path before diving into cvv8, but didn't understand well enough what my compiler should output. May have to revisit that project at some point if we want to upgrade beyond node 0.10.
Thank you for your kind words. It took me a good long while to get my mind wrapped around the templates well enough to implement it. It was a very incremental process, starting with the basic type conversions and culminating in the function conversions and XTo templates. It was a really, really fun ride.
Anyway... you "could" write a wrapper which generates cvv8 bindings. i did that long ago in a predecessor to generate about a hundred ncurses bindings. But... that assumes cvv8 is usable, which it is not with current v8 versions :`(.
Anyway... you "could" write a wrapper which generates cvv8 bindings. i did that long ago in a predecessor to generate about a hundred ncurses bindings. But... that assumes cvv8 is usable, which it is not with current v8 versions :`(.
I meant to generate a full binding instead of using cvv8. bea[0] does something similar, but is somewhat limited.
[0] https://github.com/codeboost/bea
[0] https://github.com/codeboost/bea
JSC has a very stable external API and performs competitively (for some reason it always beats V8 on DOM manipulation stuff -- I don't know why v8/chrome haven't been able to make more efficient bindings).
It doesn't build outside of the WebKit tree, though... (but there are some scripts which should in theory build only jsc; I think they want all the deps to WebKit, etc, however).
It doesn't build outside of the WebKit tree, though... (but there are some scripts which should in theory build only jsc; I think they want all the deps to WebKit, etc, however).
JSC should build cleanly on its own (e.g. no dependencies on anything on top of it.) It's possible some of the build scripts have developed a bizarro dependency on other portions of the repo, but we'd happily take patches to fix those issues.
As far as APIs, JSC has provided a completely API and ABI stable C API for 7-ish years, and there's an ObjC API built on top of that now. Unfortunately the nature of C APIs for interacting with language runtimes makes it somewhat clunky and hard to use perfectly - C++ makes this easier as you can make types with constructors and destructors to make a lot of the otherwise manual work (ref()/deref(), etc) entirely automatic. Unfortunately C++ makes it easy to make an API that has a stable interface, but still isn't ABI stable.
As far as APIs, JSC has provided a completely API and ABI stable C API for 7-ish years, and there's an ObjC API built on top of that now. Unfortunately the nature of C APIs for interacting with language runtimes makes it somewhat clunky and hard to use perfectly - C++ makes this easier as you can make types with constructors and destructors to make a lot of the otherwise manual work (ref()/deref(), etc) entirely automatic. Unfortunately C++ makes it easy to make an API that has a stable interface, but still isn't ABI stable.
Well I spent around 5 minutes downloading and trying to build, immediately hitting this wall on Windows:
http://i.imgur.com/J9Gs6wb.png
Seems that nobody has tried that in a long time, or who knows why it references something in C:\tools
Seems that nobody has tried that in a long time, or who knows why it references something in C:\tools
People build jsc on its own on mac, but I have never been successful on linux. Hopefully there is some way though? I've been forced to build all of WebKit-gtk.
A bigger concern for me personally is that the jsc shell lacks typed arrays, https://bugs.webkit.org/show_bug.cgi?id=119828
edit: i should have said 'full spec for typed arrays', see the linked issue
A bigger concern for me personally is that the jsc shell lacks typed arrays, https://bugs.webkit.org/show_bug.cgi?id=119828
edit: i should have said 'full spec for typed arrays', see the linked issue
The typed arrays should be available in trunk, haves you checked recent trunk?
I believe the "need all of webkit-gtk" was a pile of sadness in build-jsc + automake that may have been fixed recently. I'm sure Zan, Xan, or Michel would be happy to help if it's still borked. There are _some_ platform requirements that make a generic "linux" not work, as we build on the platform support libraries for threads, unicode, etc so JSC may have random looking dependencies on such
I believe the "need all of webkit-gtk" was a pile of sadness in build-jsc + automake that may have been fixed recently. I'm sure Zan, Xan, or Michel would be happy to help if it's still borked. There are _some_ platform requirements that make a generic "linux" not work, as we build on the platform support libraries for threads, unicode, etc so JSC may have random looking dependencies on such
Sorry if i wasn't clear, I tried trunk when I filed that issue, last week - typed arrays are there, but they are missing things like set() and subarray() (mentioned in the issue).
I am not even sure how to try to build jsc by itself on linux, i couldn't find any instructions (online or in the dir)? All online discussions i found said i need to build all of webkit.
I am not even sure how to try to build jsc by itself on linux, i couldn't find any instructions (online or in the dir)? All online discussions i found said i need to build all of webkit.
Have you tried ToT trunk? We moved the entire TA impl into JSC itself last week and just accepted the semantic changes necessary for non-WebIDL types.
Tools/Scripts/build-jsc _should_ build just jsc.
Tools/Scripts/build-jsc _should_ build just jsc.
node.js was facing the same issue. In it's current state, node itself works fine but most native modules are broken with the latest v8. The main incompats are: Function callbacks are now a different type than they used to be before. The isolate parameter is required everywhere resulting in large scale deprecation of public api.
I totally thought the headline was referring to Campbell discontinuing their disgusting beverage. Sadly this news is much sadder.
The SWIG effort is struggling with the changing API too.
https://github.com/oliver----/swig-v8/pull/9
https://github.com/oliver----/swig-v8/pull/9
[deleted]
The "Why I quit" blog post serves as a kind of alternative exit interview process for the disgruntled and the outrage junkies. There are some tidbits of useful information, and the discussion here has some good information, but the Goodbye X theme lacks a bit of thoughtfulness in its presentation. Personally I'd avoid such a tacky thing. Google didn't disappear nor has seemed to have changed despite all the "Why I quit Google" posts and I'm sure v8 will be just the same tomorrow. There's a bit of narcissism. The authors attribute a bit of too much self-importance, and yet the thing they left carries on just fine in their absence. Given that these posts actually do contribute useful information, changing the tone to "what I learned at X" from "why I quit X" may see more uptake and less squabbling in subsequent discussions. You'll also avoid burning a bridge.
Edit: Every one of the downvotes this post gets are because this post runs counter to the HN echo chamber. I challenge someone to present an argument to explain otherwise. You've soaked yourself in outrage, and hell if anyone will say anything that may disagree with all that anger boiling up inside you. So you see some green nick and click downvote.
Edit: Every one of the downvotes this post gets are because this post runs counter to the HN echo chamber. I challenge someone to present an argument to explain otherwise. You've soaked yourself in outrage, and hell if anyone will say anything that may disagree with all that anger boiling up inside you. So you see some green nick and click downvote.
Again, Stephan has created and maintained multiple (46 in fact, [1]) treatments on what he has learned about v8. He even separated and a special wiki category, called "BitchAndMoan" [2], just so people wouldn't be confused about what were personal critiques and what were technical critiques. This isn't an "HN echo chamber" or a "woe is me circle jerk". This is a critique of Google's technical handling, feeding and nurturing of a (non-)product. I've embedded JavaScript for almost 15 years, since Mozilla's SpiderMonkey was ported to DOS/DJGPP[3][4]. There's dozens and dozens of successful embedders of various size and purpose [5][6] supported by Mozilla via SpiderMonkey and/or Rhino. While in college, I added LAM/MPI hooks and developed multi-monitor applications years before multi-monitor graphics cards were commercially available. Stephan is a former SpiderMonkey embedder[7]. I'm former colleagues with Shanti Rao of JSDB (SpiderMonkey) while at NASA/JPL [8]. There is a huge difference in support in what Mozilla provides [9] vs. what Google provides [10]. Stephan's critique is, without a doubt, a fair critique.
[1]http://code.google.com/p/v8-juice/w/list [2]http://code.google.com/p/v8-juice/wiki/CommentaryOnV8 [3]http://marc.info/?l=freedos-dev&m=94670703232745 [4]http://web.archive.org/web/*/http://dominic.liquid3.com.au/j... [5]https://developer.mozilla.org/en-US/docs/Web/JavaScript [6]http://en.wikipedia.org/wiki/JavaScript#Uses_outside_web_pag... [7]http://spiderape.sourceforge.net/ [8]http://www.jsdb.org/ [9]https://developer.mozilla.org/en-US/docs/SpiderMonkey [10]http://code.google.com/p/v8/w/list
[1]http://code.google.com/p/v8-juice/w/list [2]http://code.google.com/p/v8-juice/wiki/CommentaryOnV8 [3]http://marc.info/?l=freedos-dev&m=94670703232745 [4]http://web.archive.org/web/*/http://dominic.liquid3.com.au/j... [5]https://developer.mozilla.org/en-US/docs/Web/JavaScript [6]http://en.wikipedia.org/wiki/JavaScript#Uses_outside_web_pag... [7]http://spiderape.sourceforge.net/ [8]http://www.jsdb.org/ [9]https://developer.mozilla.org/en-US/docs/SpiderMonkey [10]http://code.google.com/p/v8/w/list
If i ever commission a biography, i'm coming to you to write it for me ;).
"Every one of the downvotes this post gets are because this post runs counter to the HN echo chamber."
It seems ironic to me that you make this statement (which would you could not possibly know to be true without being omiscient), and then add:
"... hell if anyone will say anything that may disagree with all that anger boiling up inside you."
While there are no doubt many ex-Google (and ex-other-high-profile-company) employees who put out poorly-worded and sometimes poorly-defended rants which in the end serve little purpose other than to vent their own frustration, I do not think you can lump Stephan's post in with them.
I have followed Stephan's work for many years. He is one of those rare developers who actually enjoys putting out high volumes of good documentation (which he readily admits on his own web sites). He appears to be passionately invested in every open source project he contributes to, and is quick to invest large quantities of his own time in helping others to use the code he has made generously available for others to use (much of it in the public domain, btw). I do not always agree with everything he says, but I give him great credit for his generousity.
While he may like to see himself given some credit for his contributions (who doesn't), he doesn't much struck me as the narcissistic type. Overall the interactions I've had with him (and the ones I've read online) have been quite civil. Occasionally, such as in this post, his passion for what he believes comes to the fore, and I suppose that might turn some off. However, I don't consider this post to be too far over-the-top, and it is obvious that he is trying to express a great deal of frustration while attempting to remain as civil as possbile. I think he at least mostly achieved that goal.
It is a difficult thing to abandon a project you have invested a lot of time in for over four years, especially when you feel the project could have easily been so much more if circumstances beyond your control would have been different.
Also, I don't think posts such as Stephan's ever need to have an effect on the offending party's behavior (in this case, Google and/or the V8 developer). There are many developers who have an interest in using and/or embedding V8 in their software, and having these insights into possible problems with doing so is worthwhile information by itself. The fact that Node.js uses V8 makes this information valuable in my opinion.
IMHO you owe it to yourself to take a look at Stephan's work (both inside and outside the v8-juice project), and perhaps form a different opinion.
It seems ironic to me that you make this statement (which would you could not possibly know to be true without being omiscient), and then add:
"... hell if anyone will say anything that may disagree with all that anger boiling up inside you."
While there are no doubt many ex-Google (and ex-other-high-profile-company) employees who put out poorly-worded and sometimes poorly-defended rants which in the end serve little purpose other than to vent their own frustration, I do not think you can lump Stephan's post in with them.
I have followed Stephan's work for many years. He is one of those rare developers who actually enjoys putting out high volumes of good documentation (which he readily admits on his own web sites). He appears to be passionately invested in every open source project he contributes to, and is quick to invest large quantities of his own time in helping others to use the code he has made generously available for others to use (much of it in the public domain, btw). I do not always agree with everything he says, but I give him great credit for his generousity.
While he may like to see himself given some credit for his contributions (who doesn't), he doesn't much struck me as the narcissistic type. Overall the interactions I've had with him (and the ones I've read online) have been quite civil. Occasionally, such as in this post, his passion for what he believes comes to the fore, and I suppose that might turn some off. However, I don't consider this post to be too far over-the-top, and it is obvious that he is trying to express a great deal of frustration while attempting to remain as civil as possbile. I think he at least mostly achieved that goal.
It is a difficult thing to abandon a project you have invested a lot of time in for over four years, especially when you feel the project could have easily been so much more if circumstances beyond your control would have been different.
Also, I don't think posts such as Stephan's ever need to have an effect on the offending party's behavior (in this case, Google and/or the V8 developer). There are many developers who have an interest in using and/or embedding V8 in their software, and having these insights into possible problems with doing so is worthwhile information by itself. The fact that Node.js uses V8 makes this information valuable in my opinion.
IMHO you owe it to yourself to take a look at Stephan's work (both inside and outside the v8-juice project), and perhaps form a different opinion.
A very heartfelt thank you for your depiction and endorsement. :)
PS: i don't always agree with everything i say, either ;)
PS: i don't always agree with everything i say, either ;)
hehe. You've got that problem too, eh?
Oh, and you're very welcome.
BTW, the post should read "he doesn't much strike me as ...", "not he doesn't much struck me as ..." -- one of the problems of editing a post too many times. I can't figure out how to edit the post, so I guess I'll just leave it for humor's sake. :-D
Oh, and you're very welcome.
BTW, the post should read "he doesn't much strike me as ...", "not he doesn't much struck me as ..." -- one of the problems of editing a post too many times. I can't figure out how to edit the post, so I guess I'll just leave it for humor's sake. :-D
No worries - i didn't notice the typo. It seems that comment editing expires after a short time, so you've gotta be fast. i tried it, too ;).
i would say "hubris" rather than "narcissism." My goal was not to change v8 - my goal was to explain to the cvv8/v8-juice users why i cannot afford to expend more energy supporting v8 by making their APIs more accessible (which is, in effect, what cvv8 is for). That goal, it seems, has succeeded far more than i dared hope, as is evidenced by this thread. Regarding the bridge burning: i did that on purpose. At certain points i find that i must shut down all paths which might lead me back to something which i know in my heart i don't want to go back to. It keeps me from "being weak" and trying to crawl back in the future.
PS: @leokun: FWIW, i up-voted you. Your suggestion about the change of tone from frustration to information is something i will definitely keep in mind for the next time (no, there's no sarcasm there!). Unfortunately, i didn't have that insight in advance this time (which would have made it foresight, i guess, which you have now armed me with :-).
I've had several interactions with Stephan. He's offered several times (dozens maybe) to do contract work (or otherwise) to create, update and maintain the public interfaces documentation and any supporting documents during the least 3 years. Google's stance (via its devs) has always been that v8's sole purpose (in their view) is as the JS VM for Chrome. It is not a product like GWT. Promotion of its capabilities is as of a consequence of v8 being connected to Chrome. For the most part, Stephan's offers have been completely ignored with a couple devs occasionally accusing him of grandstanding. I do feel that, however, for whatever reason, Google has demonstrated a personal affinity for certain embedders (node.js specifically, CEF maybe) vs. the lesser know embedders (v8-juice, TeaJS, SilkJS, NarwhalJS). I don't agree that Stephan has assigned himself too much self-importance. In fact, Stephan devoted significant time to identifying corner cases in the public interfaces that are beneficial to all embedders and has always chosen to make those learning experiences public, either via the GoogleCode Wiki or on the various v8 lists.
Wow... i didn't mean to start waves, which was why i unsubbed from the list within moments :/. i _really_ appreciate all of your support AND critiques (they are not unfair). Now i can't sleep. On a tablet, so won't say much more...
Happy Hacking!
Happy Hacking!
[deleted]
Does node.js have the same problem with the big API changes in v8?
Yeah, we're feeling this in nodeland too. The core team has actually done a great job making sure that the core of node works with the new V8 API; this is already working in the unstable branch, without too much fuss.
The much, much bigger issue is the native modules in npm. They all basically have to be ported by hand, and only a handful actually have been. So, transitioning to the next version of node will be an exercise in masochism for us makers.
On the other hand, improving V8 with breaking changes is the V8 team's perogative and I trust they're doing it for the right reasons.
The much, much bigger issue is the native modules in npm. They all basically have to be ported by hand, and only a handful actually have been. So, transitioning to the next version of node will be an exercise in masochism for us makers.
On the other hand, improving V8 with breaking changes is the V8 team's perogative and I trust they're doing it for the right reasons.
I'm interested to see how Go progresses over time, as they consciously made the design decision to keep the language easily parseable so that API breaking changes can be accounted for programatically rather that requiring manual porting. See gofix -- http://blog.golang.org/introducing-gofix
In theory, this should allow them greater flexibilty to iterate and refine the language over time without the negative backlash that we're seeing here.
In theory, this should allow them greater flexibilty to iterate and refine the language over time without the negative backlash that we're seeing here.
The API in question here is the C++ embedder API. It's altogether a different issue to Google changing some Go library APIs.
So are you saying there's nothing that the V8 devs could have done to ease the transition?
To their credit, they warned us some months ago (April?) on the list that the InvocationCallback signature and Persistent would change. That was a new step for them - the first time i was warned about breakage before it happened. But without concrete change details and a reference build to work against, one can't even prepare for the change until v8 has released something. A flood and backlog of client breakage was inevitable the moment they decided to make the change.
My instinct is that node probably doesn't interface as tightly with v8, or have as much of a surface area on those interfaces, as something like v8-juice.
Given Stephan's claim that the new version of v8 is incompatible with all applications, though, I suspect some energy will be expended to make node compatible. Node at least has a huge pool of users and developers to draw from when tackling a change like this.
Given Stephan's claim that the new version of v8 is incompatible with all applications, though, I suspect some energy will be expended to make node compatible. Node at least has a huge pool of users and developers to draw from when tackling a change like this.
The claim that it breaks "all v8 applications" is based on the fact that the v8::InvocationCallback signature (which is used by all client applications who actually embed anything) is now different. It can't NOT be incompatible.
The porting for Node was entirely done by one person.
The bad news is it will also break all C++ extensions, which frankly I'm not looking forward to (both as a developer of said extensions, and a user).
The bad news is it will also break all C++ extensions, which frankly I'm not looking forward to (both as a developer of said extensions, and a user).
The V8 team's anti-documentation culture is kind of problematic. As a compiler author targeting JS, it's been consistently difficult to figure out what kind of patterns V8 optimizes for, what patterns cause problems, and what the hell is going on inside their runtime. The rapid pace of changes to V8 - both new feature additions and system overhauls - means that if you're maintaining a complex project that interacts with V8, you're going to have a really hard time.
They barely document their engine internals or debugging tools, things change randomly making the existing documentation incorrect, and they frequently break the user-facing development tools like Chrome's JS debugger.
The only good things I can really say are that when V8 works, it works marvelously, and that if you can somehow track down a V8 developer they're usually incredibly helpful. Regardless, having to head over to the Google campus and buy a dev coffee isn't a scalable solution to these kinds of problems.
I've basically given up on delivering good performance in V8 as a result. The cost is too high. If I see obvious bottlenecks in their profiler (when it works at all), I tune them, but that's about it. I wasted a handful of weeks digging around in debug builds looking at bailout log entries and trying to decipher their IR using the broken log viewer they recommend, and the net result was that my code ended up faster for a few weeks until V8 changed again. Not worth it.
P.S. When I say 'anti-documentation culture', I mean it. V8 devs have personally told me they dislike documenting certain aspects of V8 and its internals because they fear that end-users and developers will rely on those internals. They lean towards occasional blog posts and Google IO presentation videos to pick up the slack.
They barely document their engine internals or debugging tools, things change randomly making the existing documentation incorrect, and they frequently break the user-facing development tools like Chrome's JS debugger.
The only good things I can really say are that when V8 works, it works marvelously, and that if you can somehow track down a V8 developer they're usually incredibly helpful. Regardless, having to head over to the Google campus and buy a dev coffee isn't a scalable solution to these kinds of problems.
I've basically given up on delivering good performance in V8 as a result. The cost is too high. If I see obvious bottlenecks in their profiler (when it works at all), I tune them, but that's about it. I wasted a handful of weeks digging around in debug builds looking at bailout log entries and trying to decipher their IR using the broken log viewer they recommend, and the net result was that my code ended up faster for a few weeks until V8 changed again. Not worth it.
P.S. When I say 'anti-documentation culture', I mean it. V8 devs have personally told me they dislike documenting certain aspects of V8 and its internals because they fear that end-users and developers will rely on those internals. They lean towards occasional blog posts and Google IO presentation videos to pick up the slack.
[disclaimer: I worked on V8]
I must say that public facing API is well documented right in the header files. Each class and majority of important functions have long comments[1]. So claiming that V8 has an anti-documenation culture is incorrect. Next I would like to note that V8 is surely open to patches that improve documentation of public V8 interfaces. I don't remember a lot of those patches flying in though.
Finally, documenting internals is completely different from documenting API. I don't think it is feasible to maintain such documentation. Internals exist only to rely on them in an extreme cases and in such extreme cases reaching to V8 developers directly or reading the source code is the only way to get correct and what is more important up to date information given how things are constantly in flux.
[1] https://code.google.com/p/v8/source/browse/trunk/include/v8....
I must say that public facing API is well documented right in the header files. Each class and majority of important functions have long comments[1]. So claiming that V8 has an anti-documenation culture is incorrect. Next I would like to note that V8 is surely open to patches that improve documentation of public V8 interfaces. I don't remember a lot of those patches flying in though.
Finally, documenting internals is completely different from documenting API. I don't think it is feasible to maintain such documentation. Internals exist only to rely on them in an extreme cases and in such extreme cases reaching to V8 developers directly or reading the source code is the only way to get correct and what is more important up to date information given how things are constantly in flux.
[1] https://code.google.com/p/v8/source/browse/trunk/include/v8....
I apologize if you got the impression I was claiming you don't document your public-facing APIs. I'm talking about V8 as a system; intended behaviors and recommended usage patterns and strategies, documentation of downsides and weaknesses and such. The debug-related parts of the 'public' (? not sure if they're actually public) API were very poorly documented when last I checked, though - some of the debug flags you can pass to d8 were only documented in old obscure blog posts.
It is true that debugging[1] options are not well documented, however it is also true that they are intended for use only in exceptional cases or by V8 developers. Many of them are not even available in release build of V8 that is shipping with Chrome (like disassembler or ability to trace IC transitions, which is a shame).
I would say that I am a bit torn on this issue.
First of all, I would love anybody using V8 to be able to troubleshoot issues they are running to as easily as possible. Ideally you should see your source annotated with optimization suggestions and internal V8 information translated into digestible form. I experimented with some ways of doing that but sadly I totally lack time to make a product out of that.
However I am not sure making some static cookbook of performance patterns is the right way to go here. It certainly can help in the short run, but in the long run its the uniformity of optimizations on V8 side and general shape of your code that matters. I fear cargo cult blindly following such a cook book without making an attempt to understand.
Additionally I fear that it is extremely hard to make and maintain such a cook book given that VMs consist of extremely many moving components and sometimes you can't decompose performance of an application into performance of small patterns, they interfere with each other. Understanding of internals, or understanding of how and where get the information on internals is much more important.
Maybe I am wrong, but I am doing my best to actually help people to become interested in V8 internals instead of following blindly recommendations.
[1] note: we are not talking about JavaScript debugger itself which should be relatively well documented as far as I understand, C++ header files look like they contain a lot of comments and I know there is a protocol description on the wiki.
I would say that I am a bit torn on this issue.
First of all, I would love anybody using V8 to be able to troubleshoot issues they are running to as easily as possible. Ideally you should see your source annotated with optimization suggestions and internal V8 information translated into digestible form. I experimented with some ways of doing that but sadly I totally lack time to make a product out of that.
However I am not sure making some static cookbook of performance patterns is the right way to go here. It certainly can help in the short run, but in the long run its the uniformity of optimizations on V8 side and general shape of your code that matters. I fear cargo cult blindly following such a cook book without making an attempt to understand.
Additionally I fear that it is extremely hard to make and maintain such a cook book given that VMs consist of extremely many moving components and sometimes you can't decompose performance of an application into performance of small patterns, they interfere with each other. Understanding of internals, or understanding of how and where get the information on internals is much more important.
Maybe I am wrong, but I am doing my best to actually help people to become interested in V8 internals instead of following blindly recommendations.
[1] note: we are not talking about JavaScript debugger itself which should be relatively well documented as far as I understand, C++ header files look like they contain a lot of comments and I know there is a protocol description on the wiki.
I definitely agree with you on the reasons why concrete performance rules/documentation is dangerous - I historically have tried to advocate for better introspection/debugging tools instead of just lists/documents. But so far the reaction to those has been pretty cold; it seems as if most engine devs I speak to view introspection tools/APIs as equivalent to documenting (and thus locking in) internals and runtime design details. I can understand how shipping a product like this as a part of a large software org is pretty hard - it's difficult to establish how many customers it has and what the benefits will be.
Not really sure how we get out of the spot we're in, unfortunately - in practice JS optimization is a mix of black magic and cargo cult wisdom that can become outdated as engines improve, and I can see that causing a lot of long term drag on productivity as engine authors have to keep tuning their engines for bad code.
I hope you keep getting traction on exposing developers to V8 internals; the stuff I've learned about both V8 and spidermonkey internals has been very useful so far - my complaint is largely that the learning process is difficult, not that there's anything wrong with your codebase :) I love that so many of the v8 JS builtins are self-hosted in JS instead of C++ - makes it much easier to peek under the hood and figure out what they're doing!
Not really sure how we get out of the spot we're in, unfortunately - in practice JS optimization is a mix of black magic and cargo cult wisdom that can become outdated as engines improve, and I can see that causing a lot of long term drag on productivity as engine authors have to keep tuning their engines for bad code.
I hope you keep getting traction on exposing developers to V8 internals; the stuff I've learned about both V8 and spidermonkey internals has been very useful so far - my complaint is largely that the learning process is difficult, not that there's anything wrong with your codebase :) I love that so many of the v8 JS builtins are self-hosted in JS instead of C++ - makes it much easier to peek under the hood and figure out what they're doing!
The problem with documenting internals is that it becomes a contract. Developers then start relying on those documented behaviors, and it later becomes much more difficult to change them.
It's bad enough as is with developers relying on undocumented behavior, see the troubles Wine (and Microsoft as well) with applications that touch Windows internals.
It's bad enough as is with developers relying on undocumented behavior, see the troubles Wine (and Microsoft as well) with applications that touch Windows internals.
The problem with NOT documenting PUBLIC APIs is that all behaviour becomes, from the client perspective, undefined.
People keep going back to the point of private API docs - private details are private for a reason. Nobody but the v8 devs need private internal docs, and those MIGHT exist in encyclopedic form somewhere in Google. Those don't interest me. Public APIs are missing (or exceedingly poor, relative to v8's scope).
People keep going back to the point of private API docs - private details are private for a reason. Nobody but the v8 devs need private internal docs, and those MIGHT exist in encyclopedic form somewhere in Google. Those don't interest me. Public APIs are missing (or exceedingly poor, relative to v8's scope).
Isn't it the right thing to do though? Open source should not be about hiding behaviors behind undocumented code. Open source is about documenting it so that others can learn and jump in and help. It seems totally non-open if this is how V8 is developed.
There is a distinction between documenting the portions of your app you intend people use, and effectively promising to maintain side effects and accidents of your existing design. mraleph says the public API is documented, which fulfills the obligation an open source project has.
However, the discussion here is about side effects of the V8 implementation, as they relate to performance. That is to say, the V8 team is reluctant to do anything that suggests "if you write javascript using this pattern, your code will/will not run fast, but only because that's just how this week's code branch works out". If the team made those sorts of promises with a project as complex as V8, they effectively become locked in to their existing code, unable to change or improve much of anything because they'd be breaking the promises of performance patterns they previously issued.
The authors who are working with V8, on the other hand, require those promises to be made. It's not enough for the performance patterns to be documented, these authors need them to be fairly stable as well.
It's a tough situation - the V8 authors have made all the promises and documentation that's practical for them to do, but it's not enough to satisfy these other project authors' needs.
However, the discussion here is about side effects of the V8 implementation, as they relate to performance. That is to say, the V8 team is reluctant to do anything that suggests "if you write javascript using this pattern, your code will/will not run fast, but only because that's just how this week's code branch works out". If the team made those sorts of promises with a project as complex as V8, they effectively become locked in to their existing code, unable to change or improve much of anything because they'd be breaking the promises of performance patterns they previously issued.
The authors who are working with V8, on the other hand, require those promises to be made. It's not enough for the performance patterns to be documented, these authors need them to be fairly stable as well.
It's a tough situation - the V8 authors have made all the promises and documentation that's practical for them to do, but it's not enough to satisfy these other project authors' needs.
you confuse open source with extensible/flexibility/maintainability.
Its open source if you have access to the source, not because you can grok the code.
The internals of V8 ought to be undocumented, if it not intended to be an api that other systems interfaces with. Api behaviour should be documented, and i assume that's what the V8 team does - but the internal IR, for example, is private and its behaviour should not be relied up on, or do so at your own risk.
Its open source if you have access to the source, not because you can grok the code.
The internals of V8 ought to be undocumented, if it not intended to be an api that other systems interfaces with. Api behaviour should be documented, and i assume that's what the V8 team does - but the internal IR, for example, is private and its behaviour should not be relied up on, or do so at your own risk.
This project is 28 days old:
http://fossil.wanderinghorse.net/repos/f2/doxygen/
it is experimental, alpha quality, and developed only by me. It has more public API docs than v8. i cannot for the life of me comprehend how, given the vast amount of Hacker Skill in the v8 team, so little documentation (often of useless quality) comes out of it. i don't have the headers with me now but the last time i looked at the docs for Context::EnterContext(), they said, "enters the context." No joke.
Not my problem, anymore.
http://fossil.wanderinghorse.net/repos/f2/doxygen/
it is experimental, alpha quality, and developed only by me. It has more public API docs than v8. i cannot for the life of me comprehend how, given the vast amount of Hacker Skill in the v8 team, so little documentation (often of useless quality) comes out of it. i don't have the headers with me now but the last time i looked at the docs for Context::EnterContext(), they said, "enters the context." No joke.
Not my problem, anymore.
> V8 devs have personally told me they dislike documenting certain aspects of V8 and its internals because they fear that end-users and developers will rely on those internals.
This doesn't strike me as too egregious, personally. I can understand library maintainers taking any steps necessary to prevent people from relying on fluid internals. If anything, this suggests (to me) that the interface might not be rich enough to accomplish what people want. But it isn't a sin to keep documentation of internals private.
This doesn't strike me as too egregious, personally. I can understand library maintainers taking any steps necessary to prevent people from relying on fluid internals. If anything, this suggests (to me) that the interface might not be rich enough to accomplish what people want. But it isn't a sin to keep documentation of internals private.
By itself there's nothing wrong with it, it just contributes (along with the other problems) to it being very difficult to interact with V8 as a whole.
I would say in an opensource project like V8 it is a sin. You don't build a community around V8 by doing this. You build a company owned project (of which the company is throwing code over a wall).
"V8 devs have personally told me they dislike documenting certain aspects of V8 and its internals because they fear that end-users and developers will rely on those internals."
Shamefully in this instance the problem is changes to the public embedders API... so we shouldn't rely on it? Why did they create the API?
Shamefully in this instance the problem is changes to the public embedders API... so we shouldn't rely on it? Why did they create the API?
Sometimes later, an API design is found to conflict with design changes that need to be done, that wasn't known at API design time. Sometimes you have to deprecate.
I work on GWT. GWT has, since 2008 at least, relied on certain low level details of the NPAPI in browsers for implementing its Development Mode. Both Chrome and Firefox break this API consistently, in fact, Firefox breaks it on every single release. Eventually NPAPI will go away, because it was designed in a bygone era of Flash plugins.
Unfortunately, it's not being replaced with anything that solves our use case, nor does PPAPI solve it.
I really feel for the author of v8-juice, but it's more important that the browser vendors drive performance and security forward, and given how tightly integrated JS is with the browser, you would have to expect that radical design changes in internals are eventually going to break low level APIs, even if they are public.
I work on GWT. GWT has, since 2008 at least, relied on certain low level details of the NPAPI in browsers for implementing its Development Mode. Both Chrome and Firefox break this API consistently, in fact, Firefox breaks it on every single release. Eventually NPAPI will go away, because it was designed in a bygone era of Flash plugins.
Unfortunately, it's not being replaced with anything that solves our use case, nor does PPAPI solve it.
I really feel for the author of v8-juice, but it's more important that the browser vendors drive performance and security forward, and given how tightly integrated JS is with the browser, you would have to expect that radical design changes in internals are eventually going to break low level APIs, even if they are public.
FWIW, i agree completely that v8 has to serve its primary target, and there was never any illusion that that wasn't Chrome. i'm not mad about them for that. i'm just frustrated at giving up something i put literally thousands of hours into creating, tweaking, and documenting. :/ It is my sincere hope that someone will pick it up and port it.
I'm curious -- why doesn't Super Dev Mode meet your needs?
Super Dev Mode doesn't work with GWT RPC. Plus as neat a technology as source maps is, and as slick as the browser dev environments are getting, for non-js languages they still aren't equivalent to a full fledged IDE like eclipse.
> Regardless, having to head over to the Google campus and buy a dev coffee
I thought coffee was free over there ;)
I thought coffee was free over there ;)
>I wasted a handful of weeks digging around in debug builds looking at bailout log entries and trying to decipher their IR using the broken log viewer they recommend, and the net result was that my code ended up faster for a few weeks until V8 changed again.
You have the same problem with any other modern JavaScript VM. There is always lots of magic and some heuristics which can easily break. As a result, performance is hard to predict.
For what it's worth, Dart fixed that. Predictable performance was one of its main goals/constrains.
You have the same problem with any other modern JavaScript VM. There is always lots of magic and some heuristics which can easily break. As a result, performance is hard to predict.
For what it's worth, Dart fixed that. Predictable performance was one of its main goals/constrains.
Isn't asm.js moving towards standardising a dialect of Javascript that definitely will be optimised well by VMs? Seems like a possible solution to struggling to figure out what V8 does and doesn't optimise well.
Except that right now you are limited to arithmetic and typed array manipulation. Subset like that is predictable because it is taking away all moving dynamic parts. You become free from questions like "where should I put methods? on prototype or on object itself?" not because you have an answer now, but because you can't have objects, prototypes and methods :-) [well you can but you have to emulate them with typed arrays and virtual dispatch tables, it is likely be slower than "native" JS method dispatch].
If asm.js eventually evolves to support a large enough subset of applications, it does 'solve' this. But it only solves it for people like me who are writing compilers; people trying to write high-performance JS by hand are left out in the cold.
To be fair - I've never seen such docs for SpiderMonkey. I may have just failed at finding them though.
Keeping up-to date documentation of compiler is a horrible job. I think it's good enough that V8 may be built as standalone binary with really extraordinary debug options and tools.
At the same time, I'd love to see more talks like http://www.youtube.com/watch?v=FrufJFBSoQY There are also guys who write about V8 from time to time (http://mrale.ph/ http://floitsch.blogspot.com/ http://wingolog.org/tags/v8) but this is needle-in-a-haystack type of learning.
Keeping up-to date documentation of compiler is a horrible job. I think it's good enough that V8 may be built as standalone binary with really extraordinary debug options and tools.
At the same time, I'd love to see more talks like http://www.youtube.com/watch?v=FrufJFBSoQY There are also guys who write about V8 from time to time (http://mrale.ph/ http://floitsch.blogspot.com/ http://wingolog.org/tags/v8) but this is needle-in-a-haystack type of learning.
SpiderMonkey isn't a shining city on a hill in this case, certainly. In my experience though the docs that are there are easier to understand, the log messages are more verbose and understandable, and the developers are easier to get help from. I think this is all just a result of the fact that Mozilla relies more on community involvement to get stuff done, so they have to be a bit more dev-friendly.
The same dislike of documenting performance details and internals exists on the SM team, though.
Standalone binaries with debug options like d8 and spidermonkey js.exe are great for narrowly-scoped tuning and microbenchmarks, but in the real world your app runs in the browser and you end up having to debug and profile it there. That's where the whole picture falls apart (and where Spidermonkey's tools are dramatically better).
Mozilla ships two different performance analysis tools for in-browser JS that give you extremely detailed, precise performance information (JIT Inspector and SPS). They're more stable than Chrome's tools on average as well. You can literally get the browser to dump the native assembly it produced when JITting a given function.
I haven't done any perf tuning in a while, so this page is probably out of date, but I've made a point of documenting the stuff I discover about each engine, and linking to sources where possible: https://github.com/sq/JSIL/wiki/JavaScript-Performance-For-M...
The same dislike of documenting performance details and internals exists on the SM team, though.
Standalone binaries with debug options like d8 and spidermonkey js.exe are great for narrowly-scoped tuning and microbenchmarks, but in the real world your app runs in the browser and you end up having to debug and profile it there. That's where the whole picture falls apart (and where Spidermonkey's tools are dramatically better).
Mozilla ships two different performance analysis tools for in-browser JS that give you extremely detailed, precise performance information (JIT Inspector and SPS). They're more stable than Chrome's tools on average as well. You can literally get the browser to dump the native assembly it produced when JITting a given function.
I haven't done any perf tuning in a while, so this page is probably out of date, but I've made a point of documenting the stuff I discover about each engine, and linking to sources where possible: https://github.com/sq/JSIL/wiki/JavaScript-Performance-For-M...
> You can literally get the browser to dump the native assembly it produced when JITting a given function.
You can literally get Chrome to dump native assembly it produces when JITting a given function. You just need to build Chromium yourself, because disassembler is not built into it. It's a dead bloat I guess for 99% of people, even for JS VM engineer disassembly is the last resort. You can get compiler IR from any Chrome build and people do get it. And usually it means that VM itself should be fixed.
People should not need to care about IR or (worse) disassembly. If your code is reasonable it should just work reasonably fast. File bugs if it does not, instead of tuning, which can break as soon as V8 moves left or right.
You can literally get Chrome to dump native assembly it produces when JITting a given function. You just need to build Chromium yourself, because disassembler is not built into it. It's a dead bloat I guess for 99% of people, even for JS VM engineer disassembly is the last resort. You can get compiler IR from any Chrome build and people do get it. And usually it means that VM itself should be fixed.
People should not need to care about IR or (worse) disassembly. If your code is reasonable it should just work reasonably fast. File bugs if it does not, instead of tuning, which can break as soon as V8 moves left or right.
To be more precise, the Firefox addon that does this lets you select a specific function and view the assembly on demand, instead of getting a multi-gigabyte IR/assembly dump from a debug build of the browser. That's what makes it useful.
I agree that in general users should not need to care about IR or disassembly. Unfortunately, I'm not sure we'll ever end up in that world.
I agree that in general users should not need to care about IR or disassembly. Unfortunately, I'm not sure we'll ever end up in that world.
I see, yes just printing a simple function on demand is indeed quite useful.
IIRC V8 has Debug.disassemble(f), requires both a special build and mirror debugger exposed to the page. Prints disassembly to stdout. Can be tweaked to return disassembly as string, but that requires knowledge of V8. (I actually don't know who is the client of this function or why it was added).
[side note: if you get function disassembly you can't really be completely sure that that's the code that was running 2 seconds ago, that's one of the reasons why V8 just dumps everything]
IIRC V8 has Debug.disassemble(f), requires both a special build and mirror debugger exposed to the page. Prints disassembly to stdout. Can be tweaked to return disassembly as string, but that requires knowledge of V8. (I actually don't know who is the client of this function or why it was added).
[side note: if you get function disassembly you can't really be completely sure that that's the code that was running 2 seconds ago, that's one of the reasons why V8 just dumps everything]
Could you point me at this extension?
I was trying to get disassembly of asm.js compiled by IonMonkey a couple of months ago. I had to run SpiderMonkey under gdb and disassemble the JITted entry points. (Or maybe I didn't...)
I was trying to get disassembly of asm.js compiled by IonMonkey a couple of months ago. I had to run SpiderMonkey under gdb and disassemble the JITted entry points. (Or maybe I didn't...)
https://addons.mozilla.org/en-us/firefox/addon/jit-inspector...
I don't know if this dumps the disassembly for asm.js. I suspect it can't, since asm.js modules are AOT compiled into a single big block of code - but maybe it'll work. Give it a try!
I don't know if this dumps the disassembly for asm.js. I suspect it can't, since asm.js modules are AOT compiled into a single big block of code - but maybe it'll work. Give it a try!
> People should not need to care about IR or (worse) disassembly. If your code is reasonable it should just work reasonably fast. File bugs if it does not, instead of tuning, which can break as soon as V8 moves left or right.
The trouble with this is that the results are not in your own hands. Developers have bosses and deadlines and all that, and when it comes down to it, they just need to make it work. In that position, it's awfully hard to "submit a bug report and pray," and though it's not optimal, the ability to get into the guts of things can satisfy the immediate need.
The trouble with this is that the results are not in your own hands. Developers have bosses and deadlines and all that, and when it comes down to it, they just need to make it work. In that position, it's awfully hard to "submit a bug report and pray," and though it's not optimal, the ability to get into the guts of things can satisfy the immediate need.
> it's awfully hard to "submit a bug report and pray,"
You don't need to be praying. If no immediate fix can be made in the V8, you will get some recommendations and explanations based on the V8 team knowledge.
I would go as far as saying that bug reports like this are a single most reliable signal of what JS developers want to be fast on the web. By not reporting performance issues and silently working around them you hinder this signal. THere might be bugs and inconsistencies in underlying VM optimizations that can go unnoticed for a long time due to that.
You don't need to be praying. If no immediate fix can be made in the V8, you will get some recommendations and explanations based on the V8 team knowledge.
I would go as far as saying that bug reports like this are a single most reliable signal of what JS developers want to be fast on the web. By not reporting performance issues and silently working around them you hinder this signal. THere might be bugs and inconsistencies in underlying VM optimizations that can go unnoticed for a long time due to that.
What's the gist of these supposedly major changes to the V8 APIs?
In short it was easy to misuse global v8::Persistent<T>`handles leading to memory corruptions and such. Unfortunately making them safe to use also means making them hard to use directly. Previously they were just interchangeable with normal v8::Local<T> handles for things like calls.
https://groups.google.com/d/msg/v8-users/6kSAbnUb-rQ/e-GI0b1...
https://groups.google.com/d/msg/v8-users/6kSAbnUb-rQ/e-GI0b1...
This seems to be the biggest issue:
Changing all callbacks that return Handle<Value> to return void and instead pass the return value in as a parameter.
Changing all callbacks that return Handle<Value> to return void and instead pass the return value in as a parameter.
This is actually easy to work around by creating trampolines. Dan Carney creating a small header file with templates to create those trampolines easily:
https://github.com/drcarney66/v8-backport
Amount of typing one needs to do is minimal, I think.
https://github.com/drcarney66/v8-backport
Amount of typing one needs to do is minimal, I think.
That approach doesn't work (without a lot of effort) with templates at the level cvv8 uses them. There are literally thousands of InvocationCallback instances in the cvv8 tree, most of which are bound via the XTo templates, which run through 4 or 5 layers of metatemplate magic to do their things. It's a bit more complicated than a drop-in replacement can solve unless the drop-in replacement has the same signature AND type name as the old one (and even then it would require more effort than i'm willing to commit to).
Stephens libraries aim to abstract this API into something more palatable (simpler marshalling back and forth, function signatures, object mapping etc), so clearly breaking changes to this API can result in huge rewrites, and the most recent changes are daunting. Disclaimer: I never used his libraries but was well aware of them.