How We Built eBay’s First Node.js Application(ebaytechblog.com)
ebaytechblog.com
How We Built eBay’s First Node.js Application
http://www.ebaytechblog.com/2013/05/17/how-we-built-ebays-first-node-js-application/
27 comments
I'm not interested in getting into a debate on the definition of a framework, but Express is not one. It's a tiny bit more than a router.
The first four words on the Express website are, in large point font, "express web application framework for node".
That doesn't really change what MatthewPhillips said. "Framework" can mean everything from Rails to startup.sh that boots up your node process. Express really doesn't do much, and is closer to middleware than a full framework.
Not to stop there, it's in the page title too, so anyone coming in from Google (for example) is basically required to click on the phrase "web application framework" and acknowledge that the creators, at least, think that it's a framework.
I wonder if they looked at:
http://mcavage.github.io/node-restify/
Both self describe as frameworks.
in a quick blurb on express v restify: "I get asked this more than anything else, so I'll just get it out of the way up front.
Express' use case is targeted at browser applications and contains a lot of functionality, such as templating and rendering, to support that. Restify does not."
FWIW I am a backend guy and have enjoyed prototyping and working with restify a great deal ( dtrace ftw :) ), just a user and YMMV of course.
http://mcavage.github.io/node-restify/
Both self describe as frameworks.
in a quick blurb on express v restify: "I get asked this more than anything else, so I'll just get it out of the way up front.
Express' use case is targeted at browser applications and contains a lot of functionality, such as templating and rendering, to support that. Restify does not."
FWIW I am a backend guy and have enjoyed prototyping and working with restify a great deal ( dtrace ftw :) ), just a user and YMMV of course.
The quote (at least now) is: "we did not want to bloat the application by introducing a proprietary framework."
I think they may have meant that they didn't want to write their own.
I think they may have meant that they didn't want to write their own.
Nice! Ebay's running webwokers. Do you guys have any plans to start implementing app cache manifests to reduce load off the clusters? Even though ebay's loading time is fast as hell, I could still imagine this would reduce not only load for static content, but also decrease http sizes for end users.
Do you know when it was discovered that the earth is round?
I'm sorry, but how is this relevant?
I would really like to see this Wiki!
I really hate posts like this. It's full of vague statements like:
This is a fluff piece that won't inform you about anything besides the fact that eBay is using Node. Cool story? But you're not telling us why
"When we found that Java did not seem to fit the
project requirements (no offense), we began exploring
the world of Node.js"
"By the end of the exercise, people understood the core
value of Node.js; indeed, some of the con arguments proved
to be part of the beauty of the language."
This is a fluff piece that won't inform you about anything besides the fact that eBay is using Node. Cool story? But you're not telling us why
They talk about how under "Scalability" how "Java did not seem to fit the project requirements"... I'm sure a Java implementation can be less scalable than a node.js implementation, but every anecdotal piece of evidence I have experienced suggests that a good Java implementation will be faster than a good node.js implementation.
If we look to popular micro-benchmarks, they support my anecdotal observation:
http://benchmarksgame.alioth.debian.org/u64/benchmark.php?te...
http://www.techempower.com/benchmarks/
On a side node: node.js also suffers from the issue that while a small node.js program might be fast, what is actually fast is the C code and as the program grows larger and increasing amounts of time are spent executing actual javascript its performance characteristics change dramatically. This fact seems often lead to unrealistic understanding of node.js performance due to people benchmarking very small examples.
If we look to popular micro-benchmarks, they support my anecdotal observation:
http://benchmarksgame.alioth.debian.org/u64/benchmark.php?te...
http://www.techempower.com/benchmarks/
On a side node: node.js also suffers from the issue that while a small node.js program might be fast, what is actually fast is the C code and as the program grows larger and increasing amounts of time are spent executing actual javascript its performance characteristics change dramatically. This fact seems often lead to unrealistic understanding of node.js performance due to people benchmarking very small examples.
This is less an issue with "very small examples" and more an issue of people throwing the practices out when they build more complex software.
Highly performant JS ends up looking a lot like C, and most of the off-the-shelf JS libraries that people use to build software are littered with '.call', '.apply', 'arguments', null/undefined/true/false usage, implicit type conversions, unnecessary closures, etc. All of these are cases that V8 doesn't optimize well, and because the libraries are built to be highly abstract and useful by using these, performance drops off precipitously. It's completely possible to write a large JS application that's competitive with Java in performance -- but nobody does it.
V8 hasn't even really begun optimizing these things, and they're going to be incredibly hard to optimize well. So perhaps a more apt question is: when JS VMs have as much time in the oven as the JVM has had, what will the performance look like?
Highly performant JS ends up looking a lot like C, and most of the off-the-shelf JS libraries that people use to build software are littered with '.call', '.apply', 'arguments', null/undefined/true/false usage, implicit type conversions, unnecessary closures, etc. All of these are cases that V8 doesn't optimize well, and because the libraries are built to be highly abstract and useful by using these, performance drops off precipitously. It's completely possible to write a large JS application that's competitive with Java in performance -- but nobody does it.
V8 hasn't even really begun optimizing these things, and they're going to be incredibly hard to optimize well. So perhaps a more apt question is: when JS VMs have as much time in the oven as the JVM has had, what will the performance look like?
I agree; it reminded me of this, in a bad way: http://www.youtube.com/watch?v=bzkRVzciAZg
Wow- That is great. Thanks for sharing!
It's most likely a resume padder so that eBay engineers can stay in the good graces of the Silicon Valley echo chamber.
The article was about the How more than the Why (ie. "How We Built eBay's First Node.JS Application). It would be interesting to know more about the pro/con debates, but it did a pretty good job of providing high level detail on their development stack. With new technologies people often ask 'what large production environments has this been used in?' Articles like this help to answer that. And since there are so many modules and options for node developers, articles that highlight some of the more popular and mature modules can really help narrow the field for people who are wondering where to start. By the way TechEmpower has some great comparisons of performance metrics that are probably targeted more to readers like yourself who want detailed comparisons and analysis of technologies under particular use cases.
The most annoying part for me was
"This being a normal scenario in any event loop-based model like Node.js, the logs are crossed between multiple URL transactions, and the reporting tool shows scrambled output. We have worked out both short-term and long-term resolutions for this issue."
which sounds like a serious problem for pretty much anybody.
I know they're not obliged to tell us their solutions, but come on, they're a big company with lots of smart engineers and billions of dollars. We already know they can solve problems, telling us they've done so is neither interesting nor useful.
"This being a normal scenario in any event loop-based model like Node.js, the logs are crossed between multiple URL transactions, and the reporting tool shows scrambled output. We have worked out both short-term and long-term resolutions for this issue."
which sounds like a serious problem for pretty much anybody.
I know they're not obliged to tell us their solutions, but come on, they're a big company with lots of smart engineers and billions of dollars. We already know they can solve problems, telling us they've done so is neither interesting nor useful.
I still dont get it, sounds like: 'look! ebay employees are also dabbling with node'.
What did you actually build? Is it your hackathon project 'Talk' that you build on node, or was it something else?
next up.. how yahoo built their first node.js application
Given that they went with Node.js and talked about scalability, I assume they wanted the async nature of Node.js to maintain a large number of connections. I'm surprised they didn't try out Netty since they are a Java shop. The last time I investigated Node.js for async connection support in a project, the benchmarks I did showed that Netty could scale substantially better than Node.js in term of number of connections, memory usage, thread utilization, and raw performance on the same hardware. That was a year and half ago. Things might be different now, but I doubt some of fundamentals have changed much.
Anyway, it's good to try out new things. Kudos to them for doing something out of their mold.
Anyway, it's good to try out new things. Kudos to them for doing something out of their mold.
httpkit / aleph if you're using Clojure and want high-scalability async.
No CPS pyramid of doom garbage though. To get that, Node.js is your one-stop-shop.
No CPS pyramid of doom garbage though. To get that, Node.js is your one-stop-shop.
who cares about ebay anymore?
I'm curious how you're using Express if not as a framework.