Node.js Versus LUA "Hello World" HTTP Server Showdown(simonhf.wordpress.com)
simonhf.wordpress.com
Node.js Versus LUA "Hello World" HTTP Server Showdown
http://simonhf.wordpress.com/2010/10/13/node-js-versus-lua-hello-world/
6 comments
silentbicycle, thanks, I updated the name as suggested.
So do you have any ideas for a more interesting benchmark program? Ideally it should be something which needs to keep state as you suggest and makes use of some kind of simple business logic, and is only a few hundred lines of code at the very most. What about a simple chat server which handles people and rooms? The benchmark might handle, say, max. 10k people chatting in max. 1k rooms. What do you think? Any better ideas?
So do you have any ideas for a more interesting benchmark program? Ideally it should be something which needs to keep state as you suggest and makes use of some kind of simple business logic, and is only a few hundred lines of code at the very most. What about a simple chat server which handles people and rooms? The benchmark might handle, say, max. 10k people chatting in max. 1k rooms. What do you think? Any better ideas?
I think a server that takes a login and password, sends the pair to another core (or server) to do a bcrypt (http://www.openbsd.org/cgi-bin/man.cgi?query=bcrypt) check, and then responds with a pass/fail would be a good benchmark.
An async web stack benchmark server should juggle loads of concurrent connections while making internal requests to another process or server, and then sending the response (when available). That's representative of a lot of common server tasks, yet can control for the work itself.
I'm suggesting bcrypt in particular because the amount of CPU time per client can be easily controlled. (In a nutshell, bcrypt is a hash function which can be made arbitrarily slow to deter password cracking.) Offhand, I'm not sure which systems have bcrypt besides OpenBSD, though.
FWIW, I have a simple bcrypt wrapper for Lua (http://github.com/silentbicycle/lua-bcrypt).
There's also a Ruby bcrypt wrapper (http://github.com/codahale/bcrypt-ruby) which includes a copy of the bcrypt source from OpenBSD. I feel vaguely uneasy about including that with my Lua wrapper (it really should be its own library, and more widely available!), but may eventually do so.
An async web stack benchmark server should juggle loads of concurrent connections while making internal requests to another process or server, and then sending the response (when available). That's representative of a lot of common server tasks, yet can control for the work itself.
I'm suggesting bcrypt in particular because the amount of CPU time per client can be easily controlled. (In a nutshell, bcrypt is a hash function which can be made arbitrarily slow to deter password cracking.) Offhand, I'm not sure which systems have bcrypt besides OpenBSD, though.
FWIW, I have a simple bcrypt wrapper for Lua (http://github.com/silentbicycle/lua-bcrypt).
There's also a Ruby bcrypt wrapper (http://github.com/codahale/bcrypt-ruby) which includes a copy of the bcrypt source from OpenBSD. I feel vaguely uneasy about including that with my Lua wrapper (it really should be its own library, and more widely available!), but may eventually do so.
silentbicycle, I like your idea. I wonder if it will be considered 'enough' by those who poopooed the "Hello World" benchmark? On a dual core or better box and assuming the bcrypt server replies immediately, I would predict that performance of the main server would halve. Why? Because the main server can only handle n events per second and by communicating with the bcrypt server we're doubling the number of read events which must happen within the same wallclock time. What I've found is that handling e.g. epoll events is slow compared to e.g. regular function calls or other IPC mechanisms on the same box. So the fastest way to do the bcrypt idea on a production server would be to use non-socket-based-IPC (e.g. shared-memory-based which is an order of magnitude faster than socket-based-IPC). Anyway, assuming we go with socket-based-IPC for the purposes of the comparison, what's the fastest way for the node.js program to maintain state and handle a pool of connections to the bcrypt server? It would be great if a node.js guru would take this on... and then I would take on the SXE and SXELua implementations. What do you think...?
bcrypt doesn't reply immediately. You could set it to require ~10ms of CPU grinding each time it checks whether a password is valid, for example. Having latency while work happens on another core or server shouldn't impact the total number of concurrent connections the server can handle, though - the benchmark should measure that, specifically.
Oh, also - the reads and writes to the bcrypt server will be really short, and could use persistent connections. Something like "$2a$07$8BzFlUuprZN4FSBpDx3ZAuPWOu4CZ3uv8Awa4EjAZhNmnIY59nh2e" -> "OK". And yes, it would increase the number of events, but a real server is probably going to be communicating with a database, memcached, etc. instead.
FWIW, this comment thread is getting buried deeper and deeper in my threads page - if you want to keep the conversation going, my email address is in my profile. Have a good weekend.
Oh, also - the reads and writes to the bcrypt server will be really short, and could use persistent connections. Something like "$2a$07$8BzFlUuprZN4FSBpDx3ZAuPWOu4CZ3uv8Awa4EjAZhNmnIY59nh2e" -> "OK". And yes, it would increase the number of events, but a real server is probably going to be communicating with a database, memcached, etc. instead.
FWIW, this comment thread is getting buried deeper and deeper in my threads page - if you want to keep the conversation going, my email address is in my profile. Have a good weekend.
His fastest "node.js+net" case is not optimal. I could easily come up with an even faster implementation by using Buffers instead of strings by not calling setEncoding(), which makes it 30% faster on my machine. This should translate to about 37500 queries/sec on his machine:
var net = require('net');
var server = net.createServer(function (stream) {
stream.on('connect', function () {});
stream.on('data', function (data) {
var l = data.length;
if (l >= 4 && data[l - 4] == 0xd && data [l - 3] == 0xa && data[l - 2] == 0xd && data[l - 1] == 0xa) {
stream.write('HTTP/1.0 200 OK\r\nConnection: Keep-Alive\r\nContent-Type: text/html\r\nContent-Length: 13\r\n\r\nHello World\r\n');
}
});
stream.on('end', function () {stream.end();});
});
server.listen(8124, 'localhost');And this other version that doesn't check for "\r\n\r\n" is 47% faster than his "node.js+net" case. This should translate to about 42000 queries/sec on his machine:
var net = require('net');
var server = net.createServer(function (stream) {
stream.on('connect', function () {});
stream.on('data', function (data) {
stream.write('HTTP/1.0 200 OK\r\nConnection: Keep-Alive\r\nContent-Type: text/html\r\nContent-Length: 13\r\n\r\nHello World\r\n');
});
stream.on('end', function () {stream.end();});
});
server.listen(8124, 'localhost');mrb, thanks for the optimized version of the 'node.js+net' script. But what is difference with the original script? I copy and pasted your script and the queries per second result remains unchanged... so where does the 47% faster come from?
The difference with the original script is, as I said, that mine does NOT call setEncoding (if you call setEncoding, node will decode Buffers to strings, thereby wasting CPU time). My 47% faster number comes from benchmarking node.js+net against my optimized version (both on my machine).
Hmmm... I'm confused because my node.js+net script also doesn't call setEncoding() and looks the same as your node.js+net script. The node.js+net+crcr script of yours is a nice optimization though on my test box it reaches 23,224 queries per second.
We are getting closer to open sourcing SXELua :-) It would be great if we could think up a better benchmark then the "Hello World" benchmark :-) What do you think about the suggestion described higher up in this thread?
We are getting closer to open sourcing SXELua :-) It would be great if we could think up a better benchmark then the "Hello World" benchmark :-) What do you think about the suggestion described higher up in this thread?
I see. I must have gotten confused somewhere then. node.js+net+crcr was calling setEncoding, but you are indeed not calling setEncoding in node.js+net...
I think the ideal is a bunch of little synthetic benchmarks, similar to the Hello World, plus a handful of real-world examples like the one checking a password. You want something that stresses the CPU / RAM (not network), since that's where the interesting differences are between these scripting environments.
I think the ideal is a bunch of little synthetic benchmarks, similar to the Hello World, plus a handful of real-world examples like the one checking a password. You want something that stresses the CPU / RAM (not network), since that's where the interesting differences are between these scripting environments.
Sounds good. How many lines of code do you think the password check test would be in node.js? If it's not too long then could you imagine doing the node.js version?
Why empty callback on 'connect'? It will go to event loop for every client, and yet do nothing of value.
mrb, thanks for the optimized node.js code. I re-ran the benchmark and the queries per second increased to 23,224 queries per second... so I agree with the 30% faster but not your queries per second estimate (which math did you do? ~ 18k * 130% = ~ 23k or?). I will update the blog later to reflect this.
I'm frustrated by Lua, because it seems such a productive, pragmatic language that is performant to boot (let alone the numbers LuaJIT is posting), but yet it never seems to gain momentum.
Why has Lua adoption been so slow outside the game world?
Why has Lua adoption been so slow outside the game world?
Lua is gaining momentum, but it's been a very slow burner. There are a lot of programs (mostly games) that use it, and they aren't required to be open about it. Lua also improved quite a bit (IMHO) between 4.0 and 5.1, but 5.1 has only been out since 2006.
Some things omitted from Lua's std libs because you'd typically just do them in C anyway, and it keeps Lua very small. While you don't need to know C to use Lua (and there are community libraries for them, see LuaRocks: http://luarocks.org/repositories/rocks/), it really shines in symbiosis with C.
Most web devs don't know C, though, so it misses quite a bit of the free hype that would come from word of mouth in that community (look at e.g. Ruby), especially compared to word of mouth in the (NDA-infested) gaming industry.
FWIW, I don't think it would take long for somebody proficient in Javascript (or Python) to learn Lua - This comment (http://news.ycombinator.com/item?id=1786280) is a good summary. The languages are very similar in overall design (prototypes, JSON/dict-style objects, etc.), but where Javascript has numerous design bugs frozen in its spec (thanks to the browser wars) and "The Good Parts" telling how to sidestepping them, Lua had 15 extra years to evolve and fixed them.
Recommended intro: Ieursalimschy's _Programming in Lua, 2nd ed._ (http://www.inf.puc-rio.br/~roberto/pil2/). The first edition is free online, but covers 4.0, and the language changed a lot (most notably, the packaging system).
Some things omitted from Lua's std libs because you'd typically just do them in C anyway, and it keeps Lua very small. While you don't need to know C to use Lua (and there are community libraries for them, see LuaRocks: http://luarocks.org/repositories/rocks/), it really shines in symbiosis with C.
Most web devs don't know C, though, so it misses quite a bit of the free hype that would come from word of mouth in that community (look at e.g. Ruby), especially compared to word of mouth in the (NDA-infested) gaming industry.
FWIW, I don't think it would take long for somebody proficient in Javascript (or Python) to learn Lua - This comment (http://news.ycombinator.com/item?id=1786280) is a good summary. The languages are very similar in overall design (prototypes, JSON/dict-style objects, etc.), but where Javascript has numerous design bugs frozen in its spec (thanks to the browser wars) and "The Good Parts" telling how to sidestepping them, Lua had 15 extra years to evolve and fixed them.
Recommended intro: Ieursalimschy's _Programming in Lua, 2nd ed._ (http://www.inf.puc-rio.br/~roberto/pil2/). The first edition is free online, but covers 4.0, and the language changed a lot (most notably, the packaging system).
Nitpick: The first edition of the PiL Book covers Lua 5.0 which is very similar to 5.1.
The most noticeable changes are the vararg handling, the module system (as you said, but it will be phased out in v5.2), and some library changes.
The differences are listed here: http://www.lua.org/manual/5.1/manual.html#7
That said, I purchased the second edition to understand the module system, only to decide not to use it anyway :-). I don't like the way the `module` function pollutes the global namespace.
The most noticeable changes are the vararg handling, the module system (as you said, but it will be phased out in v5.2), and some library changes.
The differences are listed here: http://www.lua.org/manual/5.1/manual.html#7
That said, I purchased the second edition to understand the module system, only to decide not to use it anyway :-). I don't like the way the `module` function pollutes the global namespace.
5.0, not 4.0 - you're right, thanks. Missed the edit window. :/
And yes, the 5.2 environment handling sounds like an improvement to me as well.
And yes, the 5.2 environment handling sounds like an improvement to me as well.
I liked setfenv() for its versatility.
I implemented a toy markaby [1] clone in Lua using it.
This example also highlights a weakness in the language syntax: lambdas are ugly. Functions are first-class, semantically, but not syntactically, which is sad.
[1] http://markaby.rubyforge.org/ , just in case ;-)
I implemented a toy markaby [1] clone in Lua using it.
tpl = html(function()
head(function() --> tag names should be non-locals. They will be rebound at runtime.
title "Barbazor"
link {rel = "stylesheet", type = "text/css", href = "style.css"}
end
body(function()
h1 {"Big title"; id = "title"}
div {class = "foo"; function()
for i=1,5 do
p( "counting: " .. i )
end
end}
end
end
render( tpl ) --> returns clean, well indented HTML.
This example also highlights a weakness in the language syntax: lambdas are ugly. Functions are first-class, semantically, but not syntactically, which is sad.
[1] http://markaby.rubyforge.org/ , just in case ;-)
Hah. I did exactly the same thing. I usually use discount instead, though. To my understanding, it'd work to set _ENV's metatable's __index to a table w/ HTML tag generators. I've read about _ENV but haven't actually used 5.2-work4 yet. (I started using 5.2-work2 but they went and changed it again, so now I'm waiting until it's a sure thing.)
I agree that the syntax for first class funs could be more concise, but that's true of any language that requires a "return" keyword (i.e., is statement-based rather than expression-based). I think "fun x -> x + 1 end" would be better, but maybe that's just the OCaml talking.
I agree that the syntax for first class funs could be more concise, but that's true of any language that requires a "return" keyword (i.e., is statement-based rather than expression-based). I think "fun x -> x + 1 end" would be better, but maybe that's just the OCaml talking.
Using _ENV would not be that simple.
The only way to have a clean template file would be to emulate setfenv using the debug library.
--
My main gripe with the current syntax is that, when scanning the code, function declarations and invocations are too similar.
I'd like to have somthing like
--
At last, what is discount?
* The library and template files may have different _ENVs.
* The _ENV of the tempalte file could already have an __index.
The only way to have a clean template file would be to emulate setfenv using the debug library.
--
My main gripe with the current syntax is that, when scanning the code, function declarations and invocations are too similar.
I'd like to have somthing like
:[x] print(x); -> x+1 end :]x[ -> x+1 ] somefun :]x[ dosomething( x ) ]--
At last, what is discount?
Discount (http://www.pell.portland.or.us/~orc/Code/discount/) is a very fast, C-based implementation of Markdown (http://daringfireball.net/projects/markdown/).
I have a Lua wrapper for it, but just noticed that I don't have it on my github page. I'll post it soonish. Somebody else has a Lua wrapper for an older version of discount, but includes the discount source in it (!), and it's been updated due to security-related bugs, so that's out.
Talking about reversed brackets, look at J (http://www.jsoftware.com/). The only language I've seen that uses ], [, {, and } as unpaired operators. Pretty serious heresy. Cool language, though.
I have a Lua wrapper for it, but just noticed that I don't have it on my github page. I'll post it soonish. Somebody else has a Lua wrapper for an older version of discount, but includes the discount source in it (!), and it's been updated due to security-related bugs, so that's out.
Talking about reversed brackets, look at J (http://www.jsoftware.com/). The only language I've seen that uses ], [, {, and } as unpaired operators. Pretty serious heresy. Cool language, though.
Thanks for all that :)
Also, besides LuaRocks, some more libraries are listed on the Lua wiki: http://lua-users.org/wiki/LibrariesAndBindings
I think this mostly has to do with the 'Batteries -not- included' approach of Lua.
This is mainly done to keep the language/vm compact and embeddable, yet what it leads to is many similar implementations of modules that do not play well together.
This is mainly done to keep the language/vm compact and embeddable, yet what it leads to is many similar implementations of modules that do not play well together.
I read a related article a while back: http://journal.dedasys.com/2010/03/30/where-tcl-and-tk-went-...
It's interesting to draw parallels to Lua, they both started out as embedded scripting languages with a small core.
Great article:-) I think managing the addition of batteries is definitely tricky for a language like Lua. Some of its users do not want any additional baggage: they want small, fast and easy to embed. Other people may want to use it as a 'scripting language', and thus want to have a bunch of libraries and additional goodies to get stuff done with.
This. Lua doesn't even have a standard class/inheritance model. If you want that, you implement it in-language. There are some nice general purpose ones out there, but nothing canonical.
If someone makes a killer web framework on top of one of these models, that might establish the standard and bootstrap the Lua scene, kind of like how Rails established a lot of idioms for Ruby.
If someone makes a killer web framework on top of one of these models, that might establish the standard and bootstrap the Lua scene, kind of like how Rails established a lot of idioms for Ruby.
re: makes a killer web framework
There have been two Ruby web framework ported to Lua:
* Orbit (http://orbit.luaforge.net/) - Camping port
* Mercury (http://github.com/nrk/mercury) - Sinatra port
There have been two Ruby web framework ported to Lua:
* Orbit (http://orbit.luaforge.net/) - Camping port
* Mercury (http://github.com/nrk/mercury) - Sinatra port
There's also Kepler (http://www.keplerproject.org/), which I haven't used, and WSAPI (http://keplerproject.github.com/wsapi/), which is roughly like Rack or WSGI. It's not a "web framework", but using WSAPI with FastCGI works well for CGI-ish stuff.
I think those who find it a productive tool, just use it without waiting for the whole herd to head that direction - and since it's just a tool that provides a productivity edge - why advertise too much about it on blogs ;-)
http://www.cisco.com/en/US/docs/security/csd/csd35/configura...
(disclaimer: cisco is my employer).
(disclaimer#2: I did use lua for quick prototypes that involved the low-level C code and higher level logic to glue it. So the positive feelings I got from it may be biasing me).
http://www.cisco.com/en/US/docs/security/csd/csd35/configura...
(disclaimer: cisco is my employer).
(disclaimer#2: I did use lua for quick prototypes that involved the low-level C code and higher level logic to glue it. So the positive feelings I got from it may be biasing me).
Agreed. This is the first time I've seen someone mention lua on a non-lua based community site. :(
There have been quite a few sightings of Lua here on HN: http://www.google.com/search?q=site:news.ycombinator.com+lua
Why has Lua adoption been so slow outside the game world?
No libraries.
No libraries.
Well, there are plenty at LuaRocks (http://luarocks.org/repositories/rocks/). It's no CPAN, but the essentials are there. I'm working on a couple, such as Tamale (http://github.com/silentbicycle/tamale/), which adds Erlang-style pattern matching. I'll post it on LuaRocks once the docs are done.
Of course, if you're embedding Lua (in a game, for example) you've already got the libraries you need, and extra libraries would just get in the way.
Of course, if you're embedding Lua (in a game, for example) you've already got the libraries you need, and extra libraries would just get in the way.
LuaJIT 2.00 calls "C" functions a bit slower than reference lua or luajit 1.00. Mike Pall explained that in the newsgroup, if I'm not wrong. But the idea with LuaJIT is to use "C" much less.
IIRC, code sections with foreign call cannot be traced by the compiler... Or, if they can be, the performance would be worse than not doing it (my memory is a bit foggy).
A custom API for LuaJIT is in the works, to alleviate this problem. Extensions will have to be written twice though, if one wants to support both runtimes.
A custom API for LuaJIT is in the works, to alleviate this problem. Extensions will have to be written twice though, if one wants to support both runtimes.
What versions of node and Lua are used in the benchmark? Where can SXE be found (Google can't seem to find it)...?
Let's remember that Node.js is more about predictability, scalability, and (most of all) simplicity of development, than performance. Lua(JIT) is indeed impressive, but having to learn another programming language is a huge downside.
Attitudes like yours make me sad, on many levels.
It also makes me mad, because by every metric save the what-you-already-know, Lua is at least comparable, if not outright superior.
It also makes me mad, because by every metric save the what-you-already-know, Lua is at least comparable, if not outright superior.
Strings operations is a tricky part though.
It's easy to slow the code down with "obvious" coding - which is a bad thing in a language.
Some other operations like metatable lookup might not be always the fastest.
But overall the convenience of having a 700K source tarball that compiles on any ANSI C into a modern language blows all these concerns away for my use. And the performance-critical parts can be easily pushed down to C. If they are not premature optimizations to begin with.
It's easy to slow the code down with "obvious" coding - which is a bad thing in a language.
Some other operations like metatable lookup might not be always the fastest.
But overall the convenience of having a 700K source tarball that compiles on any ANSI C into a modern language blows all these concerns away for my use. And the performance-critical parts can be easily pushed down to C. If they are not premature optimizations to begin with.
How are string ops tricky in Lua? Patterns and table.concat can get a lot done, and when you need more power, there's LPEG (which is also quite speedy).
"Patterns and table.concat can get a lot done" - sure. but it is may feel somewhat backasswards :-)
$ time lua -e 's="";for i=1,100000 do s=s.."-";end'
real 0m1.292s user 0m1.029s sys 0m0.108s
$ time lua -e 't={};for i=1,100000 do table.insert(t,"-");end; s=table.concat(t,"")'
real 0m0.284s user 0m0.062s sys 0m0.077s
times are somewhat bogus since it is cygwin under win7, but they give the impression of what I am talking about.
I do not argue it is a feature very coherent with the design of the language, and the immutable strings have other good properties. But if you do intensive string ops you need to watch out not to fall into the trap of treating the strings as mutable objects.
Don't get me wrong - I like Lua a lot and use it almost on a daily basis. Just that I think the posts like this may attract some new folks to try Lua out - and they deserve to know the obvious ways they can shoot themselves in the foot in order to avoid some disappointment.
$ time lua -e 's="";for i=1,100000 do s=s.."-";end'
real 0m1.292s user 0m1.029s sys 0m0.108s
$ time lua -e 't={};for i=1,100000 do table.insert(t,"-");end; s=table.concat(t,"")'
real 0m0.284s user 0m0.062s sys 0m0.077s
times are somewhat bogus since it is cygwin under win7, but they give the impression of what I am talking about.
I do not argue it is a feature very coherent with the design of the language, and the immutable strings have other good properties. But if you do intensive string ops you need to watch out not to fall into the trap of treating the strings as mutable objects.
Don't get me wrong - I like Lua a lot and use it almost on a daily basis. Just that I think the posts like this may attract some new folks to try Lua out - and they deserve to know the obvious ways they can shoot themselves in the foot in order to avoid some disappointment.
Context: Since Lua strings are immutable (and interned, like Erlang atoms, Lisp symbols, etc.), tables can be used as string buffers, and concatenation to large strings can be done in one step via table.concat. Appending to a string in a loop is slow, but that's true of any language with immutable strings.
$ time lua -e 't={};for i=1,100000 do table.insert(t,"-");end; s=table.concat(t,"")'
table.insert checks the table's length on each iteration, which dominates the string-handling time - len is O(lg n). If you just use the loop index (or otherwise cache it), it cuts the time in half:
$ time lua -e 't={};for i=1,100000 do t[i] = "-"; end; s=table.concat(t,"")'
$ time lua -e 't={};for i=1,100000 do table.insert(t,"-");end; s=table.concat(t,"")'
0m0.05s real 0m0.05s user 0m0.01s system
0m0.05s real 0m0.05s user 0m0.00s system
0m0.05s real 0m0.04s user 0m0.02s system
table.insert checks the table's length on each iteration, which dominates the string-handling time - len is O(lg n). If you just use the loop index (or otherwise cache it), it cuts the time in half:
$ time lua -e 't={};for i=1,100000 do t[i] = "-"; end; s=table.concat(t,"")'
0m0.02s real 0m0.02s user 0m0.00s system
0m0.02s real 0m0.02s user 0m0.01s system
0m0.02s real 0m0.03s user 0m0.00s system
There is also the code reuse factor. Why learn/use yet other language if you can use javascript for both server and client?
Better yet, let's go back to using ALGOL or something. Imagine how many reusable components we could have mustered up. We'd best stop all this innovation right now before it's too late.
problem is: you cannot use it for both server and client. if this were true you would've been right. with node.js you still need to be aware if you're coding client or serverside for some constructs.
Not really a problem; clients and servers always use different APIs in other languages. JavaScript is no difference.
The advantage is that you get to use the same tools for client-side development and server-side development. Testing tools, interactive development tools, editor tools, etc. When you don't have to build your toolchain twice, you can make your toolchain twice as good.
In theory. Javascript needs namespaces and modules before it will be useful for real work.
The advantage is that you get to use the same tools for client-side development and server-side development. Testing tools, interactive development tools, editor tools, etc. When you don't have to build your toolchain twice, you can make your toolchain twice as good.
In theory. Javascript needs namespaces and modules before it will be useful for real work.
Without getting into a huge my language vs your language debate: I agree that this isn't the RIGHT attitude, but it's not about me learning Lua per say, it's a reality of writing code in the real world for real projects.
> I agree that this isn't the RIGHT attitude, but it's not about me learning Lua per say, it's a reality of writing code in the real world for real projects.
Lots of people write robust, real world code in languages they are only casually familiar with. That is how you learn a language well.
The other reality you're facing here is your career. While it is gradually slowing, our field moves quickly. You need to be constantly studying and reading and learning to be any good at it at all. The reality is: grow and keep up or stop and be left behind.
To me, a new language is an opportunity. “What insights do these people have?” “What new design patterns will I learn?” “What can I take from this to my daily coding jobs?” They excite me.
Lots of people write robust, real world code in languages they are only casually familiar with. That is how you learn a language well.
The other reality you're facing here is your career. While it is gradually slowing, our field moves quickly. You need to be constantly studying and reading and learning to be any good at it at all. The reality is: grow and keep up or stop and be left behind.
To me, a new language is an opportunity. “What insights do these people have?” “What new design patterns will I learn?” “What can I take from this to my daily coding jobs?” They excite me.
Again, I agree with you on all points. I can't count on my hands the number of languages I've tried out and written a decent bit of code in, learning things along the way. However, that doesn't translate into "I should use this platform for a business-critical application."
Seems strange to decide on node.js if that's the problem. Node.js is by far less proven than Lua.
No kidding. Lua has been used in production in industry for over 15 years.
http://www.lua.org/history.html
http://www.lua.org/history.html
Is there a framework for writing asynchronous servers with Lua?
Working on it. Libev-based. It's a spare-time project of mine. I have the "portable but not particularly scalable" pure Lua and luasocket backend done (for easy development + light deployment on Windows), working on the error handling in the libev backend now.
Much of the complexity comes from error handling, not just the async IO. Erlang gets that right, too. I don't know how much error handling node.js does. (I plan on checking it out eventually, since my framework will be compared to it.)
I don't have a time frame yet. I'm also working on a compiler and a bunch of other stuff, and I'm really not a web developer, so it's been a fairly low priority.
Much of the complexity comes from error handling, not just the async IO. Erlang gets that right, too. I don't know how much error handling node.js does. (I plan on checking it out eventually, since my framework will be compared to it.)
I don't have a time frame yet. I'm also working on a compiler and a bunch of other stuff, and I'm really not a web developer, so it's been a fairly low priority.
I don't think that's what Tichy meant. That's just a wrapper library to make error handling easier in a coroutine-based event loop.
If you're not worried about being "web scale", it's not hard to write an async socket server using luasocket's sockets for nonblocking TCP IO and luasocket.select for scanning for active connections. It gets bogged down once you have > 100 or so idle connections (the usual trade-offs with select apply), but it's easy to write, very portable, and (on x86) LuaJIT is amazingly fast. Contact me if you want example code.
I've been working on an actual framework for async servers (libev-based), but haven't had much free time lately.
If you're not worried about being "web scale", it's not hard to write an async socket server using luasocket's sockets for nonblocking TCP IO and luasocket.select for scanning for active connections. It gets bogged down once you have > 100 or so idle connections (the usual trade-offs with select apply), but it's easy to write, very portable, and (on x86) LuaJIT is amazingly fast. Contact me if you want example code.
I've been working on an actual framework for async servers (libev-based), but haven't had much free time lately.
You never really learn a new language in the case of syntax that's all peanuts. Javascript and Lua are both prototype based languages and such trivial differences exist that can be ironed out over a week of productive use.
In my case, it was a couple of days. I read the free online version of the Programming in Lua[1] book and was basically set.
Lua feels like a simpler, cleaner Javascript.
--
You have lexical scope, anonymous and first class functions, and dynamic typing.
The parser is as simple as possible (LL(1), ie top down, without backtracking), and doesn't do any magic like declaration hoisting or semi-column insertion.
The only falsy objects are `false` and `nil`. `==` doesn't do any type conversion.
The addition `+` and concatenation `..` operators are different.
Some infix operators do automatic type conversion ( +, -, *, /, %, ..) between strings and numbers, but that's about it for weak typing. There's no octal to decimal conversion for strings with prepended zeros.
There's only one way to define objects. The `self` parameter is explicit for methods, but there's syntactic sugar to pass it automatically.
Scoping is similar: variables are global by default, which allows precise scoping for closures.
--
== Where the languages diverge:
You can do some magic with metatables and environments, but once again, the mental model is extremely simple.
The other major differences are coroutines (cooperative threads) and a "generic" `for` loop that allows to build custom idiomatic iterators.
At last, the array indexing starts at 1. This may sound like heresy to devout Dijkstra followers, but it's really not a big deal in practice.
--
[1]: http://www.lua.org/pil/ by Roberto Ierusalimschy, one of the author of the language.
Lua feels like a simpler, cleaner Javascript.
--
You have lexical scope, anonymous and first class functions, and dynamic typing.
The parser is as simple as possible (LL(1), ie top down, without backtracking), and doesn't do any magic like declaration hoisting or semi-column insertion.
The only falsy objects are `false` and `nil`. `==` doesn't do any type conversion.
The addition `+` and concatenation `..` operators are different.
Some infix operators do automatic type conversion ( +, -, *, /, %, ..) between strings and numbers, but that's about it for weak typing. There's no octal to decimal conversion for strings with prepended zeros.
There's only one way to define objects. The `self` parameter is explicit for methods, but there's syntactic sugar to pass it automatically.
Scoping is similar: variables are global by default, which allows precise scoping for closures.
--
== Where the languages diverge:
You can do some magic with metatables and environments, but once again, the mental model is extremely simple.
The other major differences are coroutines (cooperative threads) and a "generic" `for` loop that allows to build custom idiomatic iterators.
At last, the array indexing starts at 1. This may sound like heresy to devout Dijkstra followers, but it's really not a big deal in practice.
--
[1]: http://www.lua.org/pil/ by Roberto Ierusalimschy, one of the author of the language.
I think the most compelling thing about Lua (to someone with no Lua experience), is that its appears (from examples, and what people have said) that its good for embedding inside another program (size wise and good FFI).
In a video I saw, by one of the designers of the languages stated that this was the original purpose (engineers using this instead of C, with the low level details done by programming team in C).
In a video I saw, by one of the designers of the languages stated that this was the original purpose (engineers using this instead of C, with the low level details done by programming team in C).
Indeed, C(++) interop is easy. The API is also described in the PiL Book linked above.
Furthermore, the VM binary weights 200k. It is extremely portable (it uses the common subset of C89 and C++), and its code is a pleasure to hack.
Furthermore, the VM binary weights 200k. It is extremely portable (it uses the common subset of C89 and C++), and its code is a pleasure to hack.
Just had a quick skim over some of that chapter, and it is very compelling, especially when I see how well its worked for AwesomeWM (my desktop environment of choice) and how many commercial games are using it for user scripting [1].
[1] http://en.wikipedia.org/wiki/Category:Lua-scripted_video_gam...
[1] http://en.wikipedia.org/wiki/Category:Lua-scripted_video_gam...
Wow - thanks for this summary. I hadn't heard a rundown like this before. That sounds great - I might give Lua a go!
A lot of that is true if you compare it against Erlang, which has a much different approach to concurrency – and has a steeper learning curve for a Python/Ruby+JavaScript web developer.
Learning Lua on the other hand? If that's not utterly trivial, I'd be worried.
The biggest downside Lua always had was that it's deeply rooted in the embedded category, where it's hard to come out from. Tcl is one the market for ages and still doesn't have a proper culture of libraries. At least Lua has its "rocks", maybe that will make it a worthwhile candidate for the non-embedded sector, too. It's a much nicer language than most of its rivals.
Learning Lua on the other hand? If that's not utterly trivial, I'd be worried.
The biggest downside Lua always had was that it's deeply rooted in the embedded category, where it's hard to come out from. Tcl is one the market for ages and still doesn't have a proper culture of libraries. At least Lua has its "rocks", maybe that will make it a worthwhile candidate for the non-embedded sector, too. It's a much nicer language than most of its rivals.
Lua developed a "culture of libraries" recently, and is catching up. Another part of the problem was that it didn't really have a proper module system until a couple years ago (partially due to the focus on embedding).
Besides LuaRocks, there's a collection on the Lua wiki (http://lua-users.org/wiki/LibrariesAndBindings), and the mailing list (http://www.lua.org/lua-l.html) is very active.
And yeah - if you know Javascript, you won't have trouble learning Lua. It's like Javascript with less gotchas and better performance.
Besides LuaRocks, there's a collection on the Lua wiki (http://lua-users.org/wiki/LibrariesAndBindings), and the mailing list (http://www.lua.org/lua-l.html) is very active.
And yeah - if you know Javascript, you won't have trouble learning Lua. It's like Javascript with less gotchas and better performance.
but having to learn another programming language is a huge downside
Not if you can also use it to write World Of Warcraft addons :)
Not if you can also use it to write World Of Warcraft addons :)
Indeed. I learned it doing that, and have come to appreciate what a marvelous little language it is. Always a pleasure to work with.
Got a link? I wasn't aware that it is possible for common people to write WoW addons?
Probably the best starting point:
http://us.blizzard.com/support/article.xml?locale=en_US&...
http://us.blizzard.com/support/article.xml?locale=en_US&...
Scalability, eh? What about that whole "scale beyond a single CPU" thing. I hear it's quite the rage.
If node can scale beyond a single machine, then it can scale beyond a single CPU on a machine. I know I'm oversimplifying, but you're oversimplifying more.
There's nothing in node that provides mechanisms for cross process scalability. Compared to Erlang, for instance, which provides mechanisms for message passing and scaling past one node, and even one node can efficiently use multiple cores. Node is quite fast but not scalable in that sense.
So last time I looked at it, Node.js did not have any language level or even library primitives for message passing. At least nothing beyond making you open a socket and work out your own protocol for IPC. I'll grant you that is the standard scalability story for most languages running on multiple hosts. However it seems deficient if one's goal is to make use of most of the cores on modern server hardware.
That's a really good point. What sort of message passing support would you like to see in node? I have some free time.
Being able to stream arbitrary data as JSON already get you a lot of flexibility. Can you stream Javascript functions, though? (You can in Lua, but doing so w/ closures is tricky.)
Translation: per-client, it uses very little resources, and V8 garbage collection is a good fit for the type of services Node.js targets. It's mostly about memory.
... and yes, as others have pointed out, Node.js supports process forking, which is arguably the better model for CPU scaling in these types of services.
... and yes, as others have pointed out, Node.js supports process forking, which is arguably the better model for CPU scaling in these types of services.
What about that whole "fork" thing. I hear it's quite the rage
Your OS's copy-on-write page policy loves when you fork things using mark-and-sweep garbage collection. It is full of rage.
Last time I checked Node.js was not able to invoke fork.
It can spawn child processes:
http://nodejs.org/api.html#child-processes-89 http://www.sitepen.com/blog/2010/07/14/multi-node-concurrent...
http://nodejs.org/api.html#child-processes-89 http://www.sitepen.com/blog/2010/07/14/multi-node-concurrent...
This comment was ranked 0 when I found it. It's exactly right. Startups are risky enough without having to hire a team of Lua developers. An async server that supports a mainstream language is awesome. Lua? No.
In this case, you would be looking at PHP and Java, not node.js.
Last time I checked node.js and libraries for it (about a couple of months ago), I could not run the library I wanted on the latest version of the node.js because the API has completely changed.
Platform software that completely changes itself over a course of a couple of months ? Thanks but no thanks.
Disclaimer: It's been couple of months. Maybe the things have changed since then, will be grateful for an educational reply from someone who has used node.js, about its maintainability, especially for the use with third party libs.
Last time I checked node.js and libraries for it (about a couple of months ago), I could not run the library I wanted on the latest version of the node.js because the API has completely changed.
Platform software that completely changes itself over a course of a couple of months ? Thanks but no thanks.
Disclaimer: It's been couple of months. Maybe the things have changed since then, will be grateful for an educational reply from someone who has used node.js, about its maintainability, especially for the use with third party libs.
Node's API was frozen with the 0.2.0 release (for most parts). Things were indeed very rocky before, but most libraries have been updated now and are unlikely to break again soon.
It looks like the zealots got to you. I agree partially with what you're saying. Node.js is still a somewhat immature platform, but people are generally building small, single-purpose servers with it, not replacing their entire infrastructure. I don't think anyone is advocating that.
Also: Serving "Hello World" is not a very informative benchmark, especially for an async web stack. Something like making an asynchronous database request and then sending the response may be more representative. If you want to eliminate the database response time variable, then write a C program that listens for incoming connections and responds with "Hello world" or something, but either way, make it actually do some work server side.