Python Community in Anguish, Pain, Despair Over Web Server(pwpwp.blogspot.com)
pwpwp.blogspot.com
Python Community in Anguish, Pain, Despair Over Web Server
http://pwpwp.blogspot.com/2009/09/python-community-in-anguish-pain.html
13 comments
The misleading graph doesn't help their case, either - tests were run at 1,5,10,25,50,75, and 100. The graph mistakenly spaces them evenly apart. In reality, twisted.web died after 50, while Tornado still worked @ 100.
Also if you're planning to do any real Comet stuff, this is an extremely small-scale test. Should try with a few thousand concurrent requests to gain any real insight.
http://antoniocangiano.com/2009/09/13/benchmarking-tornado-v...
2000 concurrent requests
2000 concurrent requests
If I may reiterate, twisted.web died between 50 and 75, and that graph showing 1-100 was very misleading.
OK OK OK, sheesh, I'm going to redo the test. I was out of town over the weekend and really haven't had a chance to update it.
It is a small scale test, it was done on my macbook pro... I'm redoing it on better hardware and tests.
It is a small scale test, it was done on my macbook pro... I'm redoing it on better hardware and tests.
"So much worry" != "a few people close to the software wrote some blog posts at each other".
In fact almost nothing of what I've seen qualifies as "worry". "Hey, my stuff performs better than you said it did" isn't worrying, and neither is "Hey, I can graft your stuff onto that other stuff and maybe get the best of both worlds."
All in all, it seems more like a "productive conversation" than some sort of worrisome flame war.
In fact almost nothing of what I've seen qualifies as "worry". "Hey, my stuff performs better than you said it did" isn't worrying, and neither is "Hey, I can graft your stuff onto that other stuff and maybe get the best of both worlds."
All in all, it seems more like a "productive conversation" than some sort of worrisome flame war.
Author takes one quote from Terry Jones and miraculously transforms it into "anguish, pain, despair" from the Python community. From the other posts on that blog I get that he's trying to be funny, but this one follows TechCrunch's "trolling and creating drama for massive pageviews" style guide so slavishly that any humor intended just falls flat imo.
It's ridiculous how every time new software is released we must turn its position vis-a-vis existing similar software into some kind of soap opera.
Hey I know how we can make this even more dramatic. Async aside, Tornado is also a web framework. How about we get the developers of Django, CherryPy, TurboGears, etc to all castigate Friendfeed devs for not choosing their framework and adding easy async support?
Just because some software has overlap with others doesn't mean we all have to automatically pick sides. I love Twisted. I look forward to building something with Tornado. I am eager to see dlsspy's efforts at interop between them come to fruition.
At the end of the day, there is new, well-documented free software available for Python. The rest is up to the community.
It's ridiculous how every time new software is released we must turn its position vis-a-vis existing similar software into some kind of soap opera.
Hey I know how we can make this even more dramatic. Async aside, Tornado is also a web framework. How about we get the developers of Django, CherryPy, TurboGears, etc to all castigate Friendfeed devs for not choosing their framework and adding easy async support?
Just because some software has overlap with others doesn't mean we all have to automatically pick sides. I love Twisted. I look forward to building something with Tornado. I am eager to see dlsspy's efforts at interop between them come to fruition.
At the end of the day, there is new, well-documented free software available for Python. The rest is up to the community.
FWIW, I tried to write a satire by using only real quotes, and making nothing up (because I find the whole affair completely ridiculous by itself.)
Judging by the comments left on the blog, I guess I failed.
Judging by the comments left on the blog, I guess I failed.
The controversy surrounding this is just really funny.
I wrote my own comet webserver in Java to run Mibbit (Hopefully I'll open source it one day soon - maybe when facebook acquire me lol). It's pretty awesome IMHO, and happily handles thousands of concurrent users in a single thread, doing thousands of http requests a second, on a medium size VPS server.
Writing your own stuff makes perfect sense. Here's why I personally did it
* Comet is very important to a realtime app. I wanted to make sure I had the best server in the world for that.
* I wanted to learn the complete low level details so I could be sure that the protocol,systems,etc were best possible, and just so I know exactly what's going on under the hood. (Knowing the whole code base inside out means I can really heavily optimize)
* One of the worst situation you can ever be in IMHO, is trying to find a bug, which ends up being in some badly written 3rd party lib. If it's in your own code, it's far easier to find+fix.
* It's not that hard. It's just webserver networking etc code.
Yes there were other comet servers out there, but they all seemed pretty 'general purpose', overly complex, etc for what I wanted.
I think the take-away from the Tornado release isn't "Wow! There's a new web framework here you should use it!". It should be "If it makes sense to reinvent the wheel, you should totally reinvent the wheel."
The amount you learn in the process is invaluable in itself.
I wrote my own comet webserver in Java to run Mibbit (Hopefully I'll open source it one day soon - maybe when facebook acquire me lol). It's pretty awesome IMHO, and happily handles thousands of concurrent users in a single thread, doing thousands of http requests a second, on a medium size VPS server.
Writing your own stuff makes perfect sense. Here's why I personally did it
* Comet is very important to a realtime app. I wanted to make sure I had the best server in the world for that.
* I wanted to learn the complete low level details so I could be sure that the protocol,systems,etc were best possible, and just so I know exactly what's going on under the hood. (Knowing the whole code base inside out means I can really heavily optimize)
* One of the worst situation you can ever be in IMHO, is trying to find a bug, which ends up being in some badly written 3rd party lib. If it's in your own code, it's far easier to find+fix.
* It's not that hard. It's just webserver networking etc code.
Yes there were other comet servers out there, but they all seemed pretty 'general purpose', overly complex, etc for what I wanted.
I think the take-away from the Tornado release isn't "Wow! There's a new web framework here you should use it!". It should be "If it makes sense to reinvent the wheel, you should totally reinvent the wheel."
The amount you learn in the process is invaluable in itself.
[deleted]
I've spent the last two weeks at Python conferences in Argentina and Brazil, have been following the news from DjangoCon, and like to think I've got a pretty good pulse on the Python web community.
I'd hardly characterize the Python web community's attitude as "anguish, pain, dispair." I see a lot more excitement, curiosity, and shiny-new-toy-syndrome; I think most reasonable Pythonistas agree that Tornado is a bit net positive to Python web development.
Those on the outside tend to mistake impassioned discussions for anger, but those who've been in this community for some time know that on balance there's a pretty strong collegial atmosphere. We compete, sure, but we're also friends and the spirit's friendly. Nobody thinks they're the be-all and end-off of web development; we all recognize our software is just yet another stepping stone along the path towards something better.
I'd hardly characterize the Python web community's attitude as "anguish, pain, dispair." I see a lot more excitement, curiosity, and shiny-new-toy-syndrome; I think most reasonable Pythonistas agree that Tornado is a bit net positive to Python web development.
Those on the outside tend to mistake impassioned discussions for anger, but those who've been in this community for some time know that on balance there's a pretty strong collegial atmosphere. We compete, sure, but we're also friends and the spirit's friendly. Nobody thinks they're the be-all and end-off of web development; we all recognize our software is just yet another stepping stone along the path towards something better.
I had a play with Tornado over the weekend. It's fabulous. The docs are easy to follow, and they do a great job of walking through the things you need to be thinking about. I like learning a framework by following exercises and gradually memorising how to set up a basic app typing into an empty console with no notes. I'm there with tornado already. The abstractions are nice.
Twisted is already cool, and I look forward to the polished version of the Tornado-on-Twisted making it into the repository. I suspect docs modeled on the current Tornado docs will make twisted a lot easier to learn than it currently is.
Twisted is already cool, and I look forward to the polished version of the Tornado-on-Twisted making it into the repository. I suspect docs modeled on the current Tornado docs will make twisted a lot easier to learn than it currently is.
The state of twisted today isn't that interesting. Without knowing when they started on working on Tornado and what that state of twisted and its documentation was at that point one cannot really comment on if it was a good idea or not.
I know I've written code that years later looked like pointless reinventing of wheels, but when I started writing the code there really wasn't anything else that did what we needed. And once I had something working it was often easier to maintain it than to port it to some other library.
I know I've written code that years later looked like pointless reinventing of wheels, but when I started writing the code there really wasn't anything else that did what we needed. And once I had something working it was often easier to maintain it than to port it to some other library.
In the Common Lisp community, we accept the writing of one's own httpd as a rite of passage. Don't fight it: RFC2616 is hard to resist.
Mmm, my first real project in Scheme was actually an HTML parser, but the spirit of it is the same. There's something enticing about an annoyingly complex spec that lets a beautiful language like Lisp shine.
But yes, I'm sure that Twisted wasn't in the shape it needed to be when FF started building this, and they've built up an alternative. That's never a bad thing.
Otherwise, there'd just be Apache... no nginx, no lighttpd, no fun!
But yes, I'm sure that Twisted wasn't in the shape it needed to be when FF started building this, and they've built up an alternative. That's never a bad thing.
Otherwise, there'd just be Apache... no nginx, no lighttpd, no fun!
It would be really nice if somebody could write a breakdown of these webservers for the uninitiated like myself. I'd like to know when to use which tool.
I started with a single hunchentoot instance running on SBCL on Win32 and I still hack on it about 16 hours a day. I deploy on a 30-instance hunchentoot farm with 4 lighttpds sittin on front of them.
Hunchentoot could either be a single-instance, one thread per request simpleton, or, given its flexible and well thoughtout architecture, an httpd development framework. The requests per second could also vary; the stock configuration is about 80r/s, but I am squeezing well over 2kr/s out of it. I have been tweaking it for the last 7 months or so and it just keeps getting better.
Hunchentoot could either be a single-instance, one thread per request simpleton, or, given its flexible and well thoughtout architecture, an httpd development framework. The requests per second could also vary; the stock configuration is about 80r/s, but I am squeezing well over 2kr/s out of it. I have been tweaking it for the last 7 months or so and it just keeps getting better.
You don't see django folks freaking out about how tornado rewrote URL routing and wrote a new template engine (they even take some shots at django's restrictions in templating). There is probably as much overlap with django as there is with twisted, yet all the "controversy" is coming from the twisted people.
What is wrong with choice?
What is wrong with choice?
Nothing is wrong!
As you said there is probably more overlap with django than with twisted, tornado is a full fledged framework that happens to a have (thanks god) a new async web server, and I really like the self contained nature of tornado (and the no twisted dependency), if you do want twisted there are orbited/cometd already!
All this situation is getting pretty silly IMHO, opensource is about freedom to choice, they evaluated twisted and decided it was not the right path, everyone should accept this and get along. Before twisted there was async, before django there was zope, before python there was another language, before git there was svn, before svn there was cvs and so on... that's how opensource works, competition is the path to better things and I LOVE THIS!
As you said there is probably more overlap with django than with twisted, tornado is a full fledged framework that happens to a have (thanks god) a new async web server, and I really like the self contained nature of tornado (and the no twisted dependency), if you do want twisted there are orbited/cometd already!
All this situation is getting pretty silly IMHO, opensource is about freedom to choice, they evaluated twisted and decided it was not the right path, everyone should accept this and get along. Before twisted there was async, before django there was zope, before python there was another language, before git there was svn, before svn there was cvs and so on... that's how opensource works, competition is the path to better things and I LOVE THIS!
Am I mis-reading that graph or is it the case that twisted.web handles half the concurrent connections that tornado does /and/ more slowly? Is it possible that twisted performing "almost as well" as tornado might be exaggerated?
Agreed, Tornado seems to handle 100% more connections and at 50 connections offers a 15% performance improvement.
It's interesting how quickly people started talking about this as being a "Facebook" thing vs. a "FriendFeed" thing.
I think this is a good thing, and it is a good thing seeing lots of energy poured into each of these approaches.
I wonder if the Zen of python "There is only one-- and preferably only one --obvious way to do it" might be influencing the community's feelings about another web server in python.
I wonder if the Zen of python "There is only one-- and preferably only one --obvious way to do it" might be influencing the community's feelings about another web server in python.
[deleted]
Bullsh*t.
You can argue all you like about whether they made good technical decisions but be angry that they released some code for free is just odd.
Sometimes people re-write already existing things, sometimes it's a good idea, sometimes it's a bad idea, sometimes it seems like a good idea at the time. Sometimes it works out quite well so it doesn't matter either way :p