Simpler long polling with Django and gevent(prg10001.blogspot.com)
prg10001.blogspot.com
Simpler long polling with Django and gevent
http://prg10001.blogspot.com/2009/09/simpler-long-polling-with-django-and.html
4 comments
That issue (accessing databases without blocking) is why I've recently started getting interested in DBSlayer from the NYTimes:
http://dbslayer.org/projects/dbslayer
It's a proxy that lets you make SQL queries against your database over HTTP. Since all of the event based web frameworks (Twisted, gevent, Tornado etc) allow you to make non-blocking HTTP requests, using DBSlayer could instantly resolve the blocking database query problem.
To my knowledge no one has written a Django ORM backend for DBSlayer yet, but it would be a fun hacking project.
http://dbslayer.org/projects/dbslayer
It's a proxy that lets you make SQL queries against your database over HTTP. Since all of the event based web frameworks (Twisted, gevent, Tornado etc) allow you to make non-blocking HTTP requests, using DBSlayer could instantly resolve the blocking database query problem.
To my knowledge no one has written a Django ORM backend for DBSlayer yet, but it would be a fun hacking project.
Thanks!
Your concern is valid though, if the database module is a wrapper around C library which uses blocking sockets, each database call will block the whole interpreter, not just a particular greenlet.
This is not specific to gevent thought. Tornado and Twisted face the same problem and Twisted has a threadpool to cope with that. I guess I would have to implement something similar for gevent.
However, if the database module was a pure Python or used libevent underneath then it would integrate with gevent seamlessly. Alas, it's rarely the case for database modules (?)
Your concern is valid though, if the database module is a wrapper around C library which uses blocking sockets, each database call will block the whole interpreter, not just a particular greenlet.
This is not specific to gevent thought. Tornado and Twisted face the same problem and Twisted has a threadpool to cope with that. I guess I would have to implement something similar for gevent.
However, if the database module was a pure Python or used libevent underneath then it would integrate with gevent seamlessly. Alas, it's rarely the case for database modules (?)
MySQL Connector/Python is a pure-Python implementation of the MySQL client protocol - I presume it uses sockets, in which case the gevent monkey-patching might get it to work transparently.
https://launchpad.net/myconnpy
https://launchpad.net/myconnpy
My big concern with this framework though would whether it could cope with accessing a database or similar datastore without blocking. My guess is that it currently doesn't.