Django's future, and Python 3(djangoproject.com)
djangoproject.com
Django's future, and Python 3
https://www.djangoproject.com/weblog/2012/mar/13/py3k/
10 comments
Just a useless datapoint, but gevent is far more important to my team. I think that is the only thing holding us back right now.
Do you know of any other large hurdles preventing the adoption of Python 3?
It may sound strange, but we're the other way around. With Python 3 being the only future of Python, the inability of something like Django to support Python 3 is, for us, a hurdle to adopting Django, not Python 3.
This has retarded any progress of mine toward learning Python. Every so often the urge strikes me, and then I remember that I want to learn Python 3, and then I remember that means no Django, and I tend to go and learn something more fun and less practical...
> learning Python
By any means, if you feel the urge, you really should help yourself and scratch that itch.
I understand that for someone new to python (hence lacking the knowledge to get what's different between 2 and 3) you might feel uneasy about it, but really you should not.
Just learn py2.7 (+ django if that's you want) now, and bulletproof your code by using u"", b"", .encode and .decode at the edge when needed, also 123L in your code, together with from __future__ import division, print_function, and whatnot. The remainder is mostly syntax (like except) or namespace (like http.server) changes and will get caught and thus easy to fix when you try to run in py3 (contrary to if you don't use u""/b"" as values can "move" through code and blow up in a completely unrelated area). I might be forgetting something, but that's the gist of it.
As for learning, really you will have zero trouble acclimating yourself to any breaking change py3 introduces. The trouble only lies in existing, 'legacy' code, not for the one you write from scratch straight in py2.7, and especially if you aim for compatibility with the provided forward-compatibility features of python.
By any means, if you feel the urge, you really should help yourself and scratch that itch.
I understand that for someone new to python (hence lacking the knowledge to get what's different between 2 and 3) you might feel uneasy about it, but really you should not.
Just learn py2.7 (+ django if that's you want) now, and bulletproof your code by using u"", b"", .encode and .decode at the edge when needed, also 123L in your code, together with from __future__ import division, print_function, and whatnot. The remainder is mostly syntax (like except) or namespace (like http.server) changes and will get caught and thus easy to fix when you try to run in py3 (contrary to if you don't use u""/b"" as values can "move" through code and blow up in a completely unrelated area). I might be forgetting something, but that's the gist of it.
As for learning, really you will have zero trouble acclimating yourself to any breaking change py3 introduces. The trouble only lies in existing, 'legacy' code, not for the one you write from scratch straight in py2.7, and especially if you aim for compatibility with the provided forward-compatibility features of python.
Great post. A point I want to emphasize is that learning py2.7 is learning Python 3, if you are careful to pay attention to the recommended/pythonic way of doing things.
For the most part Python 3 removes syntax and functionality that was long-since depreciated or at least strongly advised against.
For the most part Python 3 removes syntax and functionality that was long-since depreciated or at least strongly advised against.
Perhaps, but the lack of bytestrings seems to be what bites everyone. It's the only "deep" change and it affects anyone who wants to slice raw bytes (which includes most web frameworks).
Python 3 has bytestrings. They look like this:
b'foo'
That's interesting and informative, thanks.
The list you've given looks like a good start - is there a definitive list you'd recommend anywhere?
The list you've given looks like a good start - is there a definitive list you'd recommend anywhere?
There's the official "What's New in Python 3.x" release notes [0][1][2] and some IBM articles [3][4].
[0] http://docs.python.org/release/3.2.1/whatsnew/3.0.html
[1] http://docs.python.org/release/3.2.1/whatsnew/3.1.html
[2] http://docs.python.org/release/3.2.1/whatsnew/3.2.html
[3] http://www.ibm.com/developerworks/linux/library/l-python3-1/
[4] http://www.ibm.com/developerworks/linux/library/l-python3-2/
[0] http://docs.python.org/release/3.2.1/whatsnew/3.0.html
[1] http://docs.python.org/release/3.2.1/whatsnew/3.1.html
[2] http://docs.python.org/release/3.2.1/whatsnew/3.2.html
[3] http://www.ibm.com/developerworks/linux/library/l-python3-1/
[4] http://www.ibm.com/developerworks/linux/library/l-python3-2/
Unfortunately all the hooplah about Python 2/3 porting has created an image, reflected in your posting, which is simply not true. To a large extent Python 3 is just a superset of the functionality of Python 2.x, adding such features as keyword-only arguments and exception chaining. This is typical for Python: each new release typically includes a handful of new community-vetted syntax features. What's unique about Python 3.0 is that this one, exceptional time they allowed themselves to go back and eliminate deprecated syntax and functionality, and to tweak a few aspects of the language (namely Unicode strings) in subtle, but fundamental ways.
It's been a long, drawn-out process in part because everyone uses Unicode strings (and if you're not, you should be). Python 2.x let you be fairly lax about unicode vs. legacy/byte strings. Python 3 forces you to clean up your act and be explicit. Other than that the two languages are -basically- the same. This isn't like C vs. C++ (which share an unhealthy fetish for curly braces but are quite different under the surface), but rather more like K&R C vs ANSI C, or C++03 vs C++0x.
In other words if you're waiting for Django/SQLAlchemy/whatever to transition, don't bother. Learn Python 2.7 today, use it, love it, and transition to Python 3 only when all your dependencies have made the switch. You'll then have to change a few minor things like "raise Exception, reason" to "raise Exception(reason)", but I'm sure you'll adapt just fine.
It's been a long, drawn-out process in part because everyone uses Unicode strings (and if you're not, you should be). Python 2.x let you be fairly lax about unicode vs. legacy/byte strings. Python 3 forces you to clean up your act and be explicit. Other than that the two languages are -basically- the same. This isn't like C vs. C++ (which share an unhealthy fetish for curly braces but are quite different under the surface), but rather more like K&R C vs ANSI C, or C++03 vs C++0x.
In other words if you're waiting for Django/SQLAlchemy/whatever to transition, don't bother. Learn Python 2.7 today, use it, love it, and transition to Python 3 only when all your dependencies have made the switch. You'll then have to change a few minor things like "raise Exception, reason" to "raise Exception(reason)", but I'm sure you'll adapt just fine.
A couple years ago I decided to learn Python. I took a quick look, saw Python 3 was the "latest", and started working on my app, using http://docs.python.org/py3k/ as my guide.
It wasn't until a month later when I went to deploy that I discovered Python 3 wasn't what everyone was using. It was a small app, so it only took me a a day to port it to 2.6.
My point is go ahead and learn Python 3. The hurdles of back-porting are pretty minor when you don't have any version specific habits.
It wasn't until a month later when I went to deploy that I discovered Python 3 wasn't what everyone was using. It was a small app, so it only took me a a day to port it to 2.6.
My point is go ahead and learn Python 3. The hurdles of back-porting are pretty minor when you don't have any version specific habits.
It may sound the same, but I had that same feeling towards Python. I eventually dipped my toe into Python 3, but my main desire was to use Django for a work project and naturally we ditched it for ASP.NET (a decision I don't regret as ASP.NET is a great framework).
Flask (http://flask.pocoo.org/docs/foreword/) and NLTK (http://www.nltk.org/faq)
matplotlib is the defacto standard for plotting data in Python and could definitely be considered a hurdle. I haven't followed to closely but it looks like support for 3.x is coming along nicely[1].
GUI toolkits like wxPython and PyQT/PySide supporting Python3 will also be important.
I'm not sure how outdated this is, but there's a list of most wanted Py3k packages available here: http://www.python.org/3kpoll
1. https://github.com/matplotlib/matplotlib-py3
GUI toolkits like wxPython and PyQT/PySide supporting Python3 will also be important.
I'm not sure how outdated this is, but there's a list of most wanted Py3k packages available here: http://www.python.org/3kpoll
1. https://github.com/matplotlib/matplotlib-py3
Well, it's not completely backwards compatible.
PIL isn’t ported yet, I don’t think.
PIL is also somewhat labyrinthine, so porting may be particularly slow. I've been using ctypes to load GraphicsMagick (JAI on Jython), which is easier to start and less of a challenge where PyPy is concerned:
https://github.com/acdha/NativeImaging
There's also a very new project started here at PyCon to build a completely native Python imaging library:
https://github.com/ojii/pymaging
Obviously it's nowhere near as fast, although it'll be interesting to see how PyPy performs
https://github.com/acdha/NativeImaging
There's also a very new project started here at PyCon to build a completely native Python imaging library:
https://github.com/ojii/pymaging
Obviously it's nowhere near as fast, although it'll be interesting to see how PyPy performs
> pixels: A list of arrays (array.array). Each array in the list is a line in the image. Each array is the actual image data for that line in the image.
This seems like a silly thing to do. It would make for much more readable code to use numpy arrays. But regardless, the pixel data should be one single array of data.
It looks to me like the guy working on this hasn’t spent tremendously much time thinking about good image APIs. Is there any document that describes the design goals and process?
This seems like a silly thing to do. It would make for much more readable code to use numpy arrays. But regardless, the pixel data should be one single array of data.
It looks to me like the guy working on this hasn’t spent tremendously much time thinking about good image APIs. Is there any document that describes the design goals and process?
Numpy support in PyPy is being worked on. Meanwhile arrays are well-tested and jit-accelerated.
array.array is fine, I guess, but using a list of array.array objects is inelegant and inflexible. It’s much better to put all your image data in one big array, keep track of the various stride lengths to go down rows or across pixels, and then implement proper multi-dimensional indexing.
I suppose it doesn’t really matter. This library is also missing all the other flexibility you’d want in a general imaging toolkit – e.g. the assumption of 8 bit/channel RGB is baked into everything, with no notion of color space or gamma encoding (and the JPEG decoder doesn’t seem to have any mention of color spaces, meaning that many JPEG images will be wrongly interpreted). It’s not usable for any but the most basic image processing tasks.
The JPEG decoder might make an okay benchmark for pypy performance, though it’s a straight port from C++, so it’s probably not the style of python code that pypy is designed to be fast on.
I suppose it doesn’t really matter. This library is also missing all the other flexibility you’d want in a general imaging toolkit – e.g. the assumption of 8 bit/channel RGB is baked into everything, with no notion of color space or gamma encoding (and the JPEG decoder doesn’t seem to have any mention of color spaces, meaning that many JPEG images will be wrongly interpreted). It’s not usable for any but the most basic image processing tasks.
The JPEG decoder might make an okay benchmark for pypy performance, though it’s a straight port from C++, so it’s probably not the style of python code that pypy is designed to be fast on.
I believe Jonas is approaching it from his position as a web developer and Django author who's spent a LOT of time dealing with requests from people who've had trouble with PIL and generally only use 8-bit RGB.
I'd be reluctant to condemn a project which is a few days old. Having spent time wrestling with optimizing the horror which is the PIL code, I'd definitely suggest that having a pure Python implementation will make it much easier for people to experiment with API and performance improvements.
I'd be reluctant to condemn a project which is a few days old. Having spent time wrestling with optimizing the horror which is the PIL code, I'd definitely suggest that having a pure Python implementation will make it much easier for people to experiment with API and performance improvements.
I think it's crazy that they are only now talking about dropping support for Python 2.4 which came out in 2004 and has been replaced by Python 2.5 in 2006.
So only now they are moving from depending on 8 years old technology to depending on 6 years old technology.
The reason for this is probably the proliferation of really old versions in the default installation of still supported OSes like RHEL. Worse: Due to the fact that these distros largely depend on the Python installation they came with, it's likely that admins are reluctant to touch anything python related on the system.
Compare this to ruby/rails where the common practice is to treat system ruby as a hands-off thing: Usually, the deployment payload just includes its own ruby (using rvm or rbenv). This allows for much less conservative minimum requirements.
Being forced to constantly target way outdated software during development is really annoying and kills motivation as you are always reminded that if only you could update, you would be spared re-implementing feature x or you could use much more expressive code while implementing feature y.
And don't get me started on PH-4.x is good enough-P.
So only now they are moving from depending on 8 years old technology to depending on 6 years old technology.
The reason for this is probably the proliferation of really old versions in the default installation of still supported OSes like RHEL. Worse: Due to the fact that these distros largely depend on the Python installation they came with, it's likely that admins are reluctant to touch anything python related on the system.
Compare this to ruby/rails where the common practice is to treat system ruby as a hands-off thing: Usually, the deployment payload just includes its own ruby (using rvm or rbenv). This allows for much less conservative minimum requirements.
Being forced to constantly target way outdated software during development is really annoying and kills motivation as you are always reminded that if only you could update, you would be spared re-implementing feature x or you could use much more expressive code while implementing feature y.
And don't get me started on PH-4.x is good enough-P.
It is perfectly possible to use a deployment-specific version of Python, just like it is with Ruby. But there are organizations where using old technology is (unfortunately) the status quo. This means Django gives you the possibility of still running it in these environments. Hell, there's loads of companies using older versions of Windows and IE. Is it optimal? No, but that's something bureaucracy tends to do.
I am not trying to say Django is a better framework better for making this decision, but it is a decision that makes sense in the Python context, which is careful about breaking backwards compatibility. There's no reason to stop a developer who has that choice to work with Python 2.7 though (this is actually recommended in the FAQ). There is no need to touch the system's Python to do that.
I am not trying to say Django is a better framework better for making this decision, but it is a decision that makes sense in the Python context, which is careful about breaking backwards compatibility. There's no reason to stop a developer who has that choice to work with Python 2.7 though (this is actually recommended in the FAQ). There is no need to touch the system's Python to do that.
Django's conservatism in this regard doesn't really impact on application developers building on top of Django.
Usual practice is to use virtualenv which allows for an up to date python on OSes with an ancient system python.
Usual practice is to use virtualenv which allows for an up to date python on OSes with an ancient system python.
Python 2.4 seem to still be a version you can find on "recent" system, like CentOS 5. So it made sense to be compatible with it, don't you think ? And RoR was conservatively still working with ruby 1.8 too...
I know not of any equivalent of rbenv for python, but I've not looked after, so here we are.
I wonder why more people don't realize this. Fact is, there are those who have no other option at their day jobs then to deploy to "ancient" systems for one reason or another, developers often do not have a say in the matter. It's good that these useful frameworks are willing to support the real world for as long as they can.
You can do this in Python with virtualenv. Which makes you wonder why Django doesn't use say "Always use virtualenv"?!
Historical reasons mostly.
Virtualenv is now on the ascendancy but it wasn't always like this and there were many people who had workflows based on different tools including zc.buildout (for example). Virtualenv has gone from an up-and-comer to a top-dog over the last 4.5 years.
Django core devs have tried to be hands-off on dev-op issues perhaps to a fault.
Virtualenv is now on the ascendancy but it wasn't always like this and there were many people who had workflows based on different tools including zc.buildout (for example). Virtualenv has gone from an up-and-comer to a top-dog over the last 4.5 years.
Django core devs have tried to be hands-off on dev-op issues perhaps to a fault.
Django team: "We're planning to shift to the 4 year old Python 3 over the next 2 years"
Rails team: "New Ruby alpha? Deprecate everything!"
Rails team: "New Ruby alpha? Deprecate everything!"
I think this is a bit disingenuous. Ruby 1.9.0 was released Christmas 2007, 1.9.1 was released in January 2009, and 1.9.2 was released August 2010. Ruby 1.9.2 was the first release that most Rubyists considered "stable".
For the uninitiated, Ruby's tiny version is the equivalent of Python's minor version (Ruby 1.9.2=>1.9.3 ~ Python 3.2=>3.3).
Rails 2.3 supported almost the entire Ruby 1.8 series (except for one release, 1.8.3 I think, that had a serious bug). Rails 2.3 had experimental support for Ruby 1.9.1 when it shipped, but most people used Ruby 1.8.6 or 1.8.7, as Ruby 1.9 was still somewhat unstable at that point.
Rails 3.0 (which also shipped in August 2010) dropped support for Ruby 1.8 versions before Ruby 1.8.7, and shipped with stable support for Ruby 1.9.2. Ruby 1.8.7 shipped in May 2008, and was very compatible with Ruby 1.8.6. We (the Rails team) continued full support for Rails 2.3, which supported Ruby 1.8.6, until August 2011, when Rails 3.1 was released.
We have announced that Rails 4.0, which we still have not shipped, will drop support for Ruby 1.8.7. This means that we have dropped support for one version of Ruby (we will still support 1.9.2) in more than 2 years. That hardly qualifies as "New Ruby alpha? Deprecate everything!"
Short timeline:
- August 2006: Ruby 1.8.5
- March 2007: Ruby 1.8.6
- December 2007: Ruby 1.9.0 / Rails 2.0
- May 2008: Ruby 1.8.7
- June 2008: Rails 2.1
- November 2008: Rails 2.2
- January 2009: Ruby 1.9.1
- March 2009: Rails 2.3
- August 2010: Ruby 1.9.2 / Rails 3.0
- August 2011: Rails 3.1
- October 2011: Ruby 1.9.3
- January 2012: Rails 3.2
Releases of Rails 2.x all supported Ruby 1.8.2 and above. Releases of Rails 3 have supported Ruby 1.8.7 and Ruby 1.9.2 and above. The only change in Ruby support since Rails 2.0 was requiring at least Ruby 1.8.7 in Rails 3.0. There is an announced change for Rails 4.0, but Rails 4 has not been released yet, even in betas.
Rails is moving more quickly at adopting Ruby 1.9 than Django is at adopting Python 3, but that is because it was possible, from the get-go to support both Ruby 1.8 and Ruby 1.9 in a single codebase. That was not possible with the move from Python 2 to Python 3.
The typical argument I've heard about the difference is that Ruby 1.8 to 1.9 was a relatively smaller release. I think this underestimates the work that the Ruby team put into compatibility while developing Ruby 1.9, especially given the large encoding-related change made to Strings in Ruby 1.9 (Ruby 1.9 Strings are encoding aware, while Ruby 1.8 Strings are semantically simple byte Arrays).
For the uninitiated, Ruby's tiny version is the equivalent of Python's minor version (Ruby 1.9.2=>1.9.3 ~ Python 3.2=>3.3).
Rails 2.3 supported almost the entire Ruby 1.8 series (except for one release, 1.8.3 I think, that had a serious bug). Rails 2.3 had experimental support for Ruby 1.9.1 when it shipped, but most people used Ruby 1.8.6 or 1.8.7, as Ruby 1.9 was still somewhat unstable at that point.
Rails 3.0 (which also shipped in August 2010) dropped support for Ruby 1.8 versions before Ruby 1.8.7, and shipped with stable support for Ruby 1.9.2. Ruby 1.8.7 shipped in May 2008, and was very compatible with Ruby 1.8.6. We (the Rails team) continued full support for Rails 2.3, which supported Ruby 1.8.6, until August 2011, when Rails 3.1 was released.
We have announced that Rails 4.0, which we still have not shipped, will drop support for Ruby 1.8.7. This means that we have dropped support for one version of Ruby (we will still support 1.9.2) in more than 2 years. That hardly qualifies as "New Ruby alpha? Deprecate everything!"
Short timeline:
- August 2006: Ruby 1.8.5
- March 2007: Ruby 1.8.6
- December 2007: Ruby 1.9.0 / Rails 2.0
- May 2008: Ruby 1.8.7
- June 2008: Rails 2.1
- November 2008: Rails 2.2
- January 2009: Ruby 1.9.1
- March 2009: Rails 2.3
- August 2010: Ruby 1.9.2 / Rails 3.0
- August 2011: Rails 3.1
- October 2011: Ruby 1.9.3
- January 2012: Rails 3.2
Releases of Rails 2.x all supported Ruby 1.8.2 and above. Releases of Rails 3 have supported Ruby 1.8.7 and Ruby 1.9.2 and above. The only change in Ruby support since Rails 2.0 was requiring at least Ruby 1.8.7 in Rails 3.0. There is an announced change for Rails 4.0, but Rails 4 has not been released yet, even in betas.
Rails is moving more quickly at adopting Ruby 1.9 than Django is at adopting Python 3, but that is because it was possible, from the get-go to support both Ruby 1.8 and Ruby 1.9 in a single codebase. That was not possible with the move from Python 2 to Python 3.
The typical argument I've heard about the difference is that Ruby 1.8 to 1.9 was a relatively smaller release. I think this underestimates the work that the Ruby team put into compatibility while developing Ruby 1.9, especially given the large encoding-related change made to Strings in Ruby 1.9 (Ruby 1.9 Strings are encoding aware, while Ruby 1.8 Strings are semantically simple byte Arrays).
> we will still support 1.9.2 [in Rails 4.0]
How can the Core team claim to support 1.9.2 in Rails 4.0 when the minimum version of Ruby required to run the test suite is 1.9.3?
How can the Core team claim to support 1.9.2 in Rails 4.0 when the minimum version of Ruby required to run the test suite is 1.9.3?
Either it's a bug or I missed something. It could be that there were fundamental bugs in Ruby 1.9.2.
Again, Rails 4.0 is not out yet, even in betas. We're currently in the planning stages, with a lot of movement on master, so master is in a state of flux at the moment that doesn't really reflect what a release will look like.
Again, Rails 4.0 is not out yet, even in betas. We're currently in the planning stages, with a lot of movement on master, so master is in a state of flux at the moment that doesn't really reflect what a release will look like.
DHH says Rails 4.0 will require Ruby 1.9.3 and higher
http://weblog.rubyonrails.org/2012/1/20/rails-3-2-0-faster-d...
"The master git branch for Rails is now targeting Rails 4.0, which will require Ruby 1.9.3 and above."
"The master git branch for Rails is now targeting Rails 4.0, which will require Ruby 1.9.3 and above."
Having investigated this a bit, it seems like there are a number of bugs in Ruby 1.9.2 (including some crashers) that make us want to steer people to 1.9.3.
All Rails core team members have running apps on the latest version of Rails, and Ruby 1.9.3 was the first version that everyone felt comfortable shipping to production with. So Rails 4 removes support for Ruby 1.8.7, and requires the first "stable" version of Ruby 1.9, based on our experience bringing apps to production.
All Rails core team members have running apps on the latest version of Rails, and Ruby 1.9.3 was the first version that everyone felt comfortable shipping to production with. So Rails 4 removes support for Ruby 1.8.7, and requires the first "stable" version of Ruby 1.9, based on our experience bringing apps to production.
Now which would you bet your business on?
It'd be nice if things could move a little faster. But the alternative is wild-wild-west.
It'd be nice if things could move a little faster. But the alternative is wild-wild-west.
>Now which would you bet your business on?
Scala/Play! I was fed up of waiting.
Scala/Play! I was fed up of waiting.
In both worlds high-stakes businesses are going to stick to the "stable" release while personal projects explore the new waters. Django's failure to make the shift has resulted in all those personal projects not making the shift.
Django, from my point of view, seems to be approaching this from the wrong direction. They are waiting for the community to shift to Python 3, but at the same time the community is waiting for Django to shift.
Django, from my point of view, seems to be approaching this from the wrong direction. They are waiting for the community to shift to Python 3, but at the same time the community is waiting for Django to shift.
Er, no, we're not "waiting for the community to shift".
When Django 1.0 was released, we supported minimum Python 2.3. Dropping 2.3 and then 2.4 were major logistical hurdles; they were incredibly widely deployed, and were the stock Python for many platforms with long-term support. So it was going to take a while, no matter what, for us to sensibly deprecate those Python versions without just throwing a bunch of our users under a bus. And, of course, deprecating everything older than 2.6 is a prerequisite for starting to actually adopt 3.x.
Meanwhile, the first few releases in the Python 3 series weren't really meant for widespread adoption, ever. They were there so people could start getting a taste of Python 3 and working out a migration process while shaking out remaining issues for Python itself.
We are now at a confluence of two very happy things:
1. We've dropped our support for the older Python 2.x releases, without just screwing over a bunch of people, and by year's end we'll be a minimum-2.6 project, and
2. Python 3.2 (the current 3.x) is really quite a nice release, and Python 3.3 (later this year) looks like it'll really put the spit and polish on the Python 3.x series, meaning it's a very attractive time to be targeting Python 3.
This is the first time we've realistically been in a position to actually make the jump, and we're taking that opportunity.
When Django 1.0 was released, we supported minimum Python 2.3. Dropping 2.3 and then 2.4 were major logistical hurdles; they were incredibly widely deployed, and were the stock Python for many platforms with long-term support. So it was going to take a while, no matter what, for us to sensibly deprecate those Python versions without just throwing a bunch of our users under a bus. And, of course, deprecating everything older than 2.6 is a prerequisite for starting to actually adopt 3.x.
Meanwhile, the first few releases in the Python 3 series weren't really meant for widespread adoption, ever. They were there so people could start getting a taste of Python 3 and working out a migration process while shaking out remaining issues for Python itself.
We are now at a confluence of two very happy things:
1. We've dropped our support for the older Python 2.x releases, without just screwing over a bunch of people, and by year's end we'll be a minimum-2.6 project, and
2. Python 3.2 (the current 3.x) is really quite a nice release, and Python 3.3 (later this year) looks like it'll really put the spit and polish on the Python 3.x series, meaning it's a very attractive time to be targeting Python 3.
This is the first time we've realistically been in a position to actually make the jump, and we're taking that opportunity.
Fair enough, the impression I got from the above link was that Django was waiting for the community.
I'll hold back on commenting now because my insight into the Python community is limited to what I hear from friends who develop on Python. Though something I've noticed is that many of them are making the switch to Node.
I'll hold back on commenting now because my insight into the Python community is limited to what I hear from friends who develop on Python. Though something I've noticed is that many of them are making the switch to Node.
there wasn't even an agreed upon spec for WSGI on python 3 until last year IIRC. That needed to happen first.
I think a lot of people who are really interested in shifting to Python 3 for those experimental projects are just using different web frameworks.
The real problem is that Django isn't the only problem. I've started cataloging the packages that are supporting Python 3 in my project, and there are still several that are critical to it that haven't made the transition. My web framework is the least of my concerns in that regard (considering it already supports Python 3).
The real problem is that Django isn't the only problem. I've started cataloging the packages that are supporting Python 3 in my project, and there are still several that are critical to it that haven't made the transition. My web framework is the least of my concerns in that regard (considering it already supports Python 3).
It depends on your business, right?
I think its important for frameworks like Rails / Django to always support the latest and greatest and at the same time give people time to upgrade for example how the rails team have done it now which is in rails 4 move to 1.9.x exclusively but in rails 3.2 still support ruby 1.8.7
developers have choice. when they're starting new app / new project they can opt for the older version or if they're feeling lucky adopt the new stack.
the important thing is developers should have a choice, because i firmly believe more choice leads to more innovation.
its not like when rails 4.0 come out they're going to force everyone to upgrade, you can upgrade at your own pace.
developers have choice. when they're starting new app / new project they can opt for the older version or if they're feeling lucky adopt the new stack.
the important thing is developers should have a choice, because i firmly believe more choice leads to more innovation.
its not like when rails 4.0 come out they're going to force everyone to upgrade, you can upgrade at your own pace.
What I like about this comment, is that it has pissed off both sides of the story.
One of the things I love about Rails is because it IS quick to adopt the latest (stable) Ruby, and doesn't hang around too long supporting old versions which really, not too many of us are (or should be?) using anyway. I have found the Ruby community are generally eager to use the most up to date Rubies ...some are even on Ruby 3.0 ;-)
I hate titles "Framework A is doing this, why framework B isn't doing this?"
Use what seems best to You. Rails always tend to have faster movement with Ruby support, but to be honest having at least 1 year is enough to redeploy app with new Rails & Ruby. If not then you're doing it wrong.
Use what seems best to You. Rails always tend to have faster movement with Ruby support, but to be honest having at least 1 year is enough to redeploy app with new Rails & Ruby. If not then you're doing it wrong.
> we hope to be targeting up to Python 3.3
All major projects seem to be adding support up to the latest Python 3 version, not the 4 year old 3.0 one.
All major projects seem to be adding support up to the latest Python 3 version, not the 4 year old 3.0 one.
Here is a Python3 packages listing on PyPI: http://pypi.python.org/pypi?:action=browse&c=533&sho...
There's some interesting stuff for web developers in there: Pyramid, Bottle, SQLAlchemy, pymongo3, Mako, Jinja2, nose, etc. It seems that the whole web development stack is pretty much covered. Although it's not listed there, Tornado should also run on Python3. I considered starting my new project with Python3 but eventually sticked with Django because of the time-saving reusable apps available, but it's interesting to note that building web apps with Python3 should already be feasible.
Maybe I spent way too long at PyCon but I'm getting very weary of hearing about Python 3. Why is anyone surprised, or cheered, or dismayed, or excited, that one of the flagship Python projects will eventually be moving to Python 3 (experimental!) compatibility?
This is a sincere question by the way, just tempered by post-PyCon weariness.
This is a sincere question by the way, just tempered by post-PyCon weariness.
If you think you're tired about hearing about Python 3 try sitting in my seat! I got asked "when's Django moving to Python 3?" about thirty times a day at PyCon, and I think the other core devs did, too. We realized that although we had a plan in mind we hadn't communicated it well. Hence the blog post.
Sorry to bore you. Back to your regularly scheduled hacking.
Sorry to bore you. Back to your regularly scheduled hacking.
> I got asked "when's Django moving to Python 3?" about thirty times a day at PyCon, and I think the other core devs did, too.
Get custom T-shirts printed? ;)
Get custom T-shirts printed? ;)
Wonder how many times you got asked how long you are going to hang on to Python 2.5 or even whether you/Django core devs get asked why can't they have Python 2.4 support.
Not that I care that much for environments stuck in 2.4 land, its last update was 7 years ago.
Not that I care that much for environments stuck in 2.4 land, its last update was 7 years ago.
It's not that this post bored me, it's the "This is amazing/great/terrible/stupid" reaction to the news things are getting ported. Progress, time marches on, etc.
Thanks for the post, the road map should be a lot more clearer for users.
Django's lack of Python 3 support is big reason Python 3 adoption is slow, at least in the web dev faction of Python's community.
Why get comfy with Python 3 when you must remain attuned to the conventions and quirks of 2.x in order to use Python's most popular web framework? Folks keep talking about Python 3 because they really want to use it, but have reservations about adopting 3 when 3 isn't supported by useful frameworks.
Once Django supports Python 3, that reservation vanishes. Python 3 will be adopted by more people and get support from more frameworks.
tl:dr; Django's popularity holds dev community captive to Python 2.x, slows adoption of Python 3. Django team announces Python 3 support not in distant future but near future. Folks get excited. "About damn time."
Why get comfy with Python 3 when you must remain attuned to the conventions and quirks of 2.x in order to use Python's most popular web framework? Folks keep talking about Python 3 because they really want to use it, but have reservations about adopting 3 when 3 isn't supported by useful frameworks.
Once Django supports Python 3, that reservation vanishes. Python 3 will be adopted by more people and get support from more frameworks.
tl:dr; Django's popularity holds dev community captive to Python 2.x, slows adoption of Python 3. Django team announces Python 3 support not in distant future but near future. Folks get excited. "About damn time."
Python != Web frameworks
You're severely underestimating the impact that the scientific computing community has over Python 3's slow adoption.
You're severely underestimating the impact that the scientific computing community has over Python 3's slow adoption.
I don't understand your statement. Numpy has been ported on python 3 more than 18 months ago, and scipy followed quickly afterwards. Our download stats show rather little interest for python 3, BTW.
The numpy and scipy core may have been ported, but many of the Scikits (add on libraries to scipy), third party libraries and surrounding tools haven't. So most people will still go with python2 just in case they might need to use one of those tools or libraries later down the road.
Personally the only reason that I am not using Python 3 right now is that Django doesn't support it, as that is the main library that I am supporting and using everyday.
Other libraries I use are smaller, can be replaced, or I could rewrite them.
I am quite sure that I am not in a minority.
Other libraries I use are smaller, can be replaced, or I could rewrite them.
I am quite sure that I am not in a minority.
+1
I had started learning Python 3. But my job now is on Django, so I am forced to use Python 2. I know a good number of people who had the same problem.
Discussing Django/Python on HN isn't easy—slightest criticism leads to immediate downvoting, outfaded comments and lowered karma. People either avoid strong opinions or just stay away from Django/Python threads. So, it's very hard to get an good understanding and unbiased opinions on various web framework compared to Django.
I'm sorry for the noob question, but how much additional performance/functionality/flexibility could be gained by moving to Python 3?
Maybe my standards are too low since I am using Django as a vehicle to learn programming and web-development as a hobby right now, but as far as I can tell the Python/Django environment seems to work exceptionally well meeting, and even exceeding, its intended objective (The web framework for perfectionists with deadlines)
It seems a lot of people have this child-like concept of time where a few months is a "long time" and upgrading to the latest and/or greatest thing is so much of a priority that they are regarding it as a "fix". But as far as I can tell nothing is broken by having Django run on 2.x. If a bicycle works really, really well; is replacing the chain a top priority?
Maybe my standards are too low since I am using Django as a vehicle to learn programming and web-development as a hobby right now, but as far as I can tell the Python/Django environment seems to work exceptionally well meeting, and even exceeding, its intended objective (The web framework for perfectionists with deadlines)
It seems a lot of people have this child-like concept of time where a few months is a "long time" and upgrading to the latest and/or greatest thing is so much of a priority that they are regarding it as a "fix". But as far as I can tell nothing is broken by having Django run on 2.x. If a bicycle works really, really well; is replacing the chain a top priority?
> Python 3 is the future of the Python language
... and it always will be.
... and it always will be.
I don't understand these guys (and girls?). The language sounds like Oracle or SAP talking about migrating WalMart to a new inventory management system. If "porting code is relatively straightforward" it would be a good idea to get that port done and let people decide for themselves when to use the new version.
I don't understand what makes phraseology like this useful to anyone but a corporate lawyer: "the Django project is and will remain committed to achieving Python 3 compatibility"
Yes, I know, I'm not supposed to talk like this about work being done by volunteers free of charge. It's of course their right to do whatever they like. My anger is more about the Python 3 migration going off the rails in a way that poses an existential threat to my favorite programming language. Django isn't helping at all to say the least.
I don't understand what makes phraseology like this useful to anyone but a corporate lawyer: "the Django project is and will remain committed to achieving Python 3 compatibility"
Yes, I know, I'm not supposed to talk like this about work being done by volunteers free of charge. It's of course their right to do whatever they like. My anger is more about the Python 3 migration going off the rails in a way that poses an existential threat to my favorite programming language. Django isn't helping at all to say the least.
> If "porting code is relatively straightforward" it would be a good idea to get that port done and let people decide for themselves when to use the new version.
In fact, that's exactly what we're doing. Django's going to support Python 2 for quite some time yet; this move will simply let those who want to switch to 3 do so.
> I don't understand what makes phraseology like this useful to anyone but a corporate lawyer: "the Django project is and will remain committed to achieving Python 3 compatibility"
Because we weren't clear that Python 3 support was in the pipeline some folks in our community started to wonder if we planned to move the Python 3 at all. That's pretty much the point of the post: to clarify that we do indeed plan to port to Python 3 (and when). How would you have worded it differently?
> Yes, I know, I'm not supposed to talk like this about work being done by volunteers free of charge.
That's not true at all: if you have substantive critiques to make, then you should make them. We try incredibly hard to listen to our critics and take away what we can. However, your criticism here is pretty thin and rather personal -- seriously, accusing a bunch of volunteers of being like Oracle or SAP? Low blow, man, low blow. It's hard for me to figure out exactly why you felt compelled to post this criticism, so it's hard to figure out what the action you'd like us to take.
> My anger is more about the Python 3 migration going off the rails in a way that poses an existential threat to my favorite programming language. Django isn't helping at all to say the least.
OK, so here we go -- there is something you'd like to see us doing differently. But what?
In fact, that's exactly what we're doing. Django's going to support Python 2 for quite some time yet; this move will simply let those who want to switch to 3 do so.
> I don't understand what makes phraseology like this useful to anyone but a corporate lawyer: "the Django project is and will remain committed to achieving Python 3 compatibility"
Because we weren't clear that Python 3 support was in the pipeline some folks in our community started to wonder if we planned to move the Python 3 at all. That's pretty much the point of the post: to clarify that we do indeed plan to port to Python 3 (and when). How would you have worded it differently?
> Yes, I know, I'm not supposed to talk like this about work being done by volunteers free of charge.
That's not true at all: if you have substantive critiques to make, then you should make them. We try incredibly hard to listen to our critics and take away what we can. However, your criticism here is pretty thin and rather personal -- seriously, accusing a bunch of volunteers of being like Oracle or SAP? Low blow, man, low blow. It's hard for me to figure out exactly why you felt compelled to post this criticism, so it's hard to figure out what the action you'd like us to take.
> My anger is more about the Python 3 migration going off the rails in a way that poses an existential threat to my favorite programming language. Django isn't helping at all to say the least.
OK, so here we go -- there is something you'd like to see us doing differently. But what?
I thought it was rather obvious what I would like to see you do: Always support the latest version of Python as soon as possible after it is released. Not supporting Python 3 years after it was released isn't just a problem for Django, it is a problem for Python. I'm assuming that Django's developers know that as well and that they (you) care about Python, so that's why I don't understand why this hasn't been fixed a long time ago.
I can totally understand that you don't like my Oracle/SAP comparison and I do apologize for the tone, but there is actually some substance behind it. Oracle and SAP provide core transactional capabilities to very large corporations. These corporations make money based on their scale, not because they are particularly nimble on the IT side.
They're not startups. They rarely do any real greenfield stuff. They don't hire developers who want to be on the bleeding edge. Their IT culture is one of mediocrity and that's why it doesn't matter how many generations/years the software they use lags behind the latest platform version.
I would have thought and hoped that Django is not just for that type of corporation.
But judging by the reactions here you seem to be in sync with the majority of your users, so the problem isn't yours so much as it is mine. I should probably not get agitated about it and just move on.
I can totally understand that you don't like my Oracle/SAP comparison and I do apologize for the tone, but there is actually some substance behind it. Oracle and SAP provide core transactional capabilities to very large corporations. These corporations make money based on their scale, not because they are particularly nimble on the IT side.
They're not startups. They rarely do any real greenfield stuff. They don't hire developers who want to be on the bleeding edge. Their IT culture is one of mediocrity and that's why it doesn't matter how many generations/years the software they use lags behind the latest platform version.
I would have thought and hoped that Django is not just for that type of corporation.
But judging by the reactions here you seem to be in sync with the majority of your users, so the problem isn't yours so much as it is mine. I should probably not get agitated about it and just move on.
I think the problem is that our definitions of "as soon as possible" diverge. To wit:
* Database drivers (psycopg2, MySQLdb, etc.) weren't updated for Python 3 until... last year some time, I think? Django supporting Python 3 is somewhat academic if you can't use a database with it, don't you think?
* The WSGI spec wasn't updated for Python 3 until summer 2011 (July, IIRC). Again, Django supporting Python 3 is pretty academic if you can't run it under a web server, right?
* The previous suggested approach for dual Python 2 and 3 support (using 2to3) proved to be untenable for Django [1], and it wasn't until this last year (or so) that a better approach (single shared source) emerged.
So to my mind, "as soon as possible" was something like July 2011. We had unmet dependencies before then; moving would have been... well, not impossible, but incredibly painful and nearly useless.
I will, however, take full responsibility for not getting Python 3 support shipped between then and how. Our current release workflow [2] got in the way, and about 7-8 months because of it. That's my fault, and if you want to rake me over the coals for those months go right ahead: I deserve it.
[1] Details omitted to be concise, though I'm happy to share them if anyone's interested.
[2] Which we're changing, mostly because of this silly workflow-induced delay.
* Database drivers (psycopg2, MySQLdb, etc.) weren't updated for Python 3 until... last year some time, I think? Django supporting Python 3 is somewhat academic if you can't use a database with it, don't you think?
* The WSGI spec wasn't updated for Python 3 until summer 2011 (July, IIRC). Again, Django supporting Python 3 is pretty academic if you can't run it under a web server, right?
* The previous suggested approach for dual Python 2 and 3 support (using 2to3) proved to be untenable for Django [1], and it wasn't until this last year (or so) that a better approach (single shared source) emerged.
So to my mind, "as soon as possible" was something like July 2011. We had unmet dependencies before then; moving would have been... well, not impossible, but incredibly painful and nearly useless.
I will, however, take full responsibility for not getting Python 3 support shipped between then and how. Our current release workflow [2] got in the way, and about 7-8 months because of it. That's my fault, and if you want to rake me over the coals for those months go right ahead: I deserve it.
[1] Details omitted to be concise, though I'm happy to share them if anyone's interested.
[2] Which we're changing, mostly because of this silly workflow-induced delay.
I'm fully aware that the whole migration disaster isn't primarily Django's fault. But Django is in a position to put pressure on others by simply creating facts and that hasn't happened.
I'm sure you could have found a way to talk to web servers without waiting years for the WSGI spec to be finalized. And if Django had supported Python 3 using py-postgresql, which was released in 2009 (I know Python 3 only), others would have worked a little faster to make their database systems available on Python 3 as well. Upgrading psycopg can't be such an incredibly difficult task either.
The reality is, everyone has basically been waiting for everyone else to make a move and it has resulted in deadlock.
I'm sure you could have found a way to talk to web servers without waiting years for the WSGI spec to be finalized. And if Django had supported Python 3 using py-postgresql, which was released in 2009 (I know Python 3 only), others would have worked a little faster to make their database systems available on Python 3 as well. Upgrading psycopg can't be such an incredibly difficult task either.
The reality is, everyone has basically been waiting for everyone else to make a move and it has resulted in deadlock.
> But Django is in a position to put pressure on others by simply creating facts and that hasn't happened.
How exactly does one create a fact? That would be a neat trick.
> I'm sure you could have found a way to talk to web servers without waiting years for the WSGI spec to be finalized.
Indeed, we could have, and it would have benefited Django at the expense of diversity in the Python web world. It probably would have helped our "market share". But that's not how we roll. I'd rather work on something that the whole community can benefit from, and that means standards and specifications.
> And if Django had supported Python 3 using py-postgresql, which was released in 2009 [...]
That's true - I'd forgotten about that library.
> [...] others would have worked a little faster to make their database systems available on Python 3 as well.
That... I'm not so sure about. Volunteer community, limited by free time, etc. I'm fairly sure most people have worked on this stuff just about as fast as they can.
You do realize, of course, that the criticism here applies to you as well. If you feel this strongly you could have added a py-postgresql backend in 2009, you could have come up with a way to run Django under Apache and Python 3, etc. I'm happy to take a lot of the blame for Django's slow movement, but you're also responsible for not channelling your frustration into action.
> The reality is, everyone has basically been waiting for everyone else to make a move and it has resulted in deadlock.
I don't see a deadlock. I see steady movement -- slow, too slow, but movement nonetheless -- towards Python 3. First Python 3.0 came out, and it was horrifically slow, taking away any impetus to use it. So people fixed that. Then, developers ported the low-level libraries (database drivers, network protocol implementations, etc). Then we started moving to higher level libraries and tooling (numpy, wsgi, ...). Now the highest-level libraries (web frameworks, scipy, matplotlib, ...) are porting. Next, and finally, you'll get to port your apps. It's a classic dependency upgrade chain. You can't upgrade layer N until layer N-1 is done.
I share your frustration about the slowness, but that's frankly a characteristic of the Python community. We're relatively conservative, tending to prioritize stability and backwards compatibility over rapid movement. Everyone pretty much expected this process to take a while, and turns out they're right.
How exactly does one create a fact? That would be a neat trick.
> I'm sure you could have found a way to talk to web servers without waiting years for the WSGI spec to be finalized.
Indeed, we could have, and it would have benefited Django at the expense of diversity in the Python web world. It probably would have helped our "market share". But that's not how we roll. I'd rather work on something that the whole community can benefit from, and that means standards and specifications.
> And if Django had supported Python 3 using py-postgresql, which was released in 2009 [...]
That's true - I'd forgotten about that library.
> [...] others would have worked a little faster to make their database systems available on Python 3 as well.
That... I'm not so sure about. Volunteer community, limited by free time, etc. I'm fairly sure most people have worked on this stuff just about as fast as they can.
You do realize, of course, that the criticism here applies to you as well. If you feel this strongly you could have added a py-postgresql backend in 2009, you could have come up with a way to run Django under Apache and Python 3, etc. I'm happy to take a lot of the blame for Django's slow movement, but you're also responsible for not channelling your frustration into action.
> The reality is, everyone has basically been waiting for everyone else to make a move and it has resulted in deadlock.
I don't see a deadlock. I see steady movement -- slow, too slow, but movement nonetheless -- towards Python 3. First Python 3.0 came out, and it was horrifically slow, taking away any impetus to use it. So people fixed that. Then, developers ported the low-level libraries (database drivers, network protocol implementations, etc). Then we started moving to higher level libraries and tooling (numpy, wsgi, ...). Now the highest-level libraries (web frameworks, scipy, matplotlib, ...) are porting. Next, and finally, you'll get to port your apps. It's a classic dependency upgrade chain. You can't upgrade layer N until layer N-1 is done.
I share your frustration about the slowness, but that's frankly a characteristic of the Python community. We're relatively conservative, tending to prioritize stability and backwards compatibility over rapid movement. Everyone pretty much expected this process to take a while, and turns out they're right.
How exactly does one create a fact? That would be a neat trick.
As I said, by implementing something instead of waiting for the spec. I totally disagree that it would have helped Django at the expense of everyone else, unless you patented it. Any implementation, even lots of fragmented ones, would have been better than years of stalemate. I guess we'll just have to agree to disagree on that one.
You do realize, of course, that the criticism here applies to you as well
Yes, absolutely, that's a very fair point. I just didn't expect things to move _that_ slowly. With the benefit of hindsight I want to punch myself in the face for not doing it myself.
As I said, by implementing something instead of waiting for the spec. I totally disagree that it would have helped Django at the expense of everyone else, unless you patented it. Any implementation, even lots of fragmented ones, would have been better than years of stalemate. I guess we'll just have to agree to disagree on that one.
You do realize, of course, that the criticism here applies to you as well
Yes, absolutely, that's a very fair point. I just didn't expect things to move _that_ slowly. With the benefit of hindsight I want to punch myself in the face for not doing it myself.
There isn't any "migration disaster". There was a five (5) year plan laid out by Guido. We are not even into year three (3) yet (or maybe just into it). If anything it is a "patience disaster".
1. http://scipy.github.com/faq.html#do-numpy-and-scipy-support-...