Python 3 in 2016(hynek.me)
hynek.me
Python 3 in 2016
https://hynek.me/articles/python3-2016/
31 comments
For anything involving data analysis and scientific computing Python really shines - though I'm often surprised the extent to which web-oriented programmers are unaware of this, I read things like "you can write better web services in Go, therefore Python is obsolete".
Python in science is kinda like the One Ring the Binds them - it is the glue that holds together the HPC code at CERN, ESA, NASA, and more, it crunches climate models and delivers your daily weather forcasts. Some really great projects like Cython, Theano, and Numba boost the performance of numeric code to native speeds with a fraction of the hassle of hand-written C. Mountains of machine learning research is done in Python, as well as statistics and analytics. And of course Numpy remains the de-facto standard of N-dimensional array libraries - if you look at numeric computing libraries in many other languages they will be framed as "Numpy-for-X".
Python in science is kinda like the One Ring the Binds them - it is the glue that holds together the HPC code at CERN, ESA, NASA, and more, it crunches climate models and delivers your daily weather forcasts. Some really great projects like Cython, Theano, and Numba boost the performance of numeric code to native speeds with a fraction of the hassle of hand-written C. Mountains of machine learning research is done in Python, as well as statistics and analytics. And of course Numpy remains the de-facto standard of N-dimensional array libraries - if you look at numeric computing libraries in many other languages they will be framed as "Numpy-for-X".
Yeah, I don't know
While Numpy/Scipy is great with a capital G, it seems the underlying infrastructure is flaky.
I'm talking about the messy process of building and installing it. And it seems everything is held with duct-tape
Also it seems that "inside Scipy' it is a different world, and that to really "talk to the outside world" there seems to be a data-translation step needed.
While Numpy/Scipy is great with a capital G, it seems the underlying infrastructure is flaky.
I'm talking about the messy process of building and installing it. And it seems everything is held with duct-tape
Also it seems that "inside Scipy' it is a different world, and that to really "talk to the outside world" there seems to be a data-translation step needed.
The complexity of building NumPy or SciPy is pretty much par for the course in HPC. Most of the libraries it is based on are battle-hardened and ridiculously well tested to a degree not seen in many places, but they date from a different era, including libraries where originally "building" it involved a hole puncher and a bucket of coffee. Getting all those libraries to play nicely together at all is a genuine miracle.
Most any significant scientific package I've ever had to build (which is quite a few) had a complex build system that felt like it was built on top of shaky glass pillars. It's just the nature of what's involved and it's a fundamentally hard problem; it's difficult to replace those libraries because the extreme amount of verification/validation and optimization that have gone into something like LAPACK makes it economically undesirable.
I'd say also that this is part of the appeal of NumPy and SciPy, which is they glue together a huge number of tools that are otherwise disjoint. You don't have to worry (most of the time) about matrix ordering, calling conventions, those mysterious "workspace" variables in LAPACK etc.
Most any significant scientific package I've ever had to build (which is quite a few) had a complex build system that felt like it was built on top of shaky glass pillars. It's just the nature of what's involved and it's a fundamentally hard problem; it's difficult to replace those libraries because the extreme amount of verification/validation and optimization that have gone into something like LAPACK makes it economically undesirable.
I'd say also that this is part of the appeal of NumPy and SciPy, which is they glue together a huge number of tools that are otherwise disjoint. You don't have to worry (most of the time) about matrix ordering, calling conventions, those mysterious "workspace" variables in LAPACK etc.
Working in Windows, my biggest issue was "the 64 bit situation", but that has been greatly solved by Microsoft's MSVS for Python (lovely move by Redmond) and the Python Wheels. Now it has truly become `pip install something` or `pip install [wheel file]` for more complex stuff.
That and Cristoph Gohlke[0]'s repository of pre-compiled modules. That guy saved me more than once. Should I ever meet him, I'll sure buy him a beer!
[0] http://www.lfd.uci.edu/~gohlke/pythonlibs/
That and Cristoph Gohlke[0]'s repository of pre-compiled modules. That guy saved me more than once. Should I ever meet him, I'll sure buy him a beer!
[0] http://www.lfd.uci.edu/~gohlke/pythonlibs/
> "I'm talking about the messy process of building and installing it."
If you're using Python for scientific computing, you can use the Anaconda distribution of Python, it makes it painless to work with those packages.
https://www.continuum.io
> "Also it seems that "inside Scipy' it is a different world, and that to really "talk to the outside world" there seems to be a data-translation step needed."
Python has excellent ORMs, I'm not sure what you're getting at.
If you're using Python for scientific computing, you can use the Anaconda distribution of Python, it makes it painless to work with those packages.
https://www.continuum.io
> "Also it seems that "inside Scipy' it is a different world, and that to really "talk to the outside world" there seems to be a data-translation step needed."
Python has excellent ORMs, I'm not sure what you're getting at.
It's a really complicated piece of kit, that has also been around since long before modern build systems and conventions existed. That's why you have excellent scientific packages like Anaconda or Canopy, while on linux you can also install your distro's package. Situation is also becoming easier with pre-built python wheels.
Years ago, it used to be harder to install numpy, but this has changed. On MacOS, it's just
% pip install numpy
For more (e.g., other OS'es, build-from-source), see:
http://docs.scipy.org/doc/numpy-1.10.1/user/install.html
% pip install numpy
For more (e.g., other OS'es, build-from-source), see:
http://docs.scipy.org/doc/numpy-1.10.1/user/install.html
at least on OS X installing numpy and scipy is very easy - pip installs prebuilt wheels by default, no need to compile anything
Two days ago I took a proof-of-concept project written in Python, using the GMPY2 library (Gnu Multiple Precision library), and ported it to C.
My C code was very complicated in order to be fast. Lots of pointer swapping, bit-shifting, etc.
The code was about 4% faster in C, which was more lines of code, much harder to read (and write), compiled with every imaginable flag that could offer a speedup. Switching operating systems, this code was about 8% faster in C on Linux than Python in Windows 10. C in Linux would take about 9.5 days to complete, Python in Windows 10 would be about 10.5 days.
In addition, Python has the "pickle" module so that I can easily make the algorithm save state to pause and restore it any time I want. What's more, I can probably use mmap to share memory easily without making copies and switch to multiprocessing for an even greater speedup (my first attempt using Pool was a little slower despite using 3 cores, but that's the slowest way to multithread).
My C code was very complicated in order to be fast. Lots of pointer swapping, bit-shifting, etc.
The code was about 4% faster in C, which was more lines of code, much harder to read (and write), compiled with every imaginable flag that could offer a speedup. Switching operating systems, this code was about 8% faster in C on Linux than Python in Windows 10. C in Linux would take about 9.5 days to complete, Python in Windows 10 would be about 10.5 days.
In addition, Python has the "pickle" module so that I can easily make the algorithm save state to pause and restore it any time I want. What's more, I can probably use mmap to share memory easily without making copies and switch to multiprocessing for an even greater speedup (my first attempt using Pool was a little slower despite using 3 cores, but that's the slowest way to multithread).
All this because you didn't profile your code to see that 95% of the time was spent in GMP routines.
Yeah. I wasn't sure how GMPY was compiled, so I used C with my own compilation of GMP. I bet the flag differences made up the few percentage points.
Plus, with each variable being ~10Mb, I was concerned about memory copying (I studied mechanical engineering, I missed a lot of this stuff in school), and with C I could make memory copying vs passing pointers explicit. All of that was unnecessary.
It was another reinforcement of the idea: "Build it first, then optimize".
Plus, with each variable being ~10Mb, I was concerned about memory copying (I studied mechanical engineering, I missed a lot of this stuff in school), and with C I could make memory copying vs passing pointers explicit. All of that was unnecessary.
It was another reinforcement of the idea: "Build it first, then optimize".
Pick Python for the sheer amount of libraries that's available. Not just for web stuff, where Node.js and Ruby do fine, but also for scientific work, for statistics, for embedded work (Raspberry Pi or MicroPython), for systems administration, for command-line tools, for really just about anything you can imagine. And you can do all of those things while not having to switch contexts between different syntaxes and different standard libraries for basic things like regular expressions or HTTP requests.
Picking the best tool for the job often gets in the way of actually getting things done. Python is the true swiss army knife of today's programming languages.
Picking the best tool for the job often gets in the way of actually getting things done. Python is the true swiss army knife of today's programming languages.
That's a good reason, but note that it used to be the reason you picked Perl.
Still a valid reason for people to chose Perl.
Not advocating for Perl, just on behalf of Perl lovers like my office mate
Not advocating for Perl, just on behalf of Perl lovers like my office mate
The difference is that most people can read most other peoples' python code. Perl didn't tend to have that quality.
EDIT Added another "most" to make it clear that not everyone can write readable python.
EDIT Added another "most" to make it clear that not everyone can write readable python.
That depends entirely on who is writing the code. I have seen some truly awful Python as well as truly awful Perl.
This difference is that I've mainly seen decent Python and mainly seen awful Perl.
Although larger network effects might make it harder to turn the ship, this time around.
This is exactly the reason why I use C, C++ and sometimes Fortran. Yet to see a Python library I'd need.
I often dabble in other languages but in the end always return to python (3).
What it offers for me truly is the batteries included and easy, memorable and predictable syntax. (Probably due to experience).
Sure, Go is easier to deploy, Erlang and by extension Elixir are easier to scale.
But python does align very well with my personal though process. I don't think parallel and I don't have race conditions in my mind. If I want to quickly write something that will perform decently, I choose python, because the time I save by not having to learn a new language could be days or perhaps even weeks. I guess it boils down to this: with my current experience, I'm more then 10 times as productive in python as I would be in any other language.
Sure, it would be neat if I had as much experience in Erlang as I have in python, but that will take years, and at the end of the day, I'd rather finish my project this year with python, then next year with any other language.
What it offers for me truly is the batteries included and easy, memorable and predictable syntax. (Probably due to experience).
Sure, Go is easier to deploy, Erlang and by extension Elixir are easier to scale.
But python does align very well with my personal though process. I don't think parallel and I don't have race conditions in my mind. If I want to quickly write something that will perform decently, I choose python, because the time I save by not having to learn a new language could be days or perhaps even weeks. I guess it boils down to this: with my current experience, I'm more then 10 times as productive in python as I would be in any other language.
Sure, it would be neat if I had as much experience in Erlang as I have in python, but that will take years, and at the end of the day, I'd rather finish my project this year with python, then next year with any other language.
Concur with the 'thought process.' I'm a happier person after a day of writing Python than I am after a day of writing Go. Environments take a long time to mature.
Python finally has module packaging down fairly well (Go still drives me crazy even with the latest vending approach--cannot wait for a major build to break because the module creator switched hosting services).
Python is pretty pleasant to debug. The ability to set breakpoints and then poke around and/or modify values saves me hundreds of hours a year.
Building executables can still be a bit problematic (one reason we used Go for a few projects -- they really have this down). However, latest pyInstaller has been working smoothly on Windows and Linux.
Python is quick enough -- and we know how to scale websites with WSGI and stand-alone apps with asyncio (or Twisted). Rarely does the need for that last ounce of speed outweigh the ability to quickly create and modify apps.
Elixir, of all the new languages, intrigues me the most; not just for the language but being built on top of Erlang's OTP platform. They also seem to be doing a decent job on the packaging and distribution front.
Python finally has module packaging down fairly well (Go still drives me crazy even with the latest vending approach--cannot wait for a major build to break because the module creator switched hosting services).
Python is pretty pleasant to debug. The ability to set breakpoints and then poke around and/or modify values saves me hundreds of hours a year.
Building executables can still be a bit problematic (one reason we used Go for a few projects -- they really have this down). However, latest pyInstaller has been working smoothly on Windows and Linux.
Python is quick enough -- and we know how to scale websites with WSGI and stand-alone apps with asyncio (or Twisted). Rarely does the need for that last ounce of speed outweigh the ability to quickly create and modify apps.
Elixir, of all the new languages, intrigues me the most; not just for the language but being built on top of Erlang's OTP platform. They also seem to be doing a decent job on the packaging and distribution front.
> But python does align very well with my personal though process.
TBH, that's the most rational reason ever for choosing any language. Anything else is much less important.
But, yes, for me the experience is the opposite, Python flies in the face of my thought process in every single aspect of it. I find it limiting, inflexible and dominating. Which means, everyone think differently, and there is no single language that suits all possible combinations of problem domains and thinking styles.
TBH, that's the most rational reason ever for choosing any language. Anything else is much less important.
But, yes, for me the experience is the opposite, Python flies in the face of my thought process in every single aspect of it. I find it limiting, inflexible and dominating. Which means, everyone think differently, and there is no single language that suits all possible combinations of problem domains and thinking styles.
>Sure, Go is easier to deploy
Go's lack of a package manager making deployment more painful is the thing that made me come back to python.
Go's lack of a package manager making deployment more painful is the thing that made me come back to python.
Go is staticly linked, you don't need to deploy packages, just the binary.
But you have to build those with its dependencies. And the best thing Go came up with is vendorizing.
Readability. Ease of scripting. Libs. Multi-paradigm. Simplicity to debug. Good instrospection. Expressivity. Fantastic community. Great documentation (because hey, docstrings). It's a no brainer to teach and to learn or train for and gives you good habits.
And it does the job.
And it does the job.
All you said, and also: the REPL and ipython.
Python is still king for really boring and maintainable scripts. I mean boring in the best way. Every Python script o read is clean and elegant. It's easy to follow and well organized. Python 3 doesn't seem to make it any better at this. In fact all Python3 does is annoy me. I never know when I need it or not, it never seems to be clear either. The fact that it's packaged as a separate entity (i.e. /user/local/bin/python3) is even weirder.
Mastering Python-foo is still on my todo list, though. As a Ruby user, I've noticed that Python scripts are much more straight forward and maintable. I still write my scripts in Ruby though because nobody seems to mind. This is perhaps because I try my best write good comments. In general, Ruby is better as Perl replacement than a Python substitute. In fact, one of my coworkers who is an oldschool perl scripters was sold on Ruby over Perl just by reviewing one of my scripts. I think the issue is people don't think of Ruby outside of Rails, where it's basically a readable purely object oriented Perl.
As an aside, I think there is a bigger overall problem of using scripting languages for application development. I wouldn't use Python, Ruby, or Perl for anything that was more than, say, 500 lines. I almost cried when I tried to read the code for homebrew. Not that it's bad code (it's quite good actually), it's just way too much duck typing for me. I could never keep all that type information in my head. I prefer to pass that cognitive load to the compiler.
Mastering Python-foo is still on my todo list, though. As a Ruby user, I've noticed that Python scripts are much more straight forward and maintable. I still write my scripts in Ruby though because nobody seems to mind. This is perhaps because I try my best write good comments. In general, Ruby is better as Perl replacement than a Python substitute. In fact, one of my coworkers who is an oldschool perl scripters was sold on Ruby over Perl just by reviewing one of my scripts. I think the issue is people don't think of Ruby outside of Rails, where it's basically a readable purely object oriented Perl.
As an aside, I think there is a bigger overall problem of using scripting languages for application development. I wouldn't use Python, Ruby, or Perl for anything that was more than, say, 500 lines. I almost cried when I tried to read the code for homebrew. Not that it's bad code (it's quite good actually), it's just way too much duck typing for me. I could never keep all that type information in my head. I prefer to pass that cognitive load to the compiler.
Fundamentally, python3 fixes some of the serious WATs in python2 and makes it easier to write high quality and maintainable software. For a pretty quick overview take a look at:
https://migrateup.com/main-difference-python-3/
Perhaps if python3 annoys you more than makes you say, "Hey, this prevents lots of latent bugs I might have easily shipped to production before, cool!", then you need to learn python a bit better and re-evaluate what is annoying about it.
https://migrateup.com/main-difference-python-3/
Perhaps if python3 annoys you more than makes you say, "Hey, this prevents lots of latent bugs I might have easily shipped to production before, cool!", then you need to learn python a bit better and re-evaluate what is annoying about it.
Sorry, maybe I was unclear. I haven't really started learning python yet. I read it, and could probably write it. I don't know the ins and outs of the language. I don't even think I've written anything more than "hello world" in python, although, I'm pretty sure that program was also valid Ruby -- so it might not even count.
What I was trying to say is that the portability of python3 is bad enough that it effects non-python programmers.
For example, sometimes I want to try a new vim plugin that's written in python. I install it, and it doesn't work. Great. Then I find out it's because OS X's python can't interpret it. So I install python3 (which has a different executable name, also annoying). All good, right? Wrong. Vim needs to be compiled with python3 support because the python extension is exclusive to python. Now I have to recompile macvim with
It doesn't ruin my whole day or anything, but it's a bit irritating to have to manage several versions of an interpreter for a language I'm not even using that language in production. It doesn't help that I use several machines and VMs.
So yeah, I know exactly what is annoying about it.
What I was trying to say is that the portability of python3 is bad enough that it effects non-python programmers.
For example, sometimes I want to try a new vim plugin that's written in python. I install it, and it doesn't work. Great. Then I find out it's because OS X's python can't interpret it. So I install python3 (which has a different executable name, also annoying). All good, right? Wrong. Vim needs to be compiled with python3 support because the python extension is exclusive to python. Now I have to recompile macvim with
brew install macvim --with-python3 && brew install python3
It doesn't ruin my whole day or anything, but it's a bit irritating to have to manage several versions of an interpreter for a language I'm not even using that language in production. It doesn't help that I use several machines and VMs.
So yeah, I know exactly what is annoying about it.
I believe the parent was talking about python3 from a users perspective (they said they aren't a Python programmer) For example when a script requires python3 and another requires python. Python3 is also not included in a lot of base systems. That could be annoying.
> or Go.
If you like Go, try Nim - it's like Go but without all the dogma. I wrote a blog post about my little Nim project: https://klibert.pl/output/slock_in_nim.html
> Why should I choose Python 3 for my new projects?
You shouldn't unless it fits all the constraints of your new project. Choosing a language is not, for the most part, a technical decision: sure there are language features and implementation features (those are separate matters!), but in the end it comes down to external factors: your experience with a particular paradigm, your taste in syntax, skills of potential collaborators and so on.
One thing of note, though: there is no rule saying that you can't use more than one language for your project. On the contrary, you can mix-and-match languages with relative ease nowadays (see https://klibert.pl/output/python_interop.html) and, in my experience, it works rather well. For example, in one of my projects I need to scrape a bunch of websites for data and then process this data. I wrote scraper in Elixir, which made dealing with network errors and parallelizing requests a breeze, and used Python with Pandas for the analysis part. It was a very pleasant experience: neither Python nor Elixir were up to the task on their own, but they worked very well together. It's even easier when you use a JVM, .NET or JavaScript-based languages.
If you like Go, try Nim - it's like Go but without all the dogma. I wrote a blog post about my little Nim project: https://klibert.pl/output/slock_in_nim.html
> Why should I choose Python 3 for my new projects?
You shouldn't unless it fits all the constraints of your new project. Choosing a language is not, for the most part, a technical decision: sure there are language features and implementation features (those are separate matters!), but in the end it comes down to external factors: your experience with a particular paradigm, your taste in syntax, skills of potential collaborators and so on.
One thing of note, though: there is no rule saying that you can't use more than one language for your project. On the contrary, you can mix-and-match languages with relative ease nowadays (see https://klibert.pl/output/python_interop.html) and, in my experience, it works rather well. For example, in one of my projects I need to scrape a bunch of websites for data and then process this data. I wrote scraper in Elixir, which made dealing with network errors and parallelizing requests a breeze, and used Python with Pandas for the analysis part. It was a very pleasant experience: neither Python nor Elixir were up to the task on their own, but they worked very well together. It's even easier when you use a JVM, .NET or JavaScript-based languages.
In Nim, isn't it the case that concurrency is thread based? Something I really like about Go is that goroutines are cheap green threads, not full OS threads. This lets me spawn as many goroutines as I like without worrying about exhausting a thread pool.
There is some support for coroutines with async/await (http://nim-lang.org/docs/asyncdispatch.html), but I don't know how mature or functional it is, never tested it. As mentioned, for anything involving concurrency I'd rather use Erlang or Elixir; but it may be a good idea to see how well Nim does in this area.
Most of my development experience is in Python, and Clojure's what excites me the most these days. I've had to write a lot of Javascript for work, and it turns out that there's a large community of Javascript developers who wish they were writing Clojure, and they've built libraries like tcomb, Immutable.js and transducers.js to get pretty close.
In the transpilers era, Javascript will never be so bad that choosing another dynamic language is a clear win. I'd go back to Python and Django for a CRUD app, but since Javascript is easier to write in a functional style than Python, I've actually started to prefer it as a first choice.
In the transpilers era, Javascript will never be so bad that choosing another dynamic language is a clear win. I'd go back to Python and Django for a CRUD app, but since Javascript is easier to write in a functional style than Python, I've actually started to prefer it as a first choice.
For personal projects, you can pick whatever language you want, possibly the most fun one that is still practical. Haskell or Elixir look promising in this regard.
For companies, there are more factors to consider, eg. finding devs at a correct price, getting existing workforce to work on a new tech, size of standard/open source library, getting help and bugfixes from community instead of hacking on your own, risk of language/tech losing steam, etc. That's why several comparatively "boring" languages still thrive.
For companies, there are more factors to consider, eg. finding devs at a correct price, getting existing workforce to work on a new tech, size of standard/open source library, getting help and bugfixes from community instead of hacking on your own, risk of language/tech losing steam, etc. That's why several comparatively "boring" languages still thrive.
I personally find python in general to align almost perfectly with the way I think about writing software. It's interpreted so I can do things like compiling dynamic python in my code or fall into a python shell for debugging. Majority of time, I can use the standard library to do everything I need. And I love the clean syntax of Python — always a pleasure to look at.
As for Py3, the standard libs are very clean and streamlined, (e.g. io). Oddly enough, the point that I love about Py3 is its handling of unicode vs bytes (which most people find it difficult for some reason).
I'm not sure what you mean by "can't target the browser"? I don't find distributing/deploying python to be difficult or even involved at all. I do get annoyed at the single CPU bound limitation.
If I had to summarize why I'd pick Py3 for new projects, I'd say because it thrives in being minimalist and I hate clutter.
As for Py3, the standard libs are very clean and streamlined, (e.g. io). Oddly enough, the point that I love about Py3 is its handling of unicode vs bytes (which most people find it difficult for some reason).
I'm not sure what you mean by "can't target the browser"? I don't find distributing/deploying python to be difficult or even involved at all. I do get annoyed at the single CPU bound limitation.
If I had to summarize why I'd pick Py3 for new projects, I'd say because it thrives in being minimalist and I hate clutter.
"Better the devil you know". If it's a professional project, I prefer not to have to learn the undocumented potholes of new language while debugging before a deadline.
I never thought I'd say it, but I'm loving programming the new ES2105 version of JavaScript. It really has a great deal in common with Python now. Clearly many programming languages are converging as they implement common good ideas from other programming languages. That's going to mean that whilst Python might have distinguished itself in the past for being streets ahead, in the future it's likely that many languages will also have many of those same good ideas incorporated.
I posted about it:
What language has decorators, generators, comprehensions, modules, but isn't Python?
http://fourlightyears.blogspot.com.au/2015/08/what-language-...
I posted about it:
What language has decorators, generators, comprehensions, modules, but isn't Python?
http://fourlightyears.blogspot.com.au/2015/08/what-language-...
> I never thought I'd say it, but I'm loving programming the new ES2105 version of JavaScript.
Well, yeah, I'm sure ES2105 is cool, but most of us don't have the choice of using languages from the next century.
Well, yeah, I'm sure ES2105 is cool, but most of us don't have the choice of using languages from the next century.
You mean yavascript?
> What language has decorators, generators, comprehensions, modules, but isn't Python?
The "all async" model of Node kills it for practical programming compared to Python...
And the worst thing is, Node/v8's JIT would kill Python in performance for a;; the kind of things people write Python scripts for, even if it run all synchronous.
It would be nice if callbacks/promises/async-await were only optional and for network/web things, not for everything.
And the worst thing is, Node/v8's JIT would kill Python in performance for a;; the kind of things people write Python scripts for, even if it run all synchronous.
It would be nice if callbacks/promises/async-await were only optional and for network/web things, not for everything.
async/await isn't too bad (es7 via babel)
The difference between python an javascript wasn't in features, it was always about predictability. I can pick out errors and expected behavior in python much more easily then javascript, and unless it breaks compatibility at some point, I suspect it will stay that way.
You did not define what that project would be.
I suppose you should first figure out the domain, then figure out the language support and ecosystem around that domain and then pick one. That is, if you are implementing something practical whose aim is to provide value outside of personal learning.
For example, several constraints still lead to a situation where C++ is a prime choice unless one is mostly interestes in hacking in a new domain and not finishing a project.
Note! I'm not dissing trying out languages for the joy of it!
Personally, I would pick Python for experimenting with syntax transforms, data mining and such. And as a basis for any offline tool that needs to chug bits from one place to another. Also, the numerical use cases listed elsewhere.
Basically, for anything that does not need a GUI.
I suppose you should first figure out the domain, then figure out the language support and ecosystem around that domain and then pick one. That is, if you are implementing something practical whose aim is to provide value outside of personal learning.
For example, several constraints still lead to a situation where C++ is a prime choice unless one is mostly interestes in hacking in a new domain and not finishing a project.
Note! I'm not dissing trying out languages for the joy of it!
Personally, I would pick Python for experimenting with syntax transforms, data mining and such. And as a basis for any offline tool that needs to chug bits from one place to another. Also, the numerical use cases listed elsewhere.
Basically, for anything that does not need a GUI.
If you are building an application which does a lot of machine learning and statistics, I don't see a better language than Python. It has so many libraries to use for it. Note, I love R, but I can't imagine building an application with it.
You can build apps with R, but the result will be the ugliest code you have ever seen. For me, R for research and prototyping, then Python for productionalizing.
Ugly isn't the problem. It is the poor error catching, erratic memory use, several ways to do one thing due to a lack of features in the standard libraries (regex), etc.
Noob question: What's so difficult about deploying/distributing python?
this +1000
The python users I've encountered fall generally into two camps:
1) The engineer/scientist/researcher/hacker type that has python installed on their snowflake machine, and has a whole bunch of packages installed to make their flavor of snowcones. These users just want python to work, find joy in the syntax, and in general get their work done.
2) the developer that is trying to deploy the same thing to many machines. This is a pain. Often you end up with users from camp 1 trying to do this and you end up with a global python install with batteries and the kitchen sink included that is a dependency for a whole bunch of "server" apps. you update a package (because "my special snowcone needs it") and you break something else that you didn't know to test. This is a nightmare. With pip and venv it is easier to manage this, but I still see that relocatable venv is "experimental" when you read the help. IMHO, the python 3 community needs to make it very, very easy to distribute and deploy my special snowcone maker and all its dependencies to a system without python already installed, and not stomp on anybody elses snowcone maker in the process.
The python users I've encountered fall generally into two camps:
1) The engineer/scientist/researcher/hacker type that has python installed on their snowflake machine, and has a whole bunch of packages installed to make their flavor of snowcones. These users just want python to work, find joy in the syntax, and in general get their work done.
2) the developer that is trying to deploy the same thing to many machines. This is a pain. Often you end up with users from camp 1 trying to do this and you end up with a global python install with batteries and the kitchen sink included that is a dependency for a whole bunch of "server" apps. you update a package (because "my special snowcone needs it") and you break something else that you didn't know to test. This is a nightmare. With pip and venv it is easier to manage this, but I still see that relocatable venv is "experimental" when you read the help. IMHO, the python 3 community needs to make it very, very easy to distribute and deploy my special snowcone maker and all its dependencies to a system without python already installed, and not stomp on anybody elses snowcone maker in the process.
This should be the #1 comment/resource on this thread IMO.
I recently started writing a few Qt applications, and was strongly considering PyQt - I mean, why wouldn't I? The syntax is cleaner and I can get more work done faster using Python than C++.
I spent a few days going back and forth - comparing the performance, design time, etc of both - and in the end I'm just using Qt's JS JIT for dynamic list comprehensions and other things C++ is bad at. In the end, the fact that distributing on everything but Linux would have been reduced to having to bundle CPython in was just a deal breaker.
Linux isn't really bad for deployment, though. You can use the SUSE OBS to build packages for all the major platforms besides Arch, and since my main desktop is Arch, I'm building my software with PKGBUILD's from the get-go.
I spent a few days going back and forth - comparing the performance, design time, etc of both - and in the end I'm just using Qt's JS JIT for dynamic list comprehensions and other things C++ is bad at. In the end, the fact that distributing on everything but Linux would have been reduced to having to bundle CPython in was just a deal breaker.
Linux isn't really bad for deployment, though. You can use the SUSE OBS to build packages for all the major platforms besides Arch, and since my main desktop is Arch, I'm building my software with PKGBUILD's from the get-go.
Python isn't very big (about 10-20 MB or so). What's the problem with bundling CPython?
I've written a production app in Python 3 + QT4 (PySide as opposed to PyQT) - I've bundled with cxfreed, and it's great.
Fully cross-platform.. easy to distribute. Quite nice all around.
Fully cross-platform.. easy to distribute. Quite nice all around.
That page dismisses py2exe, but I've used it successfully for over a decade. On unix it is as simple as:
sudo pip install [package/repo]Except, you often need a full compiler toolchain for all the c dependencies. Then you need to make sure you have the required header files. Then you try and deploy to a small EC2 instance and run out of RAM building lxml or similar.
If your requirements are advanced, make packages on your dev box then deploy those:
https://nylas.com/blog/packaging-deploying-python
https://nylas.com/blog/packaging-deploying-python
It's an interesting idea, thanks.
ATM it takes work to create a deployable artifact.
We package virtualenvs using fpm, others use Docker. But Go just spits out a single binary you can scp anywhere (the build process OTOH is more painful IMHO).
That’s why I’m highly curious about https://us.pycon.org/2016/schedule/presentation/2122/ .
We package virtualenvs using fpm, others use Docker. But Go just spits out a single binary you can scp anywhere (the build process OTOH is more painful IMHO).
That’s why I’m highly curious about https://us.pycon.org/2016/schedule/presentation/2122/ .
Fair question. I can only speak from my own experience here:
Productivity, productivity, productivity.
Yes, there are faster, safer, and more modern languages out there. However Python for me still hits nails the hell out of getting stuff done, really really quickly. That applies both to the language itself, and from the standpoint of having such an extensive and mature ecosystem of packages to choose from.
Productivity, productivity, productivity.
Yes, there are faster, safer, and more modern languages out there. However Python for me still hits nails the hell out of getting stuff done, really really quickly. That applies both to the language itself, and from the standpoint of having such an extensive and mature ecosystem of packages to choose from.
> I struggle to find a compelling reason to use Python at all
My issue is I always see Python as the Second Best choice for what I am doing. There is always a better tool for the job. The one thing going for Python is that it is such a generalist BUT when doing work I prefer the Right Tool for the Job approach. I seriously can't think of a job that Python does best.
My issue is I always see Python as the Second Best choice for what I am doing. There is always a better tool for the job. The one thing going for Python is that it is such a generalist BUT when doing work I prefer the Right Tool for the Job approach. I seriously can't think of a job that Python does best.
> I seriously can't think of a job that Python does best.
For me personally it is the best choice for non-throwaway shell scripting once the scripts becomes more complex and/or long lived than a sequence of shell commands.
Python is part of the base install of just about every mainstream Linux distro, so in that regard its only real competitor is Perl.
For me personally it is the best choice for non-throwaway shell scripting once the scripts becomes more complex and/or long lived than a sequence of shell commands.
Python is part of the base install of just about every mainstream Linux distro, so in that regard its only real competitor is Perl.
[deleted]
Since everyone gives anecdotal proofs that Python 3 is just fine, I'll give mine as well. I've migrated to Python3 a rather big project (for the unicode handling) and although it was not quite easy, it'just fine now. I'm active in Qt, web, json, sql domains and I've never needed a library that is Python 2 only. Thus from my own point of view, the Python 3 ecosystem is just complete now. Super.
I think the biggest barrier to adoption right now is OS support. OSX still comes bundled with python2. There's no indication that Apple is working on a transition to 3.
Ubuntu is actively working on the transition but the current LTS release (ie the one companies/enterprise will use) is still locked onto python2.
The same can likely be said for other flavors of *nix.
From a user's perspective, python3 has been ready for 'prime time' for a while. From an OS, systems, enterprise perspective python3 still is still a ways out.
Ubuntu is actively working on the transition but the current LTS release (ie the one companies/enterprise will use) is still locked onto python2.
The same can likely be said for other flavors of *nix.
From a user's perspective, python3 has been ready for 'prime time' for a while. From an OS, systems, enterprise perspective python3 still is still a ways out.
Apple can't keep Python 2 up to date. If you are deploying a Python app on osx you unfortunately already have this problem.
I suspect that many Pythoners will install Python3 (with homebrew) or other on OSX (and whatever other platforms still come with just Python2)
I would recommend to even use pyenv (https://github.com/yyuu/pyenv) that will allow you to install any Python version in your home directory (i.e. not fucking up your system installation).
If you maintain many FOSS libs like I do, together with tox it's priceless.
If you maintain many FOSS libs like I do, together with tox it's priceless.
Homebrew installs software into /usr/local last time I checked so unless you are doing something really strange like manually overwriting system files, you'll be hard pressed to fuck up the whole system. Personally I'd install homebrew into /opt/homebrew.
The real advantage of pyenv is being able to install all interpreters in all versions. If you're developing libraries, you're going to need that.
Like I said. Python3 works for regular python users.
What if I want to distribute a package to an audience that doesn't have a strong Python background.
What would otherwise be a simple 'pip install' just turned into a complex multi-step process to install a new version, update PATH, and use virtualenv to avoid conflicts with the default install used internally by the OS.
These all seem like 'simple' fixes to a regular Python user but they set a huge barrier-of-entry for others.
'Just use [insert python specific fix]' is a UX failure for a language that emphasizes usability.
To reiterate. Python3 is ready for regular python users/devs. Not so much for the rest who consume python applications.
What if I want to distribute a package to an audience that doesn't have a strong Python background.
What would otherwise be a simple 'pip install' just turned into a complex multi-step process to install a new version, update PATH, and use virtualenv to avoid conflicts with the default install used internally by the OS.
These all seem like 'simple' fixes to a regular Python user but they set a huge barrier-of-entry for others.
'Just use [insert python specific fix]' is a UX failure for a language that emphasizes usability.
To reiterate. Python3 is ready for regular python users/devs. Not so much for the rest who consume python applications.
Yes but when you want to distribute a program at the end of the day, you don't want to have to bundle python with every release or ask the use to install it, which makes a lot of people choose python 2 since it is ubiquitous.
python 2 is not ubiquitous, it is not installed by default on Windows
YMMV, but in the past, I've found it fairly easy to just have windows users install python(x,y) for scientific computing purposes. It's huge, but afterwards, most things just work. Anaconda is also a good choice.
>Thus from my own point of view, the Python 3 ecosystem is just complete now.
We're mostly seeing the same thing, but there is a few libraries where we maintain forks, because the original author didn't test Python 3 support, but just claimed that it would work. However all but one of these libraries are by commercial companies an solely for integrating their service. For the third one the original author seems to have evaporated (I believe one of my colleague are planning a fork with some of the other users of the library).
We're mostly seeing the same thing, but there is a few libraries where we maintain forks, because the original author didn't test Python 3 support, but just claimed that it would work. However all but one of these libraries are by commercial companies an solely for integrating their service. For the third one the original author seems to have evaporated (I believe one of my colleague are planning a fork with some of the other users of the library).
I tried switching to 3 a few years ago and was burned by some libraries that didn't work or worked with bugs.
Last year though I tried again and there were absolutely no problems. It's more natural, all non-dead libraries seem to support it, and in any case I was already using "from future import" for a while.
That said, I don't really intend to port old stuff from 2.7 if I can avoid it, so I expect that 2 and 3 will coexist for a while even within the same firm
Last year though I tried again and there were absolutely no problems. It's more natural, all non-dead libraries seem to support it, and in any case I was already using "from future import" for a while.
That said, I don't really intend to port old stuff from 2.7 if I can avoid it, so I expect that 2 and 3 will coexist for a while even within the same firm
That’s exactly the place I found myself in and that made me sum it up.
The PyPi download statistics are constantly being quoted as though the are some exact representation of the state of Python 3 takeup.
I'm calling this as bullshit.
How much of these downloads are computers auto updating their configurations?
How, exactly, do these stats reflect the number of people developing current projects with Python 3?
The number of legacy projects written in Python 2 will be vastly higher than the number of Python3 projects currently in development. Maybe that's what is being updated.
When a statistician does an analysis of what these numbers really mean I will believe them but until then the numbers mean precisely nothing. Or more accurately, they seem to mean exactly what the Python 2 diehards want them to believe.
My assertion is that Python 3 is very actively and healthy and the PyPi stats are just lies, damn lies and statistics.
I'm calling this as bullshit.
How much of these downloads are computers auto updating their configurations?
How, exactly, do these stats reflect the number of people developing current projects with Python 3?
The number of legacy projects written in Python 2 will be vastly higher than the number of Python3 projects currently in development. Maybe that's what is being updated.
When a statistician does an analysis of what these numbers really mean I will believe them but until then the numbers mean precisely nothing. Or more accurately, they seem to mean exactly what the Python 2 diehards want them to believe.
My assertion is that Python 3 is very actively and healthy and the PyPi stats are just lies, damn lies and statistics.
Hmm I’m not sure how to take this comment. The main point of the article is basically that those numbers don’t tell the whole story (even explicitly in footnote 1).
Your post gives the stats some level of credit when really they tell absolutely nothing, or more precisely they tell a tale that is so open to interpretation as to mean nothing at all.
There's no gloom around Python 3. Lots of people support it. The highest ever upvoted post on Reddit/r/Python was a post asking to remove training material that asserts that Python 3 is irrelevant. https://www.reddit.com/r/Python/top/ There must be alot of people who feel positively about Python 3 to vote it up that high, presumably at the same time that the Python 2 diehards are voting the post down.
The appropriate message regarding the PyPi stats is not to give gentle credence to them but to condemn them as a completely irrelevant measure of the success or otherwise of Python 3.
There's no gloom around Python 3. Lots of people support it. The highest ever upvoted post on Reddit/r/Python was a post asking to remove training material that asserts that Python 3 is irrelevant. https://www.reddit.com/r/Python/top/ There must be alot of people who feel positively about Python 3 to vote it up that high, presumably at the same time that the Python 2 diehards are voting the post down.
The appropriate message regarding the PyPi stats is not to give gentle credence to them but to condemn them as a completely irrelevant measure of the success or otherwise of Python 3.
I get my python 2 and 3 dependencies from distribution packages (Arch, Debian), and so do all people using my software. PyPi impact: Zero.
Somehow Python 3 projects don't require updates? PyPI stats are going to be massively inflated due to Continous Integration and such but I don't see why that would tip the scales in one direction or the other by any significant amount.
The newer a pip, the more likely it is that it caches things for example. There a more things, I had long discussions with dstufft about it in the past.
Python 3 is worth it not for asyncio and unicode (though these are very good reasons if you care about them). It's worth it due to 21st century exception handling, which breaks down in Python 2 in a spectacular fashion the moment you raise in your except: clauses.
Ah yeah the small nice things. `raise from`, `__traceback__` on all raised exceptions (instead of dicking with `sys.exc_info`), `nonlocal`, `super` …
I used to write a lot of personal Python scripts (ie ~/bin). I mainly did this because I seem to always screw up bash scripts and just can't remember all the conditional flags ie ([[-z ]] and error handling with bash for me has always been tricky.
However in the last couple of years I have dropped Python in favor of Groovy in part because I know the JVM and its plethora of libraries so well. Groovy is a highly underrated language. Really my only complaint is the startup time is not nearly as fast as Python but it does seem faster than say Clojure scripts. Also one of the things that I have to say I sort of think Java got right is that it has always been backward compatible (I know this was sort of impossible with Python given its not really a byte code language). That is I really don't have to worry if I write Groovy in a pre Java 5 (ie without generics) where as with Python I wasn't sure if I should write new scripts in 3 or 2.
The other annoying thing albeit far less than Ruby is dependencies with Python. Groovy has a complete bad ass way of just declaring dependencies in a script (@Grab) and they will automatically be downloaded. I have yet to see a scripting language that does this (ie dependency install on demand).
BTW since apparently any time I mention Java-like-tech on any HN post I seem to get downvoted... I really really love Python. This was not to denigrate Python or that Groovy is better than Python.
However in the last couple of years I have dropped Python in favor of Groovy in part because I know the JVM and its plethora of libraries so well. Groovy is a highly underrated language. Really my only complaint is the startup time is not nearly as fast as Python but it does seem faster than say Clojure scripts. Also one of the things that I have to say I sort of think Java got right is that it has always been backward compatible (I know this was sort of impossible with Python given its not really a byte code language). That is I really don't have to worry if I write Groovy in a pre Java 5 (ie without generics) where as with Python I wasn't sure if I should write new scripts in 3 or 2.
The other annoying thing albeit far less than Ruby is dependencies with Python. Groovy has a complete bad ass way of just declaring dependencies in a script (@Grab) and they will automatically be downloaded. I have yet to see a scripting language that does this (ie dependency install on demand).
BTW since apparently any time I mention Java-like-tech on any HN post I seem to get downvoted... I really really love Python. This was not to denigrate Python or that Groovy is better than Python.
> I really don't have to worry if I write Groovy in a pre Java 5 (ie without generics)
Groovy has many incompatible changes between versions, esp from 1.8 to 2.0 but also 1.7 to 1.8, 1.6 to 1.7, etc. Many sites provide Groovy 1.8 as the default because no-one there wants to bother with upgrading.
Groovy has many incompatible changes between versions, esp from 1.8 to 2.0 but also 1.7 to 1.8, 1.6 to 1.7, etc. Many sites provide Groovy 1.8 as the default because no-one there wants to bother with upgrading.
Oh I guess its cause I only use it for personal scripts and only really started using it after Groovy 2 (although I used the language before I didn't switch from Python to Groovy till after Groovy 2).
I am also not as Groovy as I could be (ie write like its Java at times) so maybe thats why I haven't ran into issues.
I am also not as Groovy as I could be (ie write like its Java at times) so maybe thats why I haven't ran into issues.
Python3 is still broken in some important ways (virtual environments) on Debian/Ubuntu, which make it a but of a pain to use both in development and production.
Example: Ubuntu 14.04 (and later versions) come with Python 3.4, but pyvenv is unavailable (just try searching ubuntu 14.04 pyvenv to get an idea).
This bug from 2014 is still relevant, although at least the "python3.4-venv" package is available to patch it enough to get going. https://bugs.launchpad.net/ubuntu/+source/python3.4/+bug/129...
Example: Ubuntu 14.04 (and later versions) come with Python 3.4, but pyvenv is unavailable (just try searching ubuntu 14.04 pyvenv to get an idea).
This bug from 2014 is still relevant, although at least the "python3.4-venv" package is available to patch it enough to get going. https://bugs.launchpad.net/ubuntu/+source/python3.4/+bug/129...
I don't understand the python2 vs python3 fuss. Maybe I'm not doing anything interesting enough, but I find it fairly easy to write code that runs in python >= 2.7 and python >= 3.3. If I was working in a python2-only code base, I would just baby-step it towards python3 compatibility. It doesn't have to be a complicated, disruptive task.
Things are a lot easier with recent versions of Python 3. Not spending more time on supporting 2/3 compatible codebases was a big mistake in my opinion. I think Guido had the idea that everyone was eventually going to run 2to3 and then be done with the conversion. There is too much Python 2 code out there for that to work. Python 2 was too successful and businesses have no inventive to convert working apps.
The %-formatting (printf style) for bytes is a big deal (introduced in 3.5). Putting a 'u' prefix in front of text strings and 'b' in front of byte strings will not go a long way in making things work.
The %-formatting (printf style) for bytes is a big deal (introduced in 3.5). Putting a 'u' prefix in front of text strings and 'b' in front of byte strings will not go a long way in making things work.
Next step: bring back print.
For my code base, I'm mostly worried about division being different, which will introduce silent errors when integer types break down (or up?) into floats.
For my code base, I'm mostly worried about division being different, which will introduce silent errors when integer types break down (or up?) into floats.
Unless it is explicitly necessary to use python 2 (like legacy stuff, that hasn't been upgraded yet), I've been using python 3 solely for almost 2 years, and there is no turning back. I have no reason to stick with the old version.
The last piece that needs to change, to complete the transition, is the majority of OS makers to bundle python3 by default.
The last piece that needs to change, to complete the transition, is the majority of OS makers to bundle python3 by default.
I would be as well, but many jobs out there involve existing code on python 2.
We are using Python LDAP library[1] at work, but it is not available for Python 3 yet. Is there an alternative?
1 https://pypi.python.org/pypi/python-ldap/
1 https://pypi.python.org/pypi/python-ldap/
Yes!
RedHat needed it too and forked the project: https://pypi.python.org/pypi/pyldap
RedHat needed it too and forked the project: https://pypi.python.org/pypi/pyldap
ldap3 is not API-compatible with python-ldap, but it's pure-Python (it doesn't need the OpenLDAP libraries installed) and you might find its API is a bit nicer to use:
https://pypi.python.org/pypi/ldap3
https://pypi.python.org/pypi/ldap3
I can second the recommendation for ldap3. It's very well-engineered, well-documented, works with python 2 and 3, and has (if I remember correctly) zero dependencies.
We've been using Python 3 since we implemented a new service architecture and it's been an easy transition for our 50+ engineers.
I, for one, must admit to having almost completely ignored Python 3 until very recently. One thing I discovered that is way awesomer than you can imagine is Unicode handling: it just works! I'd say Unicode was a tricky issue in 50% of Python projects I've worked on in the past, but with Python 3, none of that matters anymore.
Been thinking about learning to code, and a lot of people have recommended Python as a good language to begin with. Taking a look at this, should I be looking elsewhere?
Python is an excellent language to begin with, you're reading this article wrong if it puts you off Python. There is drama around Python 2 or Python 3, but the answer is clear. Use Python 3 for new projects, use Python 2 when you have to (legacy projects).
It has nice features:
ps: I don't like it's oop syntax sometimes
That said, I never learned so much about one language than by learning another one. All paradigms have at least one gem to discover.
- not large syntax
- expressive overloaded keywords (in)
- slice syntax that is easy on mind and eyes
- nice set of builtin structure: lists, sets, dicts
- with literals for all of them [], {}, {:}
- lambda, generators, context
(cute yet general way to ensure releasing resources, files, connections..)
ps: I don't like it's oop syntax sometimes
That said, I never learned so much about one language than by learning another one. All paradigms have at least one gem to discover.
[deleted]
The problem is not Python 3, the problem is keeping both languages in my head at the same time. I'm tired of caring whether "print" is supposed to be a function or not, I just want to print something.
Why not just always use print() as a function? That's only a single mental change, and works in both 2 and 3.
print isn't a function (even though print("hello") works) in Python 2. But you can use __future__ to make it a function.
Three is (finally) likely to win, if only because of the constant nagging of the project leaders, and the fact that some distros will now move to it by default. But it will be a pyrrhic victory because it will have been a force-feed, not an enthusiastic, willing, adoption by the community. As a result of this mess, while Python's inertia is huge, its momentum is slowing. Its leadership, having lost credibility, cannot really lead effectively anymore.
It will continue to be a massive, unavoidable language, but something of the magic of when I first learned it, has been snuffed out.
It will continue to be a massive, unavoidable language, but something of the magic of when I first learned it, has been snuffed out.
That’s only true for people who are part of the community for a long time. Newcomers don’t understand what the fuss is about and why anyone would want to use Python 2.
My own Python bubble is more like what you described (although the mood is improving IMHO) but talking to newcomers is eye opening.
My own Python bubble is more like what you described (although the mood is improving IMHO) but talking to newcomers is eye opening.
[deleted]
[deleted]
I code in both 2 & 3, and had been nervous about that. New stuff with low dependencies in 3, and older stuff with Python C API dependencies in 2 (eg spreadserve.com). I was worried that the move to 3 would force migration of my 2 based code, so I'm reassured rather than alarmed by this article. The cognitive load switching between two Python dialects isn't as bad as, for instance, switching between C++ with and without STL & Boost.
On the web framework side of things, who is using aiohttp (web) and something like API-Hour? Seems like the performance is really great. Python 3 asyncio.
It's hard to find people blogging about it, not sure why? It seems like a great combination.
http://blog.gmludo.eu/2015/02/macro-benchmark-with-django-fl...
It's hard to find people blogging about it, not sure why? It seems like a great combination.
http://blog.gmludo.eu/2015/02/macro-benchmark-with-django-fl...
I've debated jumping from perl only dev into the python world for a couple of years now.
I've got a list of Django youtube vids queued up on my plex server right now. But I was thinking yesterday (while in the shower of course), perl has perl6 (which may or may not take off). But what does the future of python have?
Python 4,5,6? Any real innovative changes or just more of kicking the can down the road (such as perl5)?
I've got a list of Django youtube vids queued up on my plex server right now. But I was thinking yesterday (while in the shower of course), perl has perl6 (which may or may not take off). But what does the future of python have?
Python 4,5,6? Any real innovative changes or just more of kicking the can down the road (such as perl5)?
I think the best case for moving to Python is for the tooling around math and data science. If you're build other types of web apps in Perl and looking to switch to something else. I'm not sure Django is going to offer you a ton of benefit for the switching cost, IMHO.
Well the benefit I initially see is more jobs and with a side note of hopefully a cleaner code base.
There are not too many perl companies that have been founded in the last 10 years. And I am finding more and more that companies that are older than 10 years old and are using perl, that this usually means they haven't changed a dang thing in their architecture or web dev process since 1998.
Which means I get to deal with CGI spaghetti, 6+ second page load times (so count ajax anything out), and the world's worse use of javascript. I've had good experiences with newer perl companies with short contract work, but these older ones are killing me and I'm pretty bummed about it as you can tell. I've worked hard to learn best practices and modern web dev styles, none of which I can use at the older places.
There are not too many perl companies that have been founded in the last 10 years. And I am finding more and more that companies that are older than 10 years old and are using perl, that this usually means they haven't changed a dang thing in their architecture or web dev process since 1998.
Which means I get to deal with CGI spaghetti, 6+ second page load times (so count ajax anything out), and the world's worse use of javascript. I've had good experiences with newer perl companies with short contract work, but these older ones are killing me and I'm pretty bummed about it as you can tell. I've worked hard to learn best practices and modern web dev styles, none of which I can use at the older places.
What's the can here?
I started off python learning python 3, but a few months into it, I was forced do a project in python 2. It was a turning point, I immediately ported over all my python projects to python 2. To me, it just seemed more simple, elegant, and cleaner. It was the more pythonic version of python. And if I'm still working with python after 2020, I'll be using python 2.
I don't understand this, what is more "pythonic" about 2?
Not typing 'encode' or 'decode' when dealing with strings. I know it sounds stupid, but it's really distracting compared to everything else in python that just flows easily.
Then you're doing it wrong. Sorry but that's definitely better solved in Python3 than in 2. Always decode/encode when opening a file/stream/whatever and only use unicode internally.
Sounds like you are shooting yourself in the foot. Your code may work with ASCII but will immediately break in non-naïve cases.
It's quite possible that someone will come up with a good enough automatic 2->3 converter which alleviate the situation.
There are converters, and it's not too difficult to even write code that runs in both Python 2 and 3.
The main roadblock in upgrades is unicode text vs bytes handling.
Python2 was lax about using bytes for text. You could do unicode and bytes properly if you really tried hard enough, but it took a conscious effort. So most people just used bytes for text, and hoped for the best (if the data is ASCII or UTF-8, it will work most of the time).
Python3 is strict about the text vs bytes separation. Patching this into a program afterwards is a lot of work.
But if your Python2 program already does unicode carefully in Python2 (unicode type internally everywhere, from __future__ import unicode_literals, explicit encode/decode when talking to external APIs), then you are 90% done with the upgrade to Python3.
The main roadblock in upgrades is unicode text vs bytes handling.
Python2 was lax about using bytes for text. You could do unicode and bytes properly if you really tried hard enough, but it took a conscious effort. So most people just used bytes for text, and hoped for the best (if the data is ASCII or UTF-8, it will work most of the time).
Python3 is strict about the text vs bytes separation. Patching this into a program afterwards is a lot of work.
But if your Python2 program already does unicode carefully in Python2 (unicode type internally everywhere, from __future__ import unicode_literals, explicit encode/decode when talking to external APIs), then you are 90% done with the upgrade to Python3.
python-modernize[1] tries to automatically make your codebase compatible with both Python 2 and 3, using 2to3 and six[2] underneath. I haven't tried it yet, though.
[1] https://github.com/mitsuhiko/python-modernize
[2] https://pypi.python.org/pypi/six
[1] https://github.com/mitsuhiko/python-modernize
[2] https://pypi.python.org/pypi/six
Armin handed over development and now Modernize is actively developed at https://github.com/python-modernize/python-modernize
Oh, thanks!
See https://docs.python.org/3/howto/pyporting.html for the most modern approach to porting Python 2 code to Python 2/3 code.
2to3 (comes bundled with Python) will do some of the work for you.
For anyone hesitant because they're worried they will need to support 2.x installations at some point, there is 3to2[1]. You can write for 3.5 only, and then back port only if you need to.
1. https://pypi.python.org/pypi/3to2/1.1.1
1. https://pypi.python.org/pypi/3to2/1.1.1
My main gripe about Python 3 and Python in general is that it really needs a good method of distribution. Sure there's PyInstaller, but it's not straightforward enough and there's a lot of mystery problems that take a lot of time to figure out.
I think this is a nice healthy view to have about Python 2 to 3. Can't wait until python 4!
I seriously doubt 4 will include breaking changes, at least not to the semantics/syntax of the language itself.
What python really needs in 4 is better unified package management and distribution support. They have a lot to learn from the Javascript and Ruby evosystems.
What python really needs in 4 is better unified package management and distribution support. They have a lot to learn from the Javascript and Ruby evosystems.
>They have a lot to learn from the Javascript and Ruby evosystems.
Such as?
I really hope I'm not going to start needing 1,200 dependencies to build a simple app like I would in node.
Such as?
I really hope I'm not going to start needing 1,200 dependencies to build a simple app like I would in node.
Support for local, per-project installs without hacks like virtualenv.
Per-project configurations to map things like external dependencies and project info. Setup.py + MANIFEST is not the answer.
Better suport of versioned deps so multiple versions of the same dep 'just works' when its neessary.
Better package publishing for library maintainers. Have you ever published a pckage to PYPI? It's a huge pain.
Support for Markdown-based documentation on PYPI. ReStructuredText is cool and all but MD os the defacto standard
Standard support for direct install from GitHub without weird hacks.
Pick one unified standard and deprecate the rest. The current 3, distutils, distutils2 and setup.py all suck.
Write some halfway decent docs on how to publish packages. I spend more time on Stack Overflow that the python docs every time I publish a package.
Dependencies are used so heavily in the JS/Ruby ecosystem because it's extremely easy to publish a package and the package manager 'just works'.
Per-project configurations to map things like external dependencies and project info. Setup.py + MANIFEST is not the answer.
Better suport of versioned deps so multiple versions of the same dep 'just works' when its neessary.
Better package publishing for library maintainers. Have you ever published a pckage to PYPI? It's a huge pain.
Support for Markdown-based documentation on PYPI. ReStructuredText is cool and all but MD os the defacto standard
Standard support for direct install from GitHub without weird hacks.
Pick one unified standard and deprecate the rest. The current 3, distutils, distutils2 and setup.py all suck.
Write some halfway decent docs on how to publish packages. I spend more time on Stack Overflow that the python docs every time I publish a package.
Dependencies are used so heavily in the JS/Ruby ecosystem because it's extremely easy to publish a package and the package manager 'just works'.
* Virtualenv isn't a hack.
* Setup.py/MANIFEST.in seems like a pretty clear answer to me.
* I've worked with python for about 10 years and only once have I come across two packages that required conflicting versions that I wanted to deploy in the same environment.
* Yes, I do it pretty frequently.
* "MD os the defacto standard" Huh? According to whom? ReST is fine.
* "Standard support for direct install from GitHub without weird hacks." - stick "git+https://github.com/yourname/yourpackage" in your requirements.txt and off you go.
* Yeah, they should really standardize on pip, but this is relatively minor.
* Use one of the templates.
* Yes, I hear somebody even published a package that checked if a number was negative. And it had a bug.
* Setup.py/MANIFEST.in seems like a pretty clear answer to me.
* I've worked with python for about 10 years and only once have I come across two packages that required conflicting versions that I wanted to deploy in the same environment.
* Yes, I do it pretty frequently.
* "MD os the defacto standard" Huh? According to whom? ReST is fine.
* "Standard support for direct install from GitHub without weird hacks." - stick "git+https://github.com/yourname/yourpackage" in your requirements.txt and off you go.
* Yeah, they should really standardize on pip, but this is relatively minor.
* Use one of the templates.
* Yes, I hear somebody even published a package that checked if a number was negative. And it had a bug.
There are no plans to break backwards-compatibility in Python 4 like Python 3 did with Python 2. If there is any breaking change it will be something like the C API to remove the GIL or something (otherwise it will be standard stuff like removing deprecated modules and functions).
> I seriously doubt 4 will include breaking changes,
If it doesn't, it shouldn't be 4.
If it doesn't, it shouldn't be 4.
You missed the second half of the quote.
Breaking changes to the language syntax/semantics.
I'd expect breaking changes to apply to core APIs and/or to external tooling as both could probably use some revision now that the fundamental issues with the language (ex utf8 support) are mostly fixed.
Breaking changes to the language syntax/semantics.
I'd expect breaking changes to apply to core APIs and/or to external tooling as both could probably use some revision now that the fundamental issues with the language (ex utf8 support) are mostly fixed.
can we please stop writing articles in twitter?
https://twitter.com/hynek/status/699928792752594944
https://twitter.com/hynek/status/699928792752594944
This is the ultimate HN circlejerk comment
>Scala, Haskell, Clojure, Erlang (or Elixir) or Go. And there's also the less popular ones like Haxe and Dart
Are you kidding me? Did you just list the languages you see threads on HN about? Scala and Go I could maybe understand if you needed the performance but the rest of the choices are just bad. Especially if you are trying to start a business instead of a functional programming bootcamp.
Also there is nothing inherently more difficult about deploying and distributing python compared to any of the languages you chose.
>Scala, Haskell, Clojure, Erlang (or Elixir) or Go. And there's also the less popular ones like Haxe and Dart
Are you kidding me? Did you just list the languages you see threads on HN about? Scala and Go I could maybe understand if you needed the performance but the rest of the choices are just bad. Especially if you are trying to start a business instead of a functional programming bootcamp.
Also there is nothing inherently more difficult about deploying and distributing python compared to any of the languages you chose.
Please don't be rude in HN comments.
We detached this subthread from https://news.ycombinator.com/item?id=11124822 and marked it off-topic.
We detached this subthread from https://news.ycombinator.com/item?id=11124822 and marked it off-topic.
Eh, okay, Go is much easier to deploy because it's a single binary (also the same with Dart), but yeah, the rest of the comment makes no sense. Seriously? Dart? Not even Rust?
[deleted]
Since it's basically all anecdotal, I'll say my gazing-into-the-crystal-ball educated-guess feeling that Python 4 will make the entirety of Python about as popular as Perl is today if: it doesn't get rid of GIL and at the same time implement a JIT compiler, and at the same time remain 100% .py source-code compatible with Python 3.
The developers have dug themselves into this mess with frivolous differences ("print" a function, wanting to remove the "%" string operator, etc.), instead of spending the years on more productive features, now is the chance to dig themselves out again.
It's a beautiful language, too bad the development choices were a bit sketchy.
The developers have dug themselves into this mess with frivolous differences ("print" a function, wanting to remove the "%" string operator, etc.), instead of spending the years on more productive features, now is the chance to dig themselves out again.
It's a beautiful language, too bad the development choices were a bit sketchy.
Python 3 is much more than changing the print() statement (which, actually, is a great thing). It's about unicode by default. It's about removing 100 of duplicates way to do things to have only one. It's about making debug much, much easier, while at the same time allowing you to write better code with less mistakes. It's about removing a lot of boilerplate. It's about introducing new syntaxes allowing things not possible in python 2.
If you see only print, you are missing the point.
But good news, there are many projects working to improve the perfs and/or bypass the GIL this year.
If you see only print, you are missing the point.
But good news, there are many projects working to improve the perfs and/or bypass the GIL this year.
> If you see only print, you are missing the point.
The top thing, sometimes the only thing, that makes my scripts written for 2 not work in 3 is the change to print. Adding a print function is fine, I'm all on board with that; removing the statement though seems like pain for no reason. Bringing that up is not missing the point, it's hitting the nail right on the head.
The top thing, sometimes the only thing, that makes my scripts written for 2 not work in 3 is the change to print. Adding a print function is fine, I'm all on board with that; removing the statement though seems like pain for no reason. Bringing that up is not missing the point, it's hitting the nail right on the head.
I think you might want to look into the 2to3 script. It should handle the conversion of your print statements.
> "it doesn't get rid of GIL and at the same time implement a JIT compiler, and at the same time remain 100% .py source-code compatible with Python 3."
Guido could do that if he decided that Python 4 would be based on PyPy (AFAIK, many people thought Python 3 would be based on PyPy too). Compatibility isn't 100% but if it was the main platform I'm sure it'd improve even faster.
Guido could do that if he decided that Python 4 would be based on PyPy (AFAIK, many people thought Python 3 would be based on PyPy too). Compatibility isn't 100% but if it was the main platform I'm sure it'd improve even faster.
There is also pyston, which is a 'competing' LLVM backed JIT that Dropbox is working on. Guido also happens to be working at Dropbox, but I don't know if he's directly involved with pyston or not.
AFAIK, many people thought Python 3 would be based on PyPy too
Don't think so. Some people hoped that Python 3 might be based on Unladen Swallow, which was a different Python JIT that Guido was working on while at Google, but that project never really got off the ground and Google seemed to lose interest in it.
AFAIK, many people thought Python 3 would be based on PyPy too
Don't think so. Some people hoped that Python 3 might be based on Unladen Swallow, which was a different Python JIT that Guido was working on while at Google, but that project never really got off the ground and Google seemed to lose interest in it.
>Guido also happens to be working at Dropbox, but I don't know if he's directly involved with pyston or not.
With the proviso that a lot can change in two years, here's a relevant comment from the Dropbox Pyston announcement [1]:
"Guido's advice has been extremely helpful, but so far we haven't been able to get any code from him"
[1]: https://blogs.dropbox.com/tech/2014/04/introducing-pyston-an...
With the proviso that a lot can change in two years, here's a relevant comment from the Dropbox Pyston announcement [1]:
"Guido's advice has been extremely helpful, but so far we haven't been able to get any code from him"
[1]: https://blogs.dropbox.com/tech/2014/04/introducing-pyston-an...
Why do people always give a single person all the credit? I don't think Guido was ever really involved in Unladen Swallow or that he has any particular interest in JIT compilers.
I don't think I gave him all the credit. We where talking about Guido and I mentioned a relevant project I thought (perhaps erroneously) that used to work on.
pypy still has the GIL.
Hopefully not for long: http://doc.pypy.org/en/latest/stm.html
STM isn't magic. Transactions can and will silently fail and you lose your parallelism. You need additional tools to identify and debug failed transactions and you'll need to learn how to structure your code to avoid failed transactions.
This might turn out to be quite difficult for non-trivial Python applications.
STM is exciting but I think it's important to keep in mind that it's an experiment that while promising might still fail in practice.
This might turn out to be quite difficult for non-trivial Python applications.
STM is exciting but I think it's important to keep in mind that it's an experiment that while promising might still fail in practice.
> "STM isn't magic."
What difference does that make. STM will still enable the removal of the GIL. If an applications is coded that doesn't work with PyPy, that's a separate matter.
What difference does that make. STM will still enable the removal of the GIL. If an applications is coded that doesn't work with PyPy, that's a separate matter.
But first it has to actually work. So far, based on that page, it has suffered the same fate as all the other attempts to remove the GIL; mostly functional, but slower. And the numbers cited on that page put it on the rather slow side for single-threaded code for a GIL removal, too, which is what has always prevented previous efforts from getting merged in.
I remain skeptical that STM is a good fit for this problem, as I was when I first heard about this, and after their work, alas, I have to say I see little evidence to make me change my mind on that. STM is something that probably needs to be baked in from day one, possibly all the way into the language itself, not put on the side of a complicated secondary implementation of Python.
I remain skeptical that STM is a good fit for this problem, as I was when I first heard about this, and after their work, alas, I have to say I see little evidence to make me change my mind on that. STM is something that probably needs to be baked in from day one, possibly all the way into the language itself, not put on the side of a complicated secondary implementation of Python.
> "So far, based on that page..."
From the page... "THIS PAGE IS OLD, THE REST IS ABOUT STMGC-C7 WHEREAS THE CURRENT DEVELOPMENT WORK IS DONE ON STMGC-C8"
This page is probably a better starting point for the current PyPy STM work:
http://pypy.org/tmdonate2.html
From the page... "THIS PAGE IS OLD, THE REST IS ABOUT STMGC-C7 WHEREAS THE CURRENT DEVELOPMENT WORK IS DONE ON STMGC-C8"
This page is probably a better starting point for the current PyPy STM work:
http://pypy.org/tmdonate2.html
I didn't see anything newer there to suggest they've made a lot of progress since that page, though.
Part of the problem is that they're trying to wedge into a really tight window; for all its flaws, the GIL is not all that expensive (relative to how slow Python in general is, anyhow), and trying to do it on PyPy makes their window even smaller. It's a really hard thing to hit, and I'll believe it when I see it. I'm not saying it's impossible. Just, I'll believe it when I see it, and I don't recommend a high degree of confidence that it will occur. Nor do I recommend that you use it as part of your advocacy... it's nowhere near solid enough to be making promises about it to people outside the community.
Part of the problem is that they're trying to wedge into a really tight window; for all its flaws, the GIL is not all that expensive (relative to how slow Python in general is, anyhow), and trying to do it on PyPy makes their window even smaller. It's a really hard thing to hit, and I'll believe it when I see it. I'm not saying it's impossible. Just, I'll believe it when I see it, and I don't recommend a high degree of confidence that it will occur. Nor do I recommend that you use it as part of your advocacy... it's nowhere near solid enough to be making promises about it to people outside the community.
> "I'll believe it when I see it".
You may be interested in this...
"Leysin Winter Sprint (20-27th February 2016)"
"STM (Software Transaction Memory), notably: try to come up with benchmarks, and measure them carefully in order to test and improve the conflict reporting tools, and more generally to figure out how practical it is in large projects to avoid conflicts"
http://morepypy.blogspot.co.uk/2016/01/leysin-winter-sprint-...
> "Nor do I recommend that you use it as part of your advocacy."
I'm an advocate for F#, but I'll encourage the progress of any open source language, strong competition is good for the field as a whole. The question isn't 'Why can't Python be fast?' but rather 'How fast can Python be?', the difference is a difference in attitude.
You may be interested in this...
"Leysin Winter Sprint (20-27th February 2016)"
"STM (Software Transaction Memory), notably: try to come up with benchmarks, and measure them carefully in order to test and improve the conflict reporting tools, and more generally to figure out how practical it is in large projects to avoid conflicts"
http://morepypy.blogspot.co.uk/2016/01/leysin-winter-sprint-...
> "Nor do I recommend that you use it as part of your advocacy."
I'm an advocate for F#, but I'll encourage the progress of any open source language, strong competition is good for the field as a whole. The question isn't 'Why can't Python be fast?' but rather 'How fast can Python be?', the difference is a difference in attitude.
Comments like this are so misguided and yet so common on HN. Adding a freaking JIT to CPython is a ridiculous suggestion that will never happen for a whole load of reasons. Getting rid of the GIL is more noble, but seems impossible to do without breaking compatibility.
Python 4 will just be the next release after Python 3.9.
Python 4 will just be the next release after Python 3.9.
Getting rid of the GIL is more noble, but seems impossible to do without breaking compatibility.
The problem isn't just breaking compatibility, it's performance. There have been a number of experiments with removing the GIL going back (I think) to python 1.4. However every attempt has led to significant performance degradation in some part of the language.
The problem isn't just breaking compatibility, it's performance. There have been a number of experiments with removing the GIL going back (I think) to python 1.4. However every attempt has led to significant performance degradation in some part of the language.
> Adding a freaking JIT to CPython is a ridiculous suggestion that will never happen for a whole load of reasons.
That's a bold statement. Can you elaborate on what those reasons are?
That's a bold statement. Can you elaborate on what those reasons are?
It's not a bold statement, it's one made time and time again by the cPython developers. In short:
1. The Python language itself doesn't lend itself to being JIT compiled - it's too dynamic.
2. cPython is the reference implementation. It's source should be as simple as possible.
3. Only a subset of Python programs would benefit, and the rest would be slower/more memory hungry
and finally,
4. If you want/need a JIT then use PyPy, in all it's glory with all its upsides and downsides.
PyPy is a big project with a lot of very very very smart people working on it. They've done an excellent job at JITting Python (and creating an impressive interpreter framework) but even they couldn't make a JIT that is anywhere near suitable for inclusion in cPython. A bunch of cPython hackers on python-dev have no chance.
tldr: Want a JIT? Use PyPy.
1. The Python language itself doesn't lend itself to being JIT compiled - it's too dynamic.
2. cPython is the reference implementation. It's source should be as simple as possible.
3. Only a subset of Python programs would benefit, and the rest would be slower/more memory hungry
and finally,
4. If you want/need a JIT then use PyPy, in all it's glory with all its upsides and downsides.
PyPy is a big project with a lot of very very very smart people working on it. They've done an excellent job at JITting Python (and creating an impressive interpreter framework) but even they couldn't make a JIT that is anywhere near suitable for inclusion in cPython. A bunch of cPython hackers on python-dev have no chance.
tldr: Want a JIT? Use PyPy.
By "bold" I'm also referring to your use of the words "freaking" and "ridiculous".
1. This was also said about JS before people just went ahead and wrote JITs for it. Are you sure what they're saying isn't "we can't make a JIT for python" rather than "nobody can make one"? Because that's how it turned out with JS.
2. If given the choice, do you think the community would turn down a JIT to avoid making the reference implementation more complex? To whose benefit is that requirement? I've never heard people complain about other languages that they didn't have an easy-to-read reference implementation.
3. I've not heard this reported as a problem in practice for languages that have a JIT. It's usually toy programs where the JIT overhead is significant and they're so small and short-running that neither slowdown or extra memory is usually an issue.
4. Not everything that runs on cPython runs on PyPy so that's not a choice I'm necessarily able to make.
I'm not saying there aren't reasons to not have a JIT in cPython. But it's a choice not a natural law. And it's not ridiculous to suggest that they could make a different choice if they wanted, as JS eventually did.
1. This was also said about JS before people just went ahead and wrote JITs for it. Are you sure what they're saying isn't "we can't make a JIT for python" rather than "nobody can make one"? Because that's how it turned out with JS.
2. If given the choice, do you think the community would turn down a JIT to avoid making the reference implementation more complex? To whose benefit is that requirement? I've never heard people complain about other languages that they didn't have an easy-to-read reference implementation.
3. I've not heard this reported as a problem in practice for languages that have a JIT. It's usually toy programs where the JIT overhead is significant and they're so small and short-running that neither slowdown or extra memory is usually an issue.
4. Not everything that runs on cPython runs on PyPy so that's not a choice I'm necessarily able to make.
I'm not saying there aren't reasons to not have a JIT in cPython. But it's a choice not a natural law. And it's not ridiculous to suggest that they could make a different choice if they wanted, as JS eventually did.
JITs for Javascript aren't comparable; at the time, nobody had C-based extensions for Javascript. It makes it a radically simpler problem.
Remove the "maintaining compatibility for C-based extensions" requirement, and you get, well, PyPy. And you also get a community split that makes 2/3 look trivial. Vast swathes of core Python code and libraries are really not "Python", but Python C-based bindings to custom code or C libraries.
Remove the "maintaining compatibility for C-based extensions" requirement, and you get, well, PyPy. And you also get a community split that makes 2/3 look trivial. Vast swathes of core Python code and libraries are really not "Python", but Python C-based bindings to custom code or C libraries.
Sure there were C-based extensions for JavaScript, that's how it integrates with the DOM. It makes it a little trickier but not a whole lot.
Sorry, perhaps I was unclear. I mean, there were no C-based extensions not under the control of the same project as controlled the JS engine, so one entity can both JIT a JS engine and simultaneously make whatever modifications it took to make them work under the new engine. Python has an enormous amount of external C-based extensions not controlled by any one entity, which constitute huge swathes of core (in importance, not in "under control of the core team") Python libraries. It's difficult to imagine what a JIT looks like that still works with those. You don't have one entity doing both the JIT and all the C extensions.
1. By people you mean "Google", and with a whole lot of money and a room full of some of the smartest people they could find. V8 is a whole department working full time, not a open source community-led effort. JS is also a different beast in a whole lot of ways. No, I am definitely saying "we can't make a JIT for Python that doesn't significantly impact single threaded scripts, retains significant backwards compatibility and doesn't massively complicate the cPython code". Obviously someone can make one, PyPy did, and it would be insane to ignore all of the HUGE problems they encountered (and overcame) while writing a JIT for Python. What makes you think cPython could do better?
2. It would make the reference implementation insanely more complex, to the point where it is no longer a reference implementation but a JIT implementation of Python. That's not cPython's role. For whos benefit is the requirement of a JIT? My apps run fine without them, and I don't need the memory bloat or startup cost associated with a JIT. If the community really wants another JIT they could fork cPython, find a room full of JIT experts and re-do all of the work PyPy has done, with all of the tradeoffs and problems they encountered. Nice idea.
3. "so small and short-running that neither slowdown or extra memory is usually an issue." - this is exactly the situation where a JIT adds a lot of overhead and becomes an issue.
4. So you want the best of both worlds, with no clear idea as to what magical person can make this happen (or even how) and you won't be happy until you can have your cake and eat it?
2. It would make the reference implementation insanely more complex, to the point where it is no longer a reference implementation but a JIT implementation of Python. That's not cPython's role. For whos benefit is the requirement of a JIT? My apps run fine without them, and I don't need the memory bloat or startup cost associated with a JIT. If the community really wants another JIT they could fork cPython, find a room full of JIT experts and re-do all of the work PyPy has done, with all of the tradeoffs and problems they encountered. Nice idea.
3. "so small and short-running that neither slowdown or extra memory is usually an issue." - this is exactly the situation where a JIT adds a lot of overhead and becomes an issue.
4. So you want the best of both worlds, with no clear idea as to what magical person can make this happen (or even how) and you won't be happy until you can have your cake and eat it?
You're making it sound like you're disagreeing with me but it sounds like we're in complete agreement on the main point: it's not impossible to create a cPython JIT. It's a choice whether to do it or not. (And by the way, creating a JIT is a piece of engineering, it's not black magic). It's fair enough if people think that would be the wrong choice, I might even agree actually, but if the cPython developers say, time and time again, that python is inherently unsuitable to be JITted then that's wrong and it prevents that discussion from taking place.
Few things are impossible to code. It's hard to create a JIT for Python and the reference implementation for Python is not the right place to add such a JIT for the reasons stated in the comments above.
It's not that the Python developers preventing discussion, it's just this discussion is always fruitless, has been had many times before and the people invoking the discussion often blame the cPython developers for their response. Nobody wins, it's a waste of time.
It's not that the Python developers preventing discussion, it's just this discussion is always fruitless, has been had many times before and the people invoking the discussion often blame the cPython developers for their response. Nobody wins, it's a waste of time.
"tldr: Want a JIT? Use PyPy."
or spend some time helping with Pyjion to add a C API to CPython for plugging in a JIT of your choice
- https://github.com/Microsoft/Pyjion
or spend some time helping with Pyjion to add a C API to CPython for plugging in a JIT of your choice
- https://github.com/Microsoft/Pyjion
> Getting rid of the GIL is more noble, but seems impossible to do without breaking compatibility.
>
> Python 4 will just be the next release after Python 3.9.
Really? Why even change the major version then, when there are no breaking changes?
Given that many Python programmers seem to be traumatized by the last major version switch, why even bother?
Or is this just for setting up Python 5 can have breaking changes if the switch to 4 is painless?
Really? Why even change the major version then, when there are no breaking changes?
Given that many Python programmers seem to be traumatized by the last major version switch, why even bother?
Or is this just for setting up Python 5 can have breaking changes if the switch to 4 is painless?
> Really? Why even change the major version then, when there are no breaking changes?
Python releases don't follow semver. If Linus can bump the kernel to 4.0 because he felt like it so can Python.
Python releases don't follow semver. If Linus can bump the kernel to 4.0 because he felt like it so can Python.
> If Linus can bump the kernel to 4.0 because he felt like it so can Python.
"Can" is not "should": I mean, Python can go from 3.x to Python 2020, go with year numbers for a while, then have Python XP, then go Python 7, Python 8, Python 8.1, and then skip to Python 10. But there's no good reason to do that.
"Can" is not "should": I mean, Python can go from 3.x to Python 2020, go with year numbers for a while, then have Python XP, then go Python 7, Python 8, Python 8.1, and then skip to Python 10. But there's no good reason to do that.
It may very well never happen, that's a completely possible outcome.
We'll see what it does to Python's popularity.
We'll see what it does to Python's popularity.
So the question for me, for new projects is: why choose Python 3 (over all the other non python options) at all? Not what's better about Python 3 than Python 2, but what's better about Python 3 than those other choices. These other choices all have certain areas where they have great strengths. What is Python 3's strength when compared to these other languages? It's a mutable, imperative, single CPU bound language that can't target the browser and is extremely difficult to distribute and deploy. Or have these issues been fixed?
I don't mean to troll any Python fanboys. I use Python 2 every single day at work and I'm very happy with it. But what does Python 3 offer that other new languages don't? Because when you are starting a new project you can choose anything. Why should I choose Python 3 for my new projects?