PyPy 4.0.0 Released – A Jit with SIMD Vectorization and More(morepypy.blogspot.com)
morepypy.blogspot.com
PyPy 4.0.0 Released – A Jit with SIMD Vectorization and More
http://morepypy.blogspot.com/2015/10/pypy-400-released-jit-with-simd.html
15 comments
I am curious. Usually in numpy basic linear algebra are delegated to BLAS and potentially really fast if configured to use a fast BLAS implementation (OpenBLAS, MKL, etc).
Does the numpy fork of PyPy use the same approach ?
yes. However the delegation only happens if you use the numpy.linalg module, or if numpy has chosen internally to call the numpy.linalg module (i.e. logic in numpy.dot will call out for large ndarrays)
Our vectorizing JIT can use SIMD semantics on all numpy looping calls. For instance, non-matrix multiply A*B or for ndarray + scalar calls
While many numpy users are in the habit of manipulating large square matrices, there is a significant number of users who use small arrays, or process RGB pixels
Our vectorizing JIT can use SIMD semantics on all numpy looping calls. For instance, non-matrix multiply A*B or for ndarray + scalar calls
While many numpy users are in the habit of manipulating large square matrices, there is a significant number of users who use small arrays, or process RGB pixels
yes, but BLAS only helps you with some operations and not others. In fact most of "basic" operations are still done "by hand" in some templated C in numpy. Additionally, doing vectorized operations lazily (which is not present right now in pypy but was at least considered in the past) yields great improvements.
sounds great. I have a scientfic library that relies on a mix of loops and matrix multiplication. I need to find some time to test it with PyPy
Have you also checked out Numba? JIT and type inference, but native to CPython and designed for numerics:
http://numba.pydata.org
http://numba.pydata.org
How up-to-date is this? http://doc.pypy.org/en/latest/windows.html#what-is-missing-f...
I've read that a few times and I always come away confused -- it sounds like a huge fundamental type change needs to be made by someone well versed in the PyPy internals. i.e. not something you'd typically defer to an external contributor.
I'd like to experiment with PyPy and PyParallel, but I'm basically exclusively 64-bit Windows, so it sounds like a non-starter.
I've read that a few times and I always come away confused -- it sounds like a huge fundamental type change needs to be made by someone well versed in the PyPy internals. i.e. not something you'd typically defer to an external contributor.
I'd like to experiment with PyPy and PyParallel, but I'm basically exclusively 64-bit Windows, so it sounds like a non-starter.
Is there any work going towards getting pandas to work with PyPy as well? Its the only thing keeping me from using it.
Kudos to the pypy team! I suspect pypy would benefit from some kind of marketing campaign. It rarely lets me down and yet lots of python devs don't know about it.
The most popular package(s) that don't work with it have been numpy, scipy, etc. When I first encountered pypy, there was a numpypy and I wouldn't be surprised if that's different now. But bottom line, tons of python code out there needs more performance and has no dependency on numpy or any C extensions.
The most popular package(s) that don't work with it have been numpy, scipy, etc. When I first encountered pypy, there was a numpypy and I wouldn't be surprised if that's different now. But bottom line, tons of python code out there needs more performance and has no dependency on numpy or any C extensions.
Also I recently discovered that pypy supports non-ASCII characters pretty well which at least doesn't work with my version of python 2.7.6
In what way?
In variable names, for example. You can do this:
>>> π = 3.14
>>> π = 3.14
File "<stdin>", line 1
π = 3.14
^
SyntaxError: invalid syntax Python 2.7.10 (850edf14b2c7, Oct 29 2015, 17:32:05)
[PyPy 4.0.0 with GCC 5.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>> π = 3.14
File "<stdin>", line 1
π = 3.14
^
SyntaxError: Unknown character
I'm getting same result as scardine:
Python 2.7.8 (default, Sep 30 2014, 15:34:38) [GCC] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> π = 3.14
File "<stdin>", line 1
π = 3.14
^
SyntaxError: invalid syntax
>>>
Python 3.4.1 (default, May 23 2014, 17:48:28) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> π = 3.14
>>> print(π)
3.14You are right, this is Python 3 stuff. I just assumed the parent was talking about Pypy3.
we're trying to address both numpy and C extensions, stay tuned ;-)
you should kickstarter this - and also the python 3 support.
we did kickstart the python 3 support on our website. Donate!
Keep up the good work!
> But bottom line, tons of python code out there needs more performance and has no dependency on numpy or any C extensions.
What they need is nice JIT engines like PyPy and not to switch languages.
If no one invested in improving implementations for modern languages and switched to something lower level all the time, we would still be using Fortran for business applications.
What they need is nice JIT engines like PyPy and not to switch languages.
If no one invested in improving implementations for modern languages and switched to something lower level all the time, we would still be using Fortran for business applications.
We still are using FORTRAN for Business Applications and I fail to see the need to switch them to a slower language to modernize them. Unless the switch is solving a program leave them alone. This is why in R there is still plenty of FORTRAN code.
Personally I LOVE Python but I always feel it is the 2nd best choice. I do use Python but it rarely is the best tool to use for the problem you are solving. Maybe Pypy will turn this around, and I hope so.
Personally I LOVE Python but I always feel it is the 2nd best choice. I do use Python but it rarely is the best tool to use for the problem you are solving. Maybe Pypy will turn this around, and I hope so.
How can SciPy be fast if it is written in an interpreted language like Python?
Actually, the time-critical loops are usually implemented in C or Fortran. Much of SciPy is a thin layer of code on top of the scientific routines that are freely available at http://www.netlib.org/. Netlib is a huge repository of incredibly valuable and robust scientific algorithms written in C and Fortran. It would be silly to rewrite these algorithms and would take years to debug them. SciPy uses a variety of methods to generate “wrappers” around these algorithms so that they can be used in Python. Some wrappers were generated by hand coding them in C. The rest were generated using either SWIG or f2py. Some of the newer contributions to SciPy are either written entirely or wrapped with Cython.
A second answer is that for difficult problems, a better algorithm can make a tremendous difference in the time it takes to solve a problem. So using scipy’s built-in algorithms may be much faster than a simple algorithm coded in C.
http://www.scipy.org/scipylib/faq.html#how-can-scipy-be-fast...
Actually, the time-critical loops are usually implemented in C or Fortran. Much of SciPy is a thin layer of code on top of the scientific routines that are freely available at http://www.netlib.org/. Netlib is a huge repository of incredibly valuable and robust scientific algorithms written in C and Fortran. It would be silly to rewrite these algorithms and would take years to debug them. SciPy uses a variety of methods to generate “wrappers” around these algorithms so that they can be used in Python. Some wrappers were generated by hand coding them in C. The rest were generated using either SWIG or f2py. Some of the newer contributions to SciPy are either written entirely or wrapped with Cython.
A second answer is that for difficult problems, a better algorithm can make a tremendous difference in the time it takes to solve a problem. So using scipy’s built-in algorithms may be much faster than a simple algorithm coded in C.
http://www.scipy.org/scipylib/faq.html#how-can-scipy-be-fast...
This is also why the scipy/numpy crowd doesn't care much about PyPy. In this world Python is the equivalent of ducttape. You don't performance optimize ducttape, you want it to be easy to use and to fix any problem you have.
PyPy is taking a similar approach. The difference is Numpy is using CPython's CAPI as the duct tape where as PyPy is using cffi as the duct tape. Both approaches end up using the same underlying libraries.
Too many people just have the wrong impression, thinking that PyPy plans on re-implementing all the libraries that Numpy and scipy use but that's just completely false.
They have re-implemented the Numpy array so that it can take advantage of the JIT and so that parts of an algorithm implemented In Python that uses Numpy can also be optimized. Unlike what occurs when using Numpy under CPython where the Python code does not get optimized unless it is converted to Cython, C Code, or some alternative to Python to have it be optimized.
Too many people just have the wrong impression, thinking that PyPy plans on re-implementing all the libraries that Numpy and scipy use but that's just completely false.
They have re-implemented the Numpy array so that it can take advantage of the JIT and so that parts of an algorithm implemented In Python that uses Numpy can also be optimized. Unlike what occurs when using Numpy under CPython where the Python code does not get optimized unless it is converted to Cython, C Code, or some alternative to Python to have it be optimized.
I like the quote: "Python is generally the second-best language for everything".
I don't understand numerical analysis as business applications.
For me business applications are what you would use Cobol, Clipper, Java, Delphi, C#, Eifel, ....
Of course Fortran makes sense for number crushing, but that is the language domain, not doing CRUD, ETL or distributed computing stuff (not counting MPI here).
For me business applications are what you would use Cobol, Clipper, Java, Delphi, C#, Eifel, ....
Of course Fortran makes sense for number crushing, but that is the language domain, not doing CRUD, ETL or distributed computing stuff (not counting MPI here).
Accounting, spreadsheets, transaction processing, business intelligence?
Of course those are business applications as well.
Do you intend to refer to a product done in those areas Fortran in modern days, maybe with Fortran 2008?
Outside scientific computing and heavy number crushing algorithms and libraries, I don't see a use for it.
Do you intend to refer to a product done in those areas Fortran in modern days, maybe with Fortran 2008?
Outside scientific computing and heavy number crushing algorithms and libraries, I don't see a use for it.
No, I'm just saying they're business applications. A segment of business applications are numerically focused. Fortran handles them well. So, in that segment, it can be considered a language for business applications with numeric focus.
I'm not saying anyone should use it today lol. Julia and R are the best for that sort of stuff. Way the hell ahead of Fortran. Although I thought about making a nice web application in Fortran just to screw with people who eventually try to extend it and gasp in horror. ;)
http://www.aim.env.uea.ac.uk/aim/info/fortran.pdf
I'm not saying anyone should use it today lol. Julia and R are the best for that sort of stuff. Way the hell ahead of Fortran. Although I thought about making a nice web application in Fortran just to screw with people who eventually try to extend it and gasp in horror. ;)
http://www.aim.env.uea.ac.uk/aim/info/fortran.pdf
I think you guys are saying the same thing. Ie, because they don't need to switch languages, and PyPy is a JIT engine for Python, these tons of Python code would benefit from if the devs were aware of PyPy.
Devs are already aware of PyPy. If it was as easy as being aware, everyone would switch tomorrow. The problem is that tons of code uses C extensions created with swig or python boost and those take a lot of time to port to PyPy. Another problem is that PyPy is sometimes slower/less memory efficient than CPython when you wrote the code that is optimized for CPython. That is abusing dictionaries and very limited OO use.
PyPy has higher memory overhead for very small units of work but usually once you do more than a couple of seconds of work it usually uses significantly less memory than CPython. As far as I'm concern who cares if a Hello World program uses more memory or some small trivial few lines of code. I'm more interested in real world code which my experience has been that PyPy uses significantly less memory than CPython. If you know of a case where this is not true report it as I'm sure the PyPy devs will consider it a bug.
PyPy also has a slower startup time/overhead so very small amounts of work may end up slower than CPython but in general larger chunks of work will be faster if not its a bug and should be reported.
There is currently some work going on to improve the C extension issue. It's currently in its beginning stages and its goals are to provide both a significant amount of compatibility with existing extensions and a significant reduction in the overhead to call out to C extensions over what is currently implemented in PyPy. On the performance side the new approach was said to remove about 40% of the existing overhead. There are tons of corner cases that still need to be worked on for the new approach to become production worthy and there is no guarantee that there will be no show stoppers that come up or that the tail of corner cases that need to be dealt with becomes a nightmare and the approach is abandoned. If this works succeeds, it will be a game changer as suddenly the majority of libraries would then become compatible with PyPy.
PyPy also has a slower startup time/overhead so very small amounts of work may end up slower than CPython but in general larger chunks of work will be faster if not its a bug and should be reported.
There is currently some work going on to improve the C extension issue. It's currently in its beginning stages and its goals are to provide both a significant amount of compatibility with existing extensions and a significant reduction in the overhead to call out to C extensions over what is currently implemented in PyPy. On the performance side the new approach was said to remove about 40% of the existing overhead. There are tons of corner cases that still need to be worked on for the new approach to become production worthy and there is no guarantee that there will be no show stoppers that come up or that the tail of corner cases that need to be dealt with becomes a nightmare and the approach is abandoned. If this works succeeds, it will be a game changer as suddenly the majority of libraries would then become compatible with PyPy.
Indeed that's what I meant and perhaps pjmlp thinks that the feature set in numpy could have been / could be implemented in pure python, now that we have PyPy.
What I mean is that I see PyPy very positive, as a way to stay in Python instead of jumping into C, Go or whatever language.
In terms of performance naysayers only get convinced when someone proves them wrong, not because of what we think might be possible.
So it is important that we have research in GC systems, JIT compilers for dynamic languages, optimizers for FP and LP languages and so on.
Otherwise we might as well keep on using just Assembly.
In terms of performance naysayers only get convinced when someone proves them wrong, not because of what we think might be possible.
So it is important that we have research in GC systems, JIT compilers for dynamic languages, optimizers for FP and LP languages and so on.
Otherwise we might as well keep on using just Assembly.
Numpy is way faster than what you would get if you implementing things in pure Python and executed with PyPy. That's why there is numpypy in the first place.
[deleted]
I am a Java dev, I heard of PyPy a few years ago, I find it amazing and often tell people about it when talking about performance and other related topics.
The thing is I've never found another developer that new about it, something is very wrong...
The thing is I've never found another developer that new about it, something is very wrong...
If you are a Java developer and find PyPy amazing, you should also look at Truffle and Graal http://lafo.ssw.uni-linz.ac.at/papers/2013_Onward_OneVMToRul..., http://chrisseaton.com/rubytruffle/, https://bitbucket.org/ssllab/zippy/src/700f88dc2a7d2011258ae...
Is PyPy going to support Python 3? We kind of need the ecosystem to move forward, and Python 3 has a lot of niceties...
Doesn't it already? 3.2.5 it seems.
http://pypy.org/download.html
http://pypy.org/download.html
Yes - it's getting there and you can help it get there sooner by donating http://pypy.org/py3donate.html
python 3.4 please ;_;
The problem, which they keep mentioning over and over, is that companies who actually donate have zero interest in a python 3 version. While they are a non profit, they have to pay the bills somehow.
Chicken and egg... But if there was good Python 3 support maybe more companies would contribute.
[deleted]
I too want 3.3+ support but I know some claim that there's very little demand from people actually doing something with PyPy for Python 3 support. I really think it may be a chicken and egg problem: that is, support will generate demand.
In any case the 3.3 branch seemed good when I compiled it last, it'd be nice to see a partial release.
In any case the 3.3 branch seemed good when I compiled it last, it'd be nice to see a partial release.
"puhleeze give us Python 3...." repeated everywhere in this comment thread, yet the authors of pypy keep responding with the same line: pay some money and we'll do it. This speaks to what the real industry users of Python already know: 2.7 is where it's at, and they're prepared to pay for that, and not 3.x. Sure there'll be a bunch of people waving the 3 flag and saying "I use it" etc, but the bottom line is the bottom line: money. 2 attracts the dough.
If I were the PyPy guys I'd just announce that PyPy was now officially a fork of 2.7. This would bring all the numerical guys with them, all the corpos with big investments in 2 would fund them, and we could then let the 3.x zealots go off on their own tangent.
Anybody who likes what 3.x brings to the table should really spend a weekend learning Go, which does it all better, and faster.
I've been saying in these threads for ages that vectorization is the big win in Python (thank you PyPy), and not hobby projects like asyncio (miles better elsewhere), or (eyes rolling) type annotations.
If I were the PyPy guys I'd just announce that PyPy was now officially a fork of 2.7. This would bring all the numerical guys with them, all the corpos with big investments in 2 would fund them, and we could then let the 3.x zealots go off on their own tangent.
Anybody who likes what 3.x brings to the table should really spend a weekend learning Go, which does it all better, and faster.
I've been saying in these threads for ages that vectorization is the big win in Python (thank you PyPy), and not hobby projects like asyncio (miles better elsewhere), or (eyes rolling) type annotations.
Yes, this is one of those features that should be targeting the newest release, to provide a reason to upgrade, and not support lock-in to 2.7.
The value of PyPy is that it is a thing you can just drop in to make your code faster. If PyPy wants to gain any traction they pretty much have to concentrate on the version with the most existing code.
Python 3 is the future. There's a chance that when Pypy finally catches up with Python 2.7 (numpy, C extensions etc), it will be obsolete.
Python3 is a language. Python2 is also a language. Python3 might someday become wildly popular, but that doesn't magically make Python2 go away just because the name sounds the same.
From [PEP 373]:
> The End Of Life date (EOL, sunset date) for Python 2.7 has been moved five years into the future, to 2020. This decision was made to clarify the status of Python 2.7 and relieve worries for those users who cannot yet migrate to Python 3. See also PEP 466.
> This declaration does not guarantee that bugfix releases will be made on a regular basis, but it should enable volunteers who want to contribute bugfixes for Python 2.7 and it should satisfy vendors who still have to support Python 2 for years to come.
> There will be no Python 2.8 (see PEP 404).
Take note that the initial date was set to 2015 [2] and it was delayed only last year.
> The End Of Life date (EOL, sunset date) for Python 2.7 has been moved five years into the future, to 2020. This decision was made to clarify the status of Python 2.7 and relieve worries for those users who cannot yet migrate to Python 3. See also PEP 466.
> This declaration does not guarantee that bugfix releases will be made on a regular basis, but it should enable volunteers who want to contribute bugfixes for Python 2.7 and it should satisfy vendors who still have to support Python 2 for years to come.
> There will be no Python 2.8 (see PEP 404).
Take note that the initial date was set to 2015 [2] and it was delayed only last year.
[PEP 373]: http://legacy.python.org/dev/peps/pep-0373/
[2]: https://hg.python.org/peps/rev/76d43e52d978> We would also like to thank our contributors (7 new ones since PyPy 2.6.0) and encourage new people to join the project. PyPy has many layers and we need help with all of them
This doesn't match my experience of the PyPy project. I found a tiny bug in the stdlib matching against CPython, went into IRC to ask a question about test running to be sure I got it right and was quickly engaged in conversation about why I was running core tests. Next thing I know my small bug has been fixed by a core developer and my chance to contribute is gone.
I'll stick to being a user.
This doesn't match my experience of the PyPy project. I found a tiny bug in the stdlib matching against CPython, went into IRC to ask a question about test running to be sure I got it right and was quickly engaged in conversation about why I was running core tests. Next thing I know my small bug has been fixed by a core developer and my chance to contribute is gone.
I'll stick to being a user.
This is very very disingenuous.
Here are the public IRC logs for what I assume you're referring to (which is from July 2014 by the way):
https://botbot.me/freenode/pypy/2014-07-02/?msg=17362202&pag...
You found a bug in an old version of PyPy. A core developer told you you might want to try a new one. No one stole your shot out from under you.
You would undoubtedly be welcomed to contribute. Don't post FUD.
Disclaimer: I am not a PyPy core dev but I do work closely with them.
Here are the public IRC logs for what I assume you're referring to (which is from July 2014 by the way):
https://botbot.me/freenode/pypy/2014-07-02/?msg=17362202&pag...
You found a bug in an old version of PyPy. A core developer told you you might want to try a new one. No one stole your shot out from under you.
You would undoubtedly be welcomed to contribute. Don't post FUD.
Disclaimer: I am not a PyPy core dev but I do work closely with them.
> Alex_Gaynor - simpson: import operator; operator.isNumberType is the check you want
> I'll push this change quickly
> Alex_Gaynor - uhh, wait the check seems to exist?
I dunno, I kinda see both sides here. First, I don't think the devs were off-base at all here. They were friendly and looking to solve problems. However, they did seem to want to simply solve the problem, while Matthew did state that he was looking at this as a way to contribute.
The whole interaction could have been improved with a little encouragement at the end, and possibly a recommendation of an 'easy' bug that was currently in need of a fix. It was fairly clear that there was an enthusiastic new contributor, so a little effort in this direction may have been warranted.
I certainly would consider this an example of being unfriendly or unwelcoming.
> I'll push this change quickly
> Alex_Gaynor - uhh, wait the check seems to exist?
I dunno, I kinda see both sides here. First, I don't think the devs were off-base at all here. They were friendly and looking to solve problems. However, they did seem to want to simply solve the problem, while Matthew did state that he was looking at this as a way to contribute.
The whole interaction could have been improved with a little encouragement at the end, and possibly a recommendation of an 'easy' bug that was currently in need of a fix. It was fairly clear that there was an enthusiastic new contributor, so a little effort in this direction may have been warranted.
I certainly would consider this an example of being unfriendly or unwelcoming.
I apologise that I mis-remembered and characterised intending to fix a bug, investigating, and finding out it was already fixed as just fixing a bug, however I believe the rest stands.
The fact that I wanted to run tests and couldn't, and the team went on to why I wanted to run tests and looked to solve my problem is my complaint.
Also, to be clear, I'm not trying to say the PyPy channel is unfriendly, it certainly isn't. It's just not set up to help potential new contributors get started.
The fact that I wanted to run tests and couldn't, and the team went on to why I wanted to run tests and looked to solve my problem is my complaint.
Also, to be clear, I'm not trying to say the PyPy channel is unfriendly, it certainly isn't. It's just not set up to help potential new contributors get started.
On the flip side of this thread, some projects have started to adopt specific tagging in their issue systems for folks looking to get started as contributors. I really like this as it's clear communication to both new contributors and to existing contributors that the issue in question is an on-ramp for the project.
Here's Neovim's "entry-level" label, for example:
https://github.com/neovim/neovim/issues?q=is%3Aopen+is%3Aiss...
Here's Neovim's "entry-level" label, for example:
https://github.com/neovim/neovim/issues?q=is%3Aopen+is%3Aiss...
Actually pypy tries to do the same, bitbucket does not support tagging very well though (but looking for "easy" or something in the issue tracker helps). I am personally sorry if the poster had an experience like that, IRC is difficult to communicate over, it must have been a misunderstanding
As another example it seems that Rust does the same:
https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Ais...
https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Ais...
Servo inherited that policy too: https://github.com/servo/servo/issues?q=is%3Aopen+is%3Aissue...
That's a shame, it can sometimes be tough to pick out a good manageable intro-bug on an open source project you want to contribute to. Perhaps it was a new-ish core dev who was also keen to cut his teeth on small/doable bugs?
Not everyone can get it right all the time.
Fixed a bug the same day someone asked about it and some potential contribute blame you for not letting them fix entry level bug.
Answering people with "patch welcome" and people blame you for not fixing it.
Fixed a bug the same day someone asked about it and some potential contribute blame you for not letting them fix entry level bug.
Answering people with "patch welcome" and people blame you for not fixing it.
Did you tell them that you wanted to contribute, rather than get a bug fixed?
Is this a complaint or a compliment to the developers?
He didn't want to report a bug, he wanted to contribute to the project. I can think of a few reasons why. They lost a new potential committer by "being pragmatic" about it, but ignoring inter-personal relationships, like so often.
Personally I'd be quite excited to see a bug I reported fixed the same-day - of course people's expectations are different it's not always easy to balance these things.
One of the rewards of participating in OSS is to make your name visible, this can then be used as creds to gain commit rights or to talk at conferences as well as proof of competence when looking for a job.
Interestingly, I've had the same experience when reporting and asking how to contribute to Python 3, not by a core dev but somebody "stealing" my bug and quickly submitting a patch!
Interestingly, I've had the same experience when reporting and asking how to contribute to Python 3, not by a core dev but somebody "stealing" my bug and quickly submitting a patch!
Something similar happened that left me with mixed feelings recently. I recently came up with a new feature idea for Python, and a core dev ran with it. I was very happy at first, but then he wrote a pep, implemented it, and got all the congrats.
I had hoped to help, and put something on my resume, but was shut out. "Thanks for improving my pep" on a mailing list is all the credit I got (and work I was able to do on it). Kind of disappointing, though I will enjoy the feature.
Sometimes person reporting an issue along with the code, doesn't get it quite right and core dev can immediately see correct solution and can't make himself to wait until original reporter makes it right... of course it doesn't happen often but it does.
One of the project I participated the most was ansible, and in a case like this, @mpdehaan2 would just push back and comment until your PR was acceptable, which was encouraging and making you contribute more.
The open source community is not there to provide you with cred and visibility. It's there to so software can be developed in a way that certain people think is the best way to develop software.
I can understand that, I certainly would have been stoked about a bug report being fixed on the same day; I hadn't reported the bug though. I was working on providing a bug report with a patch alongside it and (perhaps foolishly) mentioned the bug informally as context as to why I was doing this.
Without being privy to the entire context, what I think might've happened is that your discussion caught the interest of a core developer who then picked it up and ran with it. I doubt they intended to step on your toes as a consequence of fixing the problem. Hackers tend to be intrigued by problems, including small ones, and especially ones that cause software behavior to deviate from expectations.
The other side of the coin is that without a patch, the developers have no idea what your motives or capabilities are. They can't read minds, so if you pique their interest sufficiently, they're very likely to take it upon themselves to do the legwork. No malice intended. I should hope you'll change your mind about contributing to FOSS projects after reading this thread, because I think it's counterproductive to take this sort of thing personally (I don't know if you are, but it appears to me that you're unhappy about this at the very least).
That said, there aren't many projects that post credits or thanks to people who have discovered bugs or whose line of questioning has lead to fixing application behavior. While it's extra work and isn't always feasible, it can encourage users to pitch in if they know their name might appear in some credits (even if it's only per-release). Then there's the question of how big the problem needs to be to credit someone...
The other side of the coin is that without a patch, the developers have no idea what your motives or capabilities are. They can't read minds, so if you pique their interest sufficiently, they're very likely to take it upon themselves to do the legwork. No malice intended. I should hope you'll change your mind about contributing to FOSS projects after reading this thread, because I think it's counterproductive to take this sort of thing personally (I don't know if you are, but it appears to me that you're unhappy about this at the very least).
That said, there aren't many projects that post credits or thanks to people who have discovered bugs or whose line of questioning has lead to fixing application behavior. While it's extra work and isn't always feasible, it can encourage users to pitch in if they know their name might appear in some credits (even if it's only per-release). Then there's the question of how big the problem needs to be to credit someone...
I'm a long-term FLOSS contributor, but I've also served on the boards of 501(c)(3)s, I've been a mentor for Google Summer of Code for most of the times the programme has run and I've run workshops at my local university about getting involved in open source. I know what is involved in making a community welcoming to new contributors. If a project is putting an explicit call for contributions in updates that implies they're not getting as many as they'd like, which fits my experiences.
What you say about hackers getting nerd-sniped is true, but "that's just how coders are" certainly isn't. If you're a core developer for a project and you want the community to grow rather than diminish it's your responsibility to make it socially welcoming as well as having a low technical barrier to entry. I personally have spent many hours responding to emails about problems I could have fixed trivially, but if I'd done that the potential new contributor may not have grown into someone that's regularly fixing bugs I can't fix trivially. I'm not taking the unwelcoming attitude personally, I'm acting accordingly. If I can't get the information from a community I need to participate but I can get them to do the work by just throwing it over the fence I'd be crazy to continue to try and help them unless I have a very strong reason to outside of requiring fixes.
As to your point about reading minds, this is specifically mentioned in my initial comment. I asked about a problem running tests, and when I provided the context that I was asked for (I'm trying to put together a patch for this bug) there should have been no question as to motivation.
What you say about hackers getting nerd-sniped is true, but "that's just how coders are" certainly isn't. If you're a core developer for a project and you want the community to grow rather than diminish it's your responsibility to make it socially welcoming as well as having a low technical barrier to entry. I personally have spent many hours responding to emails about problems I could have fixed trivially, but if I'd done that the potential new contributor may not have grown into someone that's regularly fixing bugs I can't fix trivially. I'm not taking the unwelcoming attitude personally, I'm acting accordingly. If I can't get the information from a community I need to participate but I can get them to do the work by just throwing it over the fence I'd be crazy to continue to try and help them unless I have a very strong reason to outside of requiring fixes.
As to your point about reading minds, this is specifically mentioned in my initial comment. I asked about a problem running tests, and when I provided the context that I was asked for (I'm trying to put together a patch for this bug) there should have been no question as to motivation.
> I hadn't reported the bug though.
I don't think its reasonable to expect a projects core developers, when they become aware of a bug through mechanisms other than formal bug reports, to deliberately stall confirming and resolving the problem because the person raising the issue is entitled to a crack at fixing it.
I don't think its reasonable to expect a projects core developers, when they become aware of a bug through mechanisms other than formal bug reports, to deliberately stall confirming and resolving the problem because the person raising the issue is entitled to a crack at fixing it.
You can do something correct and still be discouraging to people who would like to contribute.
How is reporting a bug not a contribution? Now, I don't know about this particular bug, but in most cases finding the bug is where the work is. Checking in the fix is trivial.
Guys, what's a "trace optimizer"? Google is failing me by returning results about MySQL.
Traces are an artifact produced by some JITs.
Basically you observe loops and loops that satisfy certain criteria are traced, that is all operations performed within one iteration are stored. Such a trace is then optimized, compiled to machine code and then executed for every iteration when the loop is encountered again.
Basically you observe loops and loops that satisfy certain criteria are traced, that is all operations performed within one iteration are stored. Such a trace is then optimized, compiled to machine code and then executed for every iteration when the loop is encountered again.
Search https://www.google.com?q=trace+optimizer+compilers
According to wikipedia: https://en.wikipedia.org/wiki/Tracing_just-in-time_compilati...
Tracing just-in-time compilation is a technique used by virtual machines to optimize the execution of a program at runtime. This is done by recording a linear sequence of frequently executed operations, compiling them to native machine code and executing them. This is opposed to traditional just-in-time (JIT) compilers that work on a per-method basis.
According to wikipedia: https://en.wikipedia.org/wiki/Tracing_just-in-time_compilati...
Tracing just-in-time compilation is a technique used by virtual machines to optimize the execution of a program at runtime. This is done by recording a linear sequence of frequently executed operations, compiling them to native machine code and executing them. This is opposed to traditional just-in-time (JIT) compilers that work on a per-method basis.
They're referring to a tracing JIT[1], I believe.
[1]:https://en.wikipedia.org/wiki/Tracing_just-in-time_compilati...
[1]:https://en.wikipedia.org/wiki/Tracing_just-in-time_compilati...
I assume it's a reference to tracing JIT (https://en.wikipedia.org/wiki/Tracing_just-in-time_compilati...).
It's a JIT that traces runtime execution, rather than looking at whole method or function bodies. In particular it eliminates control flow, which simplifies optimisation.
It's a JIT that traces runtime execution, rather than looking at whole method or function bodies. In particular it eliminates control flow, which simplifies optimisation.
> We would also like to thank our contributors (7 new ones since PyPy 2.6.0)
That's very good news, since monetary contributions don't seem to be abunding (none of the goals have been met according to pypy.org). Maybe it's a marketing issue, like said before?
Thanks for all the effort made.
That's very good news, since monetary contributions don't seem to be abunding (none of the goals have been met according to pypy.org). Maybe it's a marketing issue, like said before?
Thanks for all the effort made.
I wish it had faster startup time, which really matters for Mercurial.
Working on it (even wrote 2 blog posts recently about it). Mercurial is an interesting benchmark/use case for us.
Yay! Glad to hear it. Using Python makes me so envious of Perl's startup time. Maybe some day we can have that in Python too...
Where are those blog posts?
Where are those blog posts?
Are there recent analysis/benchmarks of startup time? This is the major blocker preventing us switching to PyPy, since we run a lot of short-lived processes.
My experience with PyPy was that it was faster for mapreduce jobs without JSON parsing and slower for JSON-heavy jobs. Because, I assume, CPython benefits from a C implementation of simplejson and PyPy doesn't. Does anyone know if this has changed? (I'll probably install 4.0 and see for myself, but curious if anyone knows what I should expect.)
We did put some effort into making built-in json faster. If you're using an optimized version like uJson, it'll be faster under cpython so far, but it all depends whether there is more processing or more parsing involved. We're happy to help you with benchmarks though
I've only used simplejson. A mapreduce job could be reading billions of lines and possibly doing very simple logic on each line, so the parsing/encoding is a big percentage of the processing, sometimes nearly all of it. I'll try 4.0 and see how it does now.
Try builtin json instead of simplejson for pypy (it's API-equivalent)
On big jobs (millions of json input/output) I have found ujson significantly faster than json + pypy.
yop, as I said, ujson is probably your best bet if all you do is parse json.
If you are on python3 python-rapidjson is faster and safer than uJSON:
https://github.com/kenrobbins/python-rapidjson
https://github.com/kenrobbins/python-rapidjson
>>> data2 = 1.23456789e-13
>>> rapidjson.loads(rapidjson.dumps(data2))
1.23456789e-13
>>> ujson.loads(ujson.dumps(data2))
0.0One has to be careful with strings in pypy, because they don't have a string-builder optimization. Or maybe they do now?
It's hard to tell from these posts under what circumstances will SIMD vectorization kick in outside of numpy. E.g., will it work for plain old list comprehensions, which are everywhere in my code? That would be awesome.
Generally speaking "no", but there are a few cases where it might, e.g. if you're using arrays/numpy arrays in list comprehensions. We could not get it to run on normal lists for a bit obscure reasons.
I did a small test, here's the gist:https://gist.github.com/arnists/a9f19af5d9469549020e
Seems like a huge improvement, don't know if it's the recent additions or in general pypy vs cpython but it's enough of a speed bump to sit up and take note.
Seems like a huge improvement, don't know if it's the recent additions or in general pypy vs cpython but it's enough of a speed bump to sit up and take note.
That's why you don't use pure python for this.
Here is your test done using numpy
import numpy as np
%timeit sum(np.sum(np.random.randint(0, 10000000, 5000)) for i in range(5000))
1 loops, best of 3: 586 ms per loopAccording to data I'm getting a 20X performance increase from the rendering engine in Django. I know this is the least of the time I spend but it's significant enough to take a deeper look.
Nice! Does this work with scipy now?
so the next version of pypy3 should be 5.0.x, lol
The good news never stops with these guys. PyPy is so good that I've stopped considering myself a "Python programmer" to a "PyPy programmer" over the last few years.
Best of every world. I get my preferred language and eliminate CPU performance worries.
For me, targeting development on PyPy has taken the place of migrating to Python3. Which just moves too fast in effort to pull people over, Python is too big for that strategy. At this point, people want speed with a great language, they want PyPy! The PyPy team has pretty much enabled the perfect development stack. I use PyPy+Django+gunicorn+Nginx and no problems.
Keep up the great work.
For me, targeting development on PyPy has taken the place of migrating to Python3. Which just moves too fast in effort to pull people over, Python is too big for that strategy. At this point, people want speed with a great language, they want PyPy! The PyPy team has pretty much enabled the perfect development stack. I use PyPy+Django+gunicorn+Nginx and no problems.
Keep up the great work.
I'll go one step further and advocate for the PyPy developers please not to port to Python 3. Here we have a project which has clearly become the go to for 2.x people who would like Python to move forward in a way which makes a real difference to them, and there is an ideal opportunity here for this to become the de-facto continuation of the 2.x line, with a real future, as an alternative to the 3.x forcefeed.
What would be the point of this? Are you suggesting PyPy forks the Python language itself? PyPy hasn't implemented new core language features, and I don't see them doing so in the future. Python 3 is actively developed, fixing many warts in the language and adding great new features.
As annoying as the transition to Python 3 is, it's the future of the language. I was actually one of the curmudgeons who stuck with 2.7 and grumbled about not wanting to switch until a few months ago. I've accepted that I need to, at the very least, start new projects in Python 3 wherever possible. The vast majority of popular third party libraries have been ported over at this point. Python 2's death is inevitable, even if it will take many more years.
As annoying as the transition to Python 3 is, it's the future of the language. I was actually one of the curmudgeons who stuck with 2.7 and grumbled about not wanting to switch until a few months ago. I've accepted that I need to, at the very least, start new projects in Python 3 wherever possible. The vast majority of popular third party libraries have been ported over at this point. Python 2's death is inevitable, even if it will take many more years.
That is exactly what I am suggesting. There is no urgent necessity to add any new core language features to 2.7. That's what you 3 guys just don't get. I certainly don't need asyncio as much as I need vectorised numerics, and if I needed asyncio, I had at least 5 libraries already. I most definitely will never be type annotating Python. Cloud cuckoo land.
Let me warn you. If you persist with your propaganda on Python3, then GOLANG will be the future of Python. Does that register with you guys?
Nobody who uses Python 2.7 all day every day thinks it has any of these imaginary "worts" you 3.x guys like to pitch incessantly.
Let me warn you. If you persist with your propaganda on Python3, then GOLANG will be the future of Python. Does that register with you guys?
Nobody who uses Python 2.7 all day every day thinks it has any of these imaginary "worts" you 3.x guys like to pitch incessantly.
> you 3 guys ... Cloud cuckoo land ... your propaganda ... Does that register with you guys?
Your comments have been crossing into incivility. Please don't do that; it breaks the site rules and leads to tedious flamewars. I'm sure you can make your argument civilly and substantively if you try.
Your comments have been crossing into incivility. Please don't do that; it breaks the site rules and leads to tedious flamewars. I'm sure you can make your argument civilly and substantively if you try.
I was overly aggressive. It's regrettable. But every point I made has been seriously contemplated, and is in my opinion worthy of consideration. The extent to which the 2.7 community and its arguments, are treated with derision by the 3.x people, makes some form of vigorous defence sometimes necessary. I'll add that unlike a large silent group of Python people who basically agree with me, I am not afraid to spend karma points speaking up against the received wisdom unambiguously and forcefully.
Now that I have made all my points, it's bye-bye.py. I won't comment on Python again.
Now that I have made all my points, it's bye-bye.py. I won't comment on Python again.
> It's regrettable.
That makes it sound like you had no control.
Being rude is not "speaking up against the received wisdom", it's just being rude, and it's against the rules here. In the future, please post civilly and substantively or not at all.
https://news.ycombinator.com/newsguidelines.html
https://news.ycombinator.com/newswelcome.html
That makes it sound like you had no control.
Being rude is not "speaking up against the received wisdom", it's just being rude, and it's against the rules here. In the future, please post civilly and substantively or not at all.
https://news.ycombinator.com/newsguidelines.html
https://news.ycombinator.com/newswelcome.html
> Let me warn you. If you persist with your propaganda on Python3, then GOLANG will be the future of Python. Does that register with you guys?
This abstract future of Python has already been ceded to something that is not Python, just as once the abstract future of Perl was Python. Ruby once tried to take the mantle but failed. Go is now taking a lot of mindshare (from 2 or 3 more I wonder?) and I agree the Python 3 crowd just makes Go more likely to succeed, which is terrifying since it's probably the worst possible choice. My message is simply despair and accept the inevitable: Python will fade -- by all means keep using it, people still use Perl, scientist will be using it for a long time -- but the rest of the software world will pass you by. If you care about what the rest of the world is doing at all, you can help make sure the future passes to better general purpose high level languages like Nim or Clojure or OCaml, and for specific niches like scientific computing goes to the likes of Julia eventually. Anything but Go.
This abstract future of Python has already been ceded to something that is not Python, just as once the abstract future of Perl was Python. Ruby once tried to take the mantle but failed. Go is now taking a lot of mindshare (from 2 or 3 more I wonder?) and I agree the Python 3 crowd just makes Go more likely to succeed, which is terrifying since it's probably the worst possible choice. My message is simply despair and accept the inevitable: Python will fade -- by all means keep using it, people still use Perl, scientist will be using it for a long time -- but the rest of the software world will pass you by. If you care about what the rest of the world is doing at all, you can help make sure the future passes to better general purpose high level languages like Nim or Clojure or OCaml, and for specific niches like scientific computing goes to the likes of Julia eventually. Anything but Go.
I'm currently registered on the Ocaml mooc ;-) Very intrigued by Nim too.
Yes I find Golang to be dry and uninspiring, but as per one of my other comments, I get 20x the performance, and the concurrency model beats Asyncio hands down. Honestly I needed to get 360 gigabytes of JSON into Cassandra and Python was going to take 6 days (there is some light conditionality on each datapoint preventing a raw dump). I took 1.5 of those days to learn how to do it in Golang, started it at midday today, and estimate it'll be done when I get into work tomorrow morning. Sometimes I just need to get stuff done.
EDIT: Just re-read your post and cannot help but agree that Python's demise is already telegraphed. History. Perl->Python->(?) is a nice analogy. Python won't die for sure, but it will not flourish like it did in the past decade. And yes I think it's good advice to look to more ambitious languages than Go, even if Go solves many "today" problems quite well. Personally wish Clojure would come off the JVM but am looking into Ocaml, Elixir, and when I need imperative, good ol' C wrapped up in Nim. All good candidates, though I really wish something would come up and go "fully vectorized" for the brave new world of GPUs. Personally need a REPL which is why Ocaml wins for now though Spark looks to me to be the "new R" for data science so I shouldn't exclude Scala (JVM notwithstanding). Decisions decisions...
Yes I find Golang to be dry and uninspiring, but as per one of my other comments, I get 20x the performance, and the concurrency model beats Asyncio hands down. Honestly I needed to get 360 gigabytes of JSON into Cassandra and Python was going to take 6 days (there is some light conditionality on each datapoint preventing a raw dump). I took 1.5 of those days to learn how to do it in Golang, started it at midday today, and estimate it'll be done when I get into work tomorrow morning. Sometimes I just need to get stuff done.
EDIT: Just re-read your post and cannot help but agree that Python's demise is already telegraphed. History. Perl->Python->(?) is a nice analogy. Python won't die for sure, but it will not flourish like it did in the past decade. And yes I think it's good advice to look to more ambitious languages than Go, even if Go solves many "today" problems quite well. Personally wish Clojure would come off the JVM but am looking into Ocaml, Elixir, and when I need imperative, good ol' C wrapped up in Nim. All good candidates, though I really wish something would come up and go "fully vectorized" for the brave new world of GPUs. Personally need a REPL which is why Ocaml wins for now though Spark looks to me to be the "new R" for data science so I shouldn't exclude Scala (JVM notwithstanding). Decisions decisions...
It will be interesting to see what the final winner of GPU programming looks like. There's also the possibility of FPGAs making their way into consumer hardware, and right now the pure-Python library MyHDL is a great choice for those. There are a few attempts at abstracting over OpenCL like https://github.com/eholk/harlan, and some intriguing domain specific attempts like http://halide-lang.org/ but I think it will be a while yet before something takes over. I also don't think whatever it is can succeed without corporate backing.
Python Numba compiler (fast as julia) has a fair amount of Cuda and AMD HSA capability.
You're looking at this from the SV startup echo-chamber perspective.
People use Python because the language gets out of the way fairly quickly to let you get things done fairly quickly. Sure, it isn't a rigorous engineering language, but the ratio of small projects that just need something good enough and easy to work with to large/complex projects requiring careful engineering is staggeringly large. While Python is definitely cooling off; people have been working with it for long enough to want something better - unfortunately there just isn't anything on the radar that is significantly enough better to overcome Python's third party library momentum at the moment.
Believe me, I'd love a language with a really flexible/optional functional type system, better metaprogramming facilities, a better concurrency design, slightly more consistent syntax, etc - just as long as it is still easy to think in and convenient to use like Python.
People use Python because the language gets out of the way fairly quickly to let you get things done fairly quickly. Sure, it isn't a rigorous engineering language, but the ratio of small projects that just need something good enough and easy to work with to large/complex projects requiring careful engineering is staggeringly large. While Python is definitely cooling off; people have been working with it for long enough to want something better - unfortunately there just isn't anything on the radar that is significantly enough better to overcome Python's third party library momentum at the moment.
Believe me, I'd love a language with a really flexible/optional functional type system, better metaprogramming facilities, a better concurrency design, slightly more consistent syntax, etc - just as long as it is still easy to think in and convenient to use like Python.
> Believe me, I'd love a language with a really flexible/optional functional type system, better metaprogramming facilities, a better concurrency design, slightly more consistent syntax, etc - just as long as it is still easy to think in and convenient to use like Python.
As someone who, among dynamic languages, dismissed Perl quite a while back as not as useful as Python and Ruby, I'm starting to think Perl 6 may be the language that hits that spot.
As someone who, among dynamic languages, dismissed Perl quite a while back as not as useful as Python and Ruby, I'm starting to think Perl 6 may be the language that hits that spot.
No please! Julia Julia Julia fills all those, but with beatiful python like syntax, lisp like macros when you wnat them, go like channels, distributed capabilities and leading class numerical stuff. Also webserver with a front end that you can build with polymer in Pure Julia.
http://escher-jl.org/
http://escher-jl.org/
Julia!!!!! The answer to all these problems, both for general and science programming.
I wonder how much faster pypy would be in parsing 360 GB of data.
I use Python 2.7 all day every day and I definitely still notice the warts and find them irritating. They are minor warts, and almost never affect my productivity, but they're definitely things that should have been, and are, fixed. super(what do I pass here again?), old vs. new style classes, integer division, input, print, exception bubbling/chaining, nonsense compares between different types, True/False/None as regular variables, no "nonlocal"... Then there are the features I can't use, like extended unpacking and merging, better yielding, function annotations, and lots of improvements to the standard library in terms of their API and functionality.
I'm not a fan of asyncio, but the other stuff is quite awesome. It's not compelling enough to port some of my older Python 2.7 projects, and isn't a revolution of the language or anything, but they're good improvements. And even if they weren't good improvements, Python 3 is the future, and Python 2.7 simply won't be a viable option for a lot longer.
I'm not a fan of asyncio, but the other stuff is quite awesome. It's not compelling enough to port some of my older Python 2.7 projects, and isn't a revolution of the language or anything, but they're good improvements. And even if they weren't good improvements, Python 3 is the future, and Python 2.7 simply won't be a viable option for a lot longer.
I would suggest toning down the arrogance. Just because you don't personally need some features does not mean that people who want the language to progress and not stagnate are idiots.
That's rich when it is the 2.7 users like myself who are constantly bombarded with arrogance and condescension. Nobody wants Python to stagnate. The argument is which direction it should take.
You contribute the same borderline FUD in every thread that is related to Python. It's unfortunate that Python is not the language you want it to be, but consider just how dismissive you are of the needs of others - frankly, it comes across as arrogant.
FUD? I have >100k lines of highly sophisticated numerical analysis Numpy/Python code in fixed income which has cost literally millions of dollars. I'm not going to let that investment get toyed with by the webified 3.x crew without vigorously stating my point.
Could Python 3 host the features you want? If it did, would you consider migrating your code-base to it? I'm positive that if your ideas have merit and you share them on the python-ideas mailing list, you'll get a good discussion out of it at the very least.
> Nobody who uses Python 2.7 all day every day thinks it has any of these imaginary "worts" you 3.x guys like to pitch all day.
Maybe people who use 2.7 all day don't see the warts as much as someone who has been rid of them?
Maybe people who use 2.7 all day don't see the warts as much as someone who has been rid of them?
It took me a week struggling with Asyncio to wrap my mind around it. It took me 12 hours to do the same with Golang channels.
I got 20 000 inserts per second from a 360 gig JSON file into Cassandra using 30 goroutines. I was maxing out at 900 in Python 3. Golang was parsing the JSON at a frankly RIDICULOUS 2 million lines per second. Python 3 was around 50 thousand.
Python 3 is adding features that other languages do better. Python smashes other languages in numerics, and approachability. Nobody needs 3 for numerics, and 3 goes backwards in approachability. If you're using 3 for its back-end tools, you're using the wrong tool.
I would never move to golang for numerics because pandas/numpy/scipy/theano stomp all over it, but its only for numerics that I'll stay with Python. Everything else I'm moving away (mostly Golang, looking at Elixir) and talking to the numeric stack over zeroMQ/msgpack.
I got 20 000 inserts per second from a 360 gig JSON file into Cassandra using 30 goroutines. I was maxing out at 900 in Python 3. Golang was parsing the JSON at a frankly RIDICULOUS 2 million lines per second. Python 3 was around 50 thousand.
Python 3 is adding features that other languages do better. Python smashes other languages in numerics, and approachability. Nobody needs 3 for numerics, and 3 goes backwards in approachability. If you're using 3 for its back-end tools, you're using the wrong tool.
I would never move to golang for numerics because pandas/numpy/scipy/theano stomp all over it, but its only for numerics that I'll stay with Python. Everything else I'm moving away (mostly Golang, looking at Elixir) and talking to the numeric stack over zeroMQ/msgpack.
> 20 000 inserts per second from a 360 gig JSON file into Cassandra using 30 goroutines. I was maxing out at 900 in Python 3. Golang was parsing the JSON at a frankly RIDICULOUS 2 million lines per second. Python 3 was around 50 thousand.
Newsflash, Go is fast. I'm sure those numbers will be worse in Py2 than Py3 as well, but aside from that what's your point? Python is slow so... don't switch to Python 3?
> Python 3 is adding features that other languages do better. Python smashes other languages in numerics, and approachability. Nobody needs 3 for numerics, and 3 goes backwards in approachability. If you're using 3 for its back-end tools, you're using the wrong tool.
Sure, people using it for numerical stuff might like Py2, but is that the only use case of Python? Should I not switch because you do numerical stuff? Django is great for backend stuff, as is the whole ecosystem (twisted, flask, DRF, asyncio).
Python 3 is also far more approachable than Python 2. Try explaining to someone why their program exploded after typing an umlaut into it, the magical "from __future__ import division" incantation or even the silly super() syntax.
Newsflash, Go is fast. I'm sure those numbers will be worse in Py2 than Py3 as well, but aside from that what's your point? Python is slow so... don't switch to Python 3?
> Python 3 is adding features that other languages do better. Python smashes other languages in numerics, and approachability. Nobody needs 3 for numerics, and 3 goes backwards in approachability. If you're using 3 for its back-end tools, you're using the wrong tool.
Sure, people using it for numerical stuff might like Py2, but is that the only use case of Python? Should I not switch because you do numerical stuff? Django is great for backend stuff, as is the whole ecosystem (twisted, flask, DRF, asyncio).
Python 3 is also far more approachable than Python 2. Try explaining to someone why their program exploded after typing an umlaut into it, the magical "from __future__ import division" incantation or even the silly super() syntax.
Newsflash. Python 3's main pragmatic selling points are Unicode and Async. Golang is Unicode by default, its concurrency model is much easier to reason about, and it's 20x faster. Why on earth would you use Python 3 (or 2) for your backend? You like buying 20 server boxes when 1 golang box will do? If you think Python will survive as a production language on servers, you're dreaming. It's done, defeated, history on the server outside of legacy code. Django? gimmeabreak.
So what's left that Python does better than any other language? Numerics. That's the only place where you still have a sticky user base that is not likely to dump Python. So don't be so quick do dismiss it. It's the only leg Python is left standing on that other languages cannot easily devour.
Approachability. Not much in it. For every 3/2 = 1 you cite I give you unintuitive lazy ranges, the complete idiocy that 01 != 1, and the unnecessary print functionization which removes one of Python's greatest first-line simplicity selling points. But I will give you that Python 3 remains approachable as a first language. However on that front, inroads every day from JS, where a new coder can get "wow" results in the browser much more quickly than the boring ol' VT100 terminal.
So what's left that Python does better than any other language? Numerics. That's the only place where you still have a sticky user base that is not likely to dump Python. So don't be so quick do dismiss it. It's the only leg Python is left standing on that other languages cannot easily devour.
Approachability. Not much in it. For every 3/2 = 1 you cite I give you unintuitive lazy ranges, the complete idiocy that 01 != 1, and the unnecessary print functionization which removes one of Python's greatest first-line simplicity selling points. But I will give you that Python 3 remains approachable as a first language. However on that front, inroads every day from JS, where a new coder can get "wow" results in the browser much more quickly than the boring ol' VT100 terminal.
Julia has multithreading and channels. Better syntax than go, more like python. Better for numerics...why use anything else?
Not a bad idea. It would certainly elevate the project to unforeseen heights and probably turn them (http://www.baroquesoftware.com/) into very wealthy men while offering a very desired and needed service for many businesses. While harming no one who didn't already harm themselves. More relevant than CPython3 overnight, which is of significant stature.
If prestige and money are desirable things, absolute no brainer if they have a business bone in their bodies. It would also secure the future of the stack for everyone concerned with Python3 threatening to cut off Python2. It could also enable a bigger return on those of us who have invested financially into PyPy development. It would certainly snowball even more financial investment.
It would then be the CPython3 project's move. Abandon and integrate all the non-breaking Python3 features into 2.7? Or just move forward with a significantly weaker and less relevant project, working against their past selves? I don't think most would really care and it's best they just move away with their own project.
Could be on the way judging from the favorable naming convention, PyPy4? Has a nice ring to it.
"the Python2.7 compatible release — PyPy 4.0.0 — (`what's new in PyPy4.0.0?`_)"
"the Python3.2.5 compatible release — PyPy3 2.4.0 — (what's new in PyPy3 2.4.0?)."
The PyPy gods are in an enviable position for sure. If they seize the opportunity it's the chance of a lifetime.
If prestige and money are desirable things, absolute no brainer if they have a business bone in their bodies. It would also secure the future of the stack for everyone concerned with Python3 threatening to cut off Python2. It could also enable a bigger return on those of us who have invested financially into PyPy development. It would certainly snowball even more financial investment.
It would then be the CPython3 project's move. Abandon and integrate all the non-breaking Python3 features into 2.7? Or just move forward with a significantly weaker and less relevant project, working against their past selves? I don't think most would really care and it's best they just move away with their own project.
Could be on the way judging from the favorable naming convention, PyPy4? Has a nice ring to it.
"the Python2.7 compatible release — PyPy 4.0.0 — (`what's new in PyPy4.0.0?`_)"
"the Python3.2.5 compatible release — PyPy3 2.4.0 — (what's new in PyPy3 2.4.0?)."
The PyPy gods are in an enviable position for sure. If they seize the opportunity it's the chance of a lifetime.
Yeah, the best thing they could possibly do right now is to fork Python and make yet another flavour. We already have this 2/3 mess so let's throw another version into the mix just as Py2 seems to be dying.
I don't get this hate for Python 3. Yeah some things changed, yeah having a big codebase in Py2 sucks if you don't have the time/resources to switch, but starting new projects in Py3 by default is awesome. Asyncio and async/await are fantastic and worth the switch alone.
I don't get this hate for Python 3. Yeah some things changed, yeah having a big codebase in Py2 sucks if you don't have the time/resources to switch, but starting new projects in Py3 by default is awesome. Asyncio and async/await are fantastic and worth the switch alone.
Yes, I agree that Python 3 is not that bad to start a new project in. But that's not the problem. The problem is that many of us have a huge investment in Python, and the project is meandering around, borderline-insulting its historic users who still like 2.7, while simultaneously failing to deliver on REAL needs such as performance and GPU support. Instead we get these little tweaks here and there, downright embarrassing PEPs like type annotations, and all this frankly distasteful pressure to move to 3.x using threats of the end-of-the-line for 2....
Basically 3 gives us lots of headaches, very marginal improvements, an ivory-tower sense of "we know best" (message to Guido - no you don't), and a real concern that the powers that be have no clue what's going on at Golang Towers, Fort Javascript, not to mention Clojure, Julia, or even Elixir.
Basically 3 gives us lots of headaches, very marginal improvements, an ivory-tower sense of "we know best" (message to Guido - no you don't), and a real concern that the powers that be have no clue what's going on at Golang Towers, Fort Javascript, not to mention Clojure, Julia, or even Elixir.
They are already willing to take money to work on Python 2.7 stuff, how does refusing to take money for Python 3 stuff lead to them making more money?
It would show leadership. Leadership which is demonstrably lacking in the 3.x line. Thus they could take a (risky but calculated) bet to take over Python completely.
[deleted]
vegabook(1)