Python performance is a fool's errand(whoosh.ca)
whoosh.ca
Python performance is a fool's errand
http://whoosh.ca/blog/fools_errand?repost
6 comments
Performance is not always the the primary issue in programming
Except when it is, python rapidly becomes depressing. Try writing a game in python/pygame, for instance, and watch how quickly things get out of hand.
Except when it is, python rapidly becomes depressing. Try writing a game in python/pygame, for instance, and watch how quickly things get out of hand.
Well, I code python for ease of use and readability. Every language has its pros and cons. To be frank speed was never a forte of python but it was never meant to be as such. If you need speed there are options available, use them. However C have a faster run-time and slow development time. There are issues in all languages. However there is always a favourite language. Mine is python :)
True, some times performance is the primary issue, but not always.
When it is, Python is probably simply not the right langauge. Unladen swallow and future developments may change this, but for now if performance is a major issue than Python is not for you. C++ is generally a much better option for that, and there are a few that are even faster if you are willing and knowledgeable enough to do the optimizations.
When it is, Python is probably simply not the right langauge. Unladen swallow and future developments may change this, but for now if performance is a major issue than Python is not for you. C++ is generally a much better option for that, and there are a few that are even faster if you are willing and knowledgeable enough to do the optimizations.
I wrote a 2D real-time spaceship shooter in Python with PyGame (called Ganymede, maybe a link from my site) and found the performance excellent. Most of the hard bits are done by native code or hardware anyway. Plus your application architecture has a big impact on performance. And number of polys or entities, etc. Perhaps trying to write a state-of-the-art 3D FPS would yield poor performance, is that more what you were talking about?
What's changed since you submitted this a few weeks ago?
http://news.ycombinator.com/item?id=684017
http://news.ycombinator.com/item?id=684017
Python is designed to be easily extended with C. If you need performance then write those performance-critical parts in C and write lightweight wrapper to python.
Or write speed-critical Python iterator directly in C.
Python source is the best reference: http://svn.python.org/view/python/trunk/Objects/dictobject.c... (look at the end).
Python/C API rules.
Python/C API rules.
Optimizing like this in python is clearly the wrong approach. Python will never be fast in the traditional sense.
The only optimization you should be worrying about is avoiding quadratic behaviour and such, which usually accounts for 99% of all inefficiencies.
If you really need speed, move the code to C and call it from python.
The only optimization you should be worrying about is avoiding quadratic behaviour and such, which usually accounts for 99% of all inefficiencies.
If you really need speed, move the code to C and call it from python.
I haven't kept up with Python developments, but the fact that almost everything is a hash table (even stack frames for interpreted function calls, I believe) would seem to place a relatively low upper-bound on performance. And this is in addition to the usual overhead of a straightforward interpreter.
I'm not sure if that's a low upper-bound. He's complaining about an attribute access speed - but he's not trying to fix it... missed a great opportunity for __dict__ usage.
From a quick py2.5 benchmark:
So... 1s (without the constructor and loop time) for 3,000,000 attribute reads and 3,000,000 attribute writes. Slow? Is he sure that it's the attribute read and write that's causing the problems? I'm not sure if I believe that without some more details - especially if he works with some documents that look like they're on some external storage (disk / db / whatever).
From a quick py2.5 benchmark:
noop 0.3s
class_create 0.76s
with_dict 1.68s
without_dict 2.06s
So... 1s (without the constructor and loop time) for 3,000,000 attribute reads and 3,000,000 attribute writes. Slow? Is he sure that it's the attribute read and write that's causing the problems? I'm not sure if I believe that without some more details - especially if he works with some documents that look like they're on some external storage (disk / db / whatever).
Stack frames in modern JIT VMs are just normal machine stack frames. When running in debugger mode, VisualWorks reifies the Object stack frames and runs the interpreter for that process as a program in the image. This way, you get speed for normal runtime, and access to everything as an object. This might seem like a bit of a stunt, but remember, the basic assumption for a JIT is that the JIT is a perfect simulation of bytecode interpretation. So if you have a valid JIT, you have a valid Debugger. This also means that anyone can extend the debugger, write tools based on it, or even write their own, as it's mostly an ordinary Smalltalk program.
Then again, there's a few bad side effects from this. You have to copy the Debugger and its infrastructure to debug anything used by the Debugger. So, for example, you can't debug ListViews in the debugger, because the debugger application uses them! (But copying that stuff, and renaming is not a big deal in Smalltalk. Plus, it makes you feel like an uber-hacker.)
Then again, there's a few bad side effects from this. You have to copy the Debugger and its infrastructure to debug anything used by the Debugger. So, for example, you can't debug ListViews in the debugger, because the debugger application uses them! (But copying that stuff, and renaming is not a big deal in Smalltalk. Plus, it makes you feel like an uber-hacker.)
First, I second the notion that switching to another tool is a good idea when performance becomes important. I was doing this by diving down into C years ago, and I have to imagine the toolset has only gotten better since then.
Second, you can make your generator method a coroutine and send it data as it iterates.
Second, you can make your generator method a coroutine and send it data as it iterates.
Performance is not always the the primary issue in programming. Using python I am much more worried about code readability, maintainability, and programmer speed than I am code speed. And Python certainly is designed in a way that makes true high performance somewhat challenge. If I were ever truly concerned about maximizing performance, I would use C++ or similar.
On the other hand, performance is always a consideration. It should never be totally forgotten about. For instance, if you will be using a single regex pattern multiple times in a program (or putting it in a loop) the performance increase from compiling it once is noticeable and does not subtract from any of the other attributes. Similarly, in many cases a list comprehension will be faster than an equivalent for loop.
Python performance is not a fool's errand, but I do think that people programming in python should realize they are making something of a trade off (unladen swallow and future research may partially change this in the future.)
(Edited to fix spelling typo.)