Numba: NumPy-aware optimizing compiler for Python(github.com)
github.com
Numba: NumPy-aware optimizing compiler for Python
https://github.com/numba/numba
23 comments
You mean sort of like how PyPy is a JIT for only a restricted subset of Python?
No, PyPy JITs all the Python by design. It also runs all the Python and JITs all the constructs. It does not make all the possible python magically faster however, but that does not change my point. A JIT for only a restricted subset of Python would mean that it segfaults/crashes/gives a wrong answer if you don't provide a correct subset. This is not true.
So basically you're saying that PyPy knows what code it can optimize and Numba requires the programmer to specify it. That probably has more to do with Numba's being version 0.2 than anything else. And it's hardly reason to moan that it's not a "real" compiler.
I did not call it not a "real" compiler (with or without quotes), I did call it not a python compiler. It has nothing to do with 0.2 - supporting all of Python is probably not even within goals of the numba project. I'm not talking about "all the bytecodes", but making sure all the corner cases work perfectly well etc. Also supporting all cases would mean that you get rid of a lot of speed benefits that you see right now, precisely because you don't care about all the corner cases. Predicting the future is obviously hard and I can't speak on behalf of numba authors, but I don't think there are plans to support all of Python.
Python code goes in, LLVM IR comes out. Sounds like a compiler to me.
As you know, the basic idea of the project is to make writing fast, vectorized code for numerical and scientific computations as easy as writing native Python. And to do it in a way that maintains compatibility with the extensive CPython universe.
I think the criticism you're fishing for is that it is not an implementation of Python. No, it's not that. But, yes, it is a compiler.
As you know, the basic idea of the project is to make writing fast, vectorized code for numerical and scientific computations as easy as writing native Python. And to do it in a way that maintains compatibility with the extensive CPython universe.
I think the criticism you're fishing for is that it is not an implementation of Python. No, it's not that. But, yes, it is a compiler.
No, it's not. There is one more requirement - the LLVM IR has to work exactly the same as intended for python (in this case as cpython does). This is not the case - it's not full and what's implemented does not support all the corner cases of the language, starting with overflow-to-long.
Just asking... How it would compare to PyPy's RPython?
Different purpose, different restrictions. Numba is written to support fast numpy ufuncs (without resorting to C) and the like. RPython is a proper subset of Python (so all corner cases either work or are declared "not RPython", you can't get a different answer) which supports more of a language but with less of integration with the host interpreter.
That restricted subset, RPython, is what PyPy itself is implemented in; but PyPy can run arbitrary Python code.
Here is the post at Travis Oliphant's blog about numba
http://technicaldiscovery.blogspot.com.es/2012/08/numba-and-...
As it is said there, it's still early software and its "road-map is being defined right now by the people involved in the project". Sure the subset is not well defined now and there are no docs, but hey - let's give the thing a few months.
Haven't tested thoroughly yet, but I think no NumPy calls can be made from inside a numba-compiled function - so this might be the case for other Python modules as well.
http://technicaldiscovery.blogspot.com.es/2012/08/numba-and-...
As it is said there, it's still early software and its "road-map is being defined right now by the people involved in the project". Sure the subset is not well defined now and there are no docs, but hey - let's give the thing a few months.
Haven't tested thoroughly yet, but I think no NumPy calls can be made from inside a numba-compiled function - so this might be the case for other Python modules as well.
Actually some NumPy calls can be made inside the numba-compiled function as there is nascent support for the NumPy C-API --- it translates the call to a C-call.
In the future, we want to support all Python, but not necessarily optimized --- just using the Python Object C-API.
In the future, we want to support all Python, but not necessarily optimized --- just using the Python Object C-API.
here is a talk form SciPy 2012 about Numba http://www.youtube.com/watch?v=WYi1cymszqY
Here is a mailing list thread with the results of a simple benchmark: https://groups.google.com/a/continuum.io/forum/m/#!msg/numba...
Looks promising!
Looks promising!
How good does this work with other python modules?
Can i use this to compile _every_ python program to native executables?
No.
In any case, doing that wouldn't be difficult, you just wrap a Python interpreter and the Python code in a single executable and you're done. What would make it worthwhile is when it would be an optmizing compiler, which is of course difficult for such a dynamic language.
In any case, doing that wouldn't be difficult, you just wrap a Python interpreter and the Python code in a single executable and you're done. What would make it worthwhile is when it would be an optmizing compiler, which is of course difficult for such a dynamic language.
The github README (https://github.com/numba/numba) explains how to install Numba, but not how to use it.
Or is it used automatically for all Python byte code? If so, can it be disabled?
Or is it used automatically for all Python byte code? If so, can it be disabled?
The main way to use it right now is by decorating the functions you want it to compile. See the examples directory [0] in the Github repository.
0. https://github.com/numba/numba/tree/master/examples
0. https://github.com/numba/numba/tree/master/examples
the tests suggest that it is only used where you tell it to be used: https://github.com/numba/numba/tree/master/tests
The whole approach might be viable and it definitely has use cases, however the sensational headline makes it so bad :/ Hackernews - you let me down.