Offset: Goroutines ported to Python(docs.google.com)
docs.google.com
Offset: Goroutines ported to Python
https://docs.google.com/file/d/0B2gNkR7hgkIRVFgwSWR4Y2JaZHc/edit?hl=en&forcehl=1
5 comments
Not really a work-around, even with great native thread support and tooling as you dream of (and don't get me wrong, I am all for it), ultra-light "goroutines" or "erlang processes" fit a different niche, you are never going to spin up 3+ million OS threads, you absolutely can (and do) in the ultra-light process space.
Having 3 million threads doesn't change the usefulness of being able to call into native syscalls and native libraries that use TLS, locks, etc. It does change the usefulness of native debugger support, since existing debuggers would have a rough time with 3 million threads, but it's not like goroutines currently have any solution to that - Go just uses a GDB script to implement goroutine commands similar to the thread ones. So I think it should absolutely be possible to spin up 3 million OS threads. The kernel needs to change to trust memory mapped into userland to keep track of threads rather than tracking all the data structures itself, and the toolchain needs to support split stacks. This would require a lot of change, but the result would be a lot cleaner than having two strongly overlapping concepts of threads.
Stackless Python support Go style concurrency with tasklets( goroutines in go) and channels for scheduling and data passing between them.
Pypy now has stackless support, which makes it the obvious candidate for doing CSP style concurrency with python.
It will not however, scale that concurrency to execute in parallel in multiple cores, since it is still bound by the GIL.
If you want to stay with CPython, you could look at Gevent.
Pypy now has stackless support, which makes it the obvious candidate for doing CSP style concurrency with python.
It will not however, scale that concurrency to execute in parallel in multiple cores, since it is still bound by the GIL.
If you want to stay with CPython, you could look at Gevent.
gevent is based on an eventloop to do the scheduling, it's quite different than the scheduling done in offset and offset will soon support the possibility to launch your goroutin on different OS processes for //. More over gevent monkey-patch the standard library which make sometimes the integration with your code difficult, offset however use its own syscall module.
Also the implementation of channels or stackless in pypy is a little different than the one you can find in offset (or go for that matter). offset channels support buffering, and you can also select them. offset like the implementation of stackless in pypy is using the continulets to do the job when you are using in pypy and python-fibers on cpython which bind the stacklet lib from pypy as well.
Also the implementation of channels or stackless in pypy is a little different than the one you can find in offset (or go for that matter). offset channels support buffering, and you can also select them. offset like the implementation of stackless in pypy is using the continulets to do the job when you are using in pypy and python-fibers on cpython which bind the stacklet lib from pypy as well.
Is this specifically for CPython? I thought Stackless Python already provided something similar barring Go-specific things like channels.
It is also working for Pypy. this is a different impementation from Python Stackless and also give you the possibility to select operations just like in Go.
Anybody have this content in a form that doesn't require a logged in Google account?
it's available on speaker deck too:
https://speakerdeck.com/benoitc/experimentation-in-porting-t...
https://speakerdeck.com/benoitc/experimentation-in-porting-t...
[deleted]
I can dream.