We could implement ownership transfer in CPython in the future, but it's a bit trickier. In Rust, "move" to transfer ownership is part of the language, but there isn't an equivalent in C or Python, so it's difficult to determine when to transfer ownership and which thread should be the new owner. We could use heuristics: we might give up or transfer ownership when putting an object in a queue.SimpleQueue, but even there it's hard to know ahead of time which thread will "get" the enqueued object.
I think the performance benefit would also be small. Many objects are only accessed by a single thread, some objects are accessed by many threads, but few objects are exclusively accessed by one thread and then exclusively accessed by a different thread.
You will always get either [1,2,3,4,5,6] or [4,5,6,1,2,3] in the upcoming `--disable-gil` builds of CPython 3.13 and the nogil forks. Most operations on mutable collections hold a per-object lock.
Part of the integration work will be to better document the thread-safety guarantees, but there is still a lot of work to do before we get there.
Memory models don't usually explicitly guarantee that writes "eventually become visible". They're usually written as ordering guarantees for when a write becomes visible, such as happens-before relationships. Obviously, for multithreaded programs to be useful, the writes have to eventually become visible to other threads/groroutines just like you want all sorts of other operations to happen in finite time that are not explicitly guaranteed by standards (like whether a thread/goroutine eventually starts.)
This misses the fact that signed integer promotions are generally not free because they require sign extension. For example, the cast in b() emits a `movsx` (move with sign-extension). `movsx` is not free in the ways that `mov` can be.
`mov` can be "free" in two ways:
- the compiler can frequently avoid emitting the `mov` when part of a larger operation
- modern CPUs handle `mov` instructions earlier in the pipeline
The PyTorch tensor library was originally basically the Python version of Torch 7. It's now moving closer towards NumPy's API (and farther from Torch 7).
Th autograd library was inspired by Chainer's design and took a lot of concepts (but not code) directly from Chainer. The neural network API is a bit of a hybrid. It's built on top of the autograd library but the layer names, implementations, and some conventions were inherited from Torch 7's NN and cuNN libraries.
(EDIT: and the name "autograd" originates from HIPS autograd library, which I think predates Chainer)
Your criteria 2, 3, and 4 doesn't make much sense to me. We often have workloads that require multiple boxes, but we still want to make effective use of each box. Common server hardware has dozens of cores, which requires a lot of parallelism to fully utilize. The GIL hinders that, even when most of the work doesn't hold the GIL (see Amdahl's law)
Python multiprocessing doesn't work well with a lot of external libraries. For example, CUDA doesn't work across forks and many system resources can be shared across threads but not processes. Python objects must be pickled to be sent to another process, but not all objects can be pickled (including some built-in objects like tracebacks).
A lot of different parallel programming models can be built on top of threads (shared memory, fork-join, message passing), and to a certain extent they can be mixed. That's not true of Python multiprocessing, which only allows a narrow form of message passing. (It's also buggy, has internal race conditions, and easily leaks resources.)
The problem for CPython is that it may not be possible to remove the GIL without breaking the C API, and a lot of the benefit of Python is the huge number of high-quality packages, many of which use the C API.
uv: https://docs.astral.sh/uv/getting-started/installation/
py-free-threading: https://py-free-threading.github.io/installing-cpython/
docker: docker run -it quay.io/pypa/manylinux_2_28_x86_64 python3.13t