Host to device bandwidth (ram to vram) is 128Gb/s for PCIe Gen 6. VRAM to GPU bandwidth is 1.8Tb/s for GDDR 7 (5090), and 8Tb/s for HBM3e (B200). So it can be faster to recompute than offload kv cache.
The Little Learner is a great book. I tried rewriting all the code in Python/JAX while following the Scheme code style as closely as possible, and it worked out great.
The appendix on autodiff is a bit rushed, in my opinion. But in all fairness, the number of pages would probably need to be doubled to give a proper exposition of autodiff.
I liked Multipass until I ran into storage corruption issues like many others [1][2] and broke all my VMs. On MacOS, Multipass uses qemu in which the apple-hvf support isn't getting much updates after 2021 [3].