I've worked a little bit with distributed Elixir using `Horde.DynamicSupervisor` on Kubernetes. Apparently there's other options like 'swarm' and DynamicSupervisor [1]. It'd be great for clear analysis of the benefits these kinds of abstractions bring vs non-BEAM approaches.
Some people say we're near the end of pre-training scaling, and RLHF etc is going to be more important in the future. I'm interested in trying out systems like https://github.com/OpenPipe/ART to be able to train agents to work on a particular codebase and learn from my development logs and previous interactions with agents.
[1] https://www.youtube.com/watch?v=nZmDEUeHeVI