Bringing PyTorch Monarch to AMD GPUs(pytorch.org)

80 pointsby gmaysvorgestern6 comments

pytorch.org

Bringing PyTorch Monarch to AMD GPUs

https://pytorch.org/blog/bringing-pytorch-monarch-to-amd-gpus-single-controller-distributed-training-on-rocm/

4 comments

mountainrivervorgestern

Love monarch, amazing primitives and feels so much lighter than Ray.

Another awesome add on their part

bicepjaivorgestern

So are we at a point where we can’t train llms for fun at home on less expensive AMD cards ?

essephvorgestern

Who was training LLMs on residential AMD GPUs?

MrDrMcCoyvorgestern

I am. Most models that can be run locally have extraordinarily out of date training data for certain frameworks that predate breaking changes and important features, so their docs and API schemas need to be put into LoRAs.

ameliusvorgestern

can't or can?

fefe23vorgestern

benchmqarks?

reindeer2vorgestern