Ask HN: How to Fine Tune LLMs?
I wanted to play around with fine tuning LLMs on my Mac. Any recommendations on how (or) where to start? Is there a blog you'd suggest?
2 comments
These instructions got me started
https://huggingface.co/docs/transformers/training
with the caveat that the default training rate is too high and you get better results with 2e-5
https://huggingface.co/docs/transformers/training
with the caveat that the default training rate is too high and you get better results with 2e-5
Great! Thanks...
There are a number of ways, with increasing methods and capabilities. It's going to depend on what you are trying to do and the constraints you have
Google put out a paper last week that talks about some of this and proposed a new method that outperforms previous: https://news.ycombinator.com/item?id=35810663
Google put out a paper last week that talks about some of this and proposed a new method that outperforms previous: https://news.ycombinator.com/item?id=35810663