[untitled]1 points·by Ambix·hace 3 años·0 comments1 commentsPost comment[–]Ambix·hace 3 añosreplyHappy to announce new version of LLaMA.go - open-source CPU inference framework for GPT models.That's a big milestone, we've embedded scalable server which allowing access to GPT model with simple REST API.Now anyone is able to integrate local GPT into micro-service mesh or build fancy ML startup :)Pre-compiled binary builds for all major platforms released too.Free access to already converted LLaMA 7B and 13B models as well.All model computations take place on regular CPU and RAM, so no GPU access needed at all.Model quantisation is not supported yet, so you'll need at least 32 Gb RAM to grok with 7B model.I'm ready to help with any questions :)
That's a big milestone, we've embedded scalable server which allowing access to GPT model with simple REST API.
Now anyone is able to integrate local GPT into micro-service mesh or build fancy ML startup :)
Pre-compiled binary builds for all major platforms released too.
Free access to already converted LLaMA 7B and 13B models as well.
All model computations take place on regular CPU and RAM, so no GPU access needed at all.
Model quantisation is not supported yet, so you'll need at least 32 Gb RAM to grok with 7B model.
I'm ready to help with any questions :)