Llama.cpp now supports tool calling (OpenAI-compatible)
github.com3 pointsby ochafik1 comments
brew install llama.cpp
llama-server --jinja -fa -hf bartowski/Qwen2.5-7B-Instruct-GGUF:Q4_K_M
I've indeed done all that on my spare time (still under Google copyright), very happy to see this used and appreciated :-)
About to start a new job / unsure if I'll be able to contribute more, but it's been a lovely ride! (largely thanks to the other contributors and ggerganov@ himself!)