p ~= 12 * n_layers * d^2_model + n_vocab * d_model
providers:
- # llama-server
id: openai:chat:qwen
config:
apiBaseUrl: http://localhost:7876
apiKey: "..."
passthrough:
chat_template_kwargs:
enable_thinking: false
It can also be used by the LLMs to provide the input and output token counts on the different APIs, though I'm not sure if this is how llama.cpp or other OpenAI-like APIs calculate the input/output tokens of a request.