[untitled]
1 pointsby xoranth0 comments
float my_func(float lhs, float rhs) {
return 2.0f * lhs - 3.0f * rhs;
}
my_func(float, float):
addss xmm0, xmm0
mulss xmm1, DWORD PTR .LC0[rip]
subss xmm0, xmm1
ret
Can you share what LLMs do you run on such small devices/what user case they address?
(Not a rhetorical question, it's just that I see a lot of work on local inference for edge devices with small models, but I could never get a small model to work for me. So I'm curious about other people's user cases.)