The switch by Artificial Analysis from per-token-cost to per-benchmark-cost shows some effect!
Its nice that labs are now trying to optimize what I actually have to pay to get an answer - It always annoys me to have to pay for all the senseless rambling of the less-capable reasoning models.
Also, for an auto complete I think a small llm trained from scratch should already work well. Have you tried on if the tinystories(also only 3gb..)/nanogpt speed runs without any fancy loss terms etc as a baseline?
How foreign is the language - was it likely included in pre training to some degree? Does it use grammar, syllables, and logic similiar to one of the large languages?
Your approach assumes there is an easy to learn mapping between context in your target language and concepts in a prettained llm.
Can you get more text written in the low resources language?
I thought the same, until I noticed a really annoying WSL2 bug: On two machines I own, waking up from hibernate or standby causes a wsl related process (vmmem) to consume 100% CPU, with wsl becoming completely unresponsive (including wsl terminate etc).
You have to kill all wsl processes, which requires admin rights. So without elevated rights, Ubuntu on windows is not usable on these laptops.
There is a typo in the in app purchase page: "unlimmited"
I clicked on one story, did not like it, so i canceled it. I wanted to give the app a second chance. But alas, only one story is allowed per day.. Maybe increase this at least for the first day or do not count unfinished stories....
So the code looks (apart from pointers) similiar to numba which feels much closer to numpy/pytorch high level code. Are there huge advantages in the triton model compared to numba that I don't see? Or is there a big performance gap?
For me numba was always the easiest way to get some new idea running on cuda, and most of the time it was fast enough..
Did anybody find performance comparison between numba and triton?