Exactly! "It just works" used to be true for macos, and linux was for tinkering. Now, ubuntu "just works" (and fast!).
It's not just that macos is slow and window management is strange. Last time I had a macbook (M2 pro) I needed 3rd party software just to use a 4k external monitor without blurring.
I like that you go beyond just prompt engineering and "LLM as a judge" and use finetuned (?) ModernBert and Llama models.
In your previous post you mentioned that you "score 20+ dimensions". Are these generic dimensions for all use cases / users, or do you finetune individually for each user?
This is much needed on mac os. But why? Back in the day, mac os was supposed to "just work", and linux was for weirdos who had too much time and customized everything. But now, it's the opposite. Vanilla flavor ubuntu just works out of the box. And mac os needs tons of third party customization. Same with external monitors, on mac one needs 3rd party software for properly scaling the UI on a 4k monitor. That's core OS functionality, not something one should need an extension for.
> Emoji suggestion: Slack might suggest emoji reactions to messages using the content and sentiment of the message, the historic usage of the emoji and the frequency of use of the emoji in the team in various contexts. For instance, if [PARTY EMOJI] is a common reaction to celebratory messages in a particular channel, we will suggest that users react to new, similarly positive messages with [PARTY EMOJI].
Finally someone has figured out a sensible application for "AI". This is the future. Soon "AI" will have a similar connotation as "NFT".
I agree that sampling only valid tokens is a very promising approach.
I experimented a bit with finetuning open source LLMs for JSON parsing (without guided token sampling). Depending on one's use case, 70B parameters might be an overkill. I've seen promising results with much much smaller models. Finetuning a small model combined with guided token sampling would be interesting.
Then again, finetuning is perhaps not perfect for very general applications. When you get input that you didn't anticipate in your training dataset, you're in trouble.
I wish it was easy & reliable to run ubuntu on an M2 macbook. That would be perfect - ubuntu is a much more capable & convenient OS at this point, but nothing comes close to the macbook in terms of hardware.
So sad that we need 3rd party apps for even the most basic functionality in mac os. It used to be that linux was a hassle to set up, and mac os worked out of the box. Now it's quite the opposite.
Similar thing with using an external 4k monitor. By default it's blurry (looks like 720p, awful), and one needs a 3rd party app (BetterDisplay) just to get a decent image. I had this issue with several macs and displays and cables. Terrible user experience. Obviously works out of the box with ubuntu, debian, and windows.
I want a system that works out of the box with sensible defaults. What a strange turn of events that this is a linux system now.