User @thin_signal developed a tool for mixed precision quantization on MLX. They perfoemd a sensitivity analysis across the model layers and applied less radical quantization on the layers more sensitive and more quantization tomlayers that are more robust.
The tool, which is documented here (https://mlx-optiq.pages.dev/) also implements the recently aanounced TurboQuant KV-Cache optimization, so in total this should greatly improve the quality of locally run LLMs.
Looking forward to an OptiQ release of the Gemma 4 family.
Along the same line is that you can watch any hour long video without interruptions unless it is music where you will get interrupted every couple of minutes with "are you there?" dialogues.
Wondering if it were feasible to vibe-code a Bot that would engage and invest the little sums that yield those bait gains, and harvest those.
Eject at the right time.
That is part of the ‚first mover advantage‘. Sometimes operating and experimenting in grey zones before they become regulated.
The European and especially German approach of regulating pre-emptive might be more fair, but apparently it also stifles innovation, as we can observe. Almost no significant players from Europe and Germany.
Have been using Spacemacs and Doom Emacs some time ago, but turned off evil mode, bc I was not used to vim style editing. Lately I understood that vim key bindings is actually an editing language and finally found joy in vim [0]. Surely going to return to Doom-Emacs with evil mode on. Maybe switching to evil god mode later.
I have been using a similar tool, Clipea, on a regular basis and I find it really helpful. I am CLI proficient enough to know whats possible on the shell, but I use it too infrequently to remember all the ins and outs and flags. At this point these tools really improve productivity.
Have not used shllm, but from the docs it looks good. Clipea and others have not been updated for quite some time, except llm from Simon Willison, the datasette developer. His llm implementation has a llm-cmd plugin which provides what shllm does. Works like a charm.