> I’d say that, in all the ways that matter, I understand the code. Sure, if I had to hand-edit it now, there’d be a serious learning curve. But I won't have to.
Famous last words, but point is taken.
The non deterministic nature of an llm breaks the metaphor that they are like a compiler.
However, it’s not foreign to compilers to receive feedback from the running program (PGOs), so there are still parallels to the feedback we provide LLMs that guide their “optimization”.
I think even calling LLMs a non-deterministic compiler isn’t accurate either so ultimately I agree the metaphor doesn’t quite work.
I do think LLMs are like compilers in terms of how they changed how we build programs from a historical context. But that’s about it.
There’s not much of a difference because open models are distilling from frontier models. If frontier models cease to exist then many of these open models will as well
This post was thought provoking, I wonder, is the hidden argument here that the posix spec for a shell is not well specified if there is so much variance between the implementations?
Or is the fundamental issue simply a matter of history? Both?
> Bindings are not IPC. The Deno runtime and the rendering backend run as threads / processes inside the same address space (CEF) or coordinated process group (WebView). Calls go through in-process channels, and the backend dispatches them from its run loop. -- https://docs.deno.com/runtime/desktop/bindings/
I don't understand how the coordinated process group works. Doesn't that mean in this multi-process mode it must be IPC? Maybe the claim "shared memory space" is more an architectural description than an OS-level claim?
I feel like the claims come from wildly different personas and use cases. A 24gb vram, 5 year old titan run 27b at 30t/s and the results are good. I use sonnet and opus at my day job and they are more capable but I can still get the same out of qwen, I just need to be mindful of ctx
Preload is the answer to speed. Basically download the clients db on init and then have cache invalidation strategies. I built starfx to basically perform the data sync aspect of this paradigm https://starfx.bower.sh/learn#data-loading-strategy-stale-wh...
I really like this idea and have been experimenting with it over a week or so.
I think there’s an opportunity to use an AST diff system for code forges where you don’t present the user with line diffs in the UI — or at least not as the first diff the user sees.
I firmly believe code review should happen in your editor.
I'm still thinking this through but I was arguing this position to colleagues to some shock: LLM's are a race-to-the-bottom and frontier models will not be able to afford to work on coding specific models (or coding features at all) in the very near future.
27B is already really good at coding-specific tasks. Fundamentally, there is little innovation on the core architecture: LLMs are all designed essentially the same, with minor differences in how they are trained. They are all feed-forward multi-headed attention models; it doesn't matter if it's a 4B model or a 1T model, that's just scale.
Further, the frontier models cannot afford to innovate: they have to scale as quickly as possible to "beat out" their competition. The frontier models fundamentally will not create the next "attention is all you need" monumental jump in AI.
Frontier companies are stuck on scale with zero capacity to innovate. You cannot point capitalism at "basic science research" and expect any ROI. This is a known reality. Innovation is much more indirect and a "random walk" style of knowledge acquisition.
Finally, these LLMs are quite literally designed with a human-in-the-loop, and we do not give ourselves enough credit for how well we ourselves tool-call. We are doing a lot of heavy lifting to make these models useful and you cannot simply remove us from the equation without also removing ourselves from the training pipieline.