I think we need more neurons in the human brain than parameters in this model for speech. I wonder what it says about the human brain vs LLM efficiency.
- litellm listens on 127.0.0.1:4242 with some pre-shared key, give the credentials to the harness.
- Tools can exfiltrate any data accessible to them given network access, so all tools run inside a VM or a container targeted to the specific tasks. For example, I've made podman spawner that runs a 'tool server' inside a container, and all of my custom harness' tool requests go to that container (which can be barred from the network). The workspace for that LLM is solely inside the ephemeral container, but it can get bind mounts from outside if needed.
On one hand, I sense that the same people who were in the shorts crowd of the Tesla 'boom or bust' Model 3 era ($TSLQQ) are now piling up against SPCX (e.g. Ed Zitron). Betting again Elon is very risky. And on the other, boy that huge valuation is scary.
The situation could be that there are two intertwined bubbles: an AI-tech-bubble and AI-financial-bubble, both at the same time and only one of them is going to really burst and affect valuations. If that happens, we can only guess the period of time it takes for the S&P 500 to recover.
And on the other hand, there's a whole movement (in niche terms, though) of people going "just add to an S&P 500 ETF for 50 years and forget about everything else".
That could happen, but should also consider that it's often the case with lockups release that they do not inflict much change on the stock price, but only if there's already enough trading volume on the stock due to other market conditions.
Understand that social media endorphin-inducing algorithm optimizers, have made these sites optimized for _all_ ages, so it should not be a surprise. Very few of us are immune.
Noticed that most of the comments here are despairing.
I think that perhaps there's a bit of hope, that by the forces of the market, the value of human distinctiveness will rise in comparison to whatever is the generated mean. This is what I am looking into.
I've read elsewhere that it's 10x less electricity for inference workloads compared to standard GPUs. It is not clear to me, are the model weights built into the silicon (e.g. per model tapeout), or is this a new kind of chip architecture that still has weights in DRAM/SRAM?
I'm glad to see this. Almost 18 years ago I implemented a similar kexec device+memory preservation for a storage vendor. It was done on a Linux kernel of that day, and it had had a memory reservation and handoff protocol between the two kernels to keep some specific PCI device alive, allowing for state restoration at the application side. I'm proud of the fact that the kernel replacement was just under 1 second in execution (after init process optimization) and the whole kernel+app was less than 10 seconds.
http://github.com/da-x
Email: alonid [at] gmail.com