Claude code's system prompt is filled with irrelevant stuff about how CC works, so that the agent can help the user set it up. And there's no way to disable all the extra stuff, AFAIK.
How is this different from the speculative decoding that we had before?
You could pair a big and small model like qwen 32b with qwen 4b and had that same dynamic of the small model generating tokens and the big one "certifiying" them.
The blog says something about re-using the big model's data?
Don't know if this fits, but given your background you might find it interesting. I work at a startup which works with structured video, video which has a certain structure and so allows for reuse of segments. Our challenge is how to show the user this structure, when videos can quickly grow in complexity and end up having multiple thousands of possible variations. Contact me at ismael at lont.ai if you're interested.
In ECS I had to recreate the cluster from scratch because some of the changes I wanted to do, CDK/CF wouldn't do.
My approach on Azure has been to rely as little as possible in their Infra-as-code, and do everything I can to setup the cluster using K8s native stuff. So, add-ons, RBAC, metrics, all I'd try to handle with Helm. That way if I ever need to change K8s provider, it "should" be easy.
I started with ECS (because I wanted to avoid the complexity of K8s) and regret it. I feel I wasted a lot of time there.
In ECS, service updates would take 15 min or more (vs basically instant in K8s).
ECS has weird limits on how many containers you can run on one instance [0]. And in the network mode where you can run more containers on a host, then the DNS is a mess (you need to lookup SRV records to find out the port).
Using ECS with CDK/Cloudformation is very painful. They don't support everything (specially regarding Blue/Green deployments), and sometimes they can't apply changes you do to a service. When initially setting up everything, I had to recreate the whole cluster from scratch several times. You can argue that's because I didn't know enough, but if that ever happened to me on prod I'd be screwed.
I haven't used EKS (I switched to Azure), so maybe EKS has their own complex painful points. I'm trying to keep my K8s as vanilla as possible to avoid the cloud lock-in.
I've found their system tries to guess the language of the text, but there are many languages that use the same words, and so it will speak in the wrong language (or accent, even). I hope they make a solution for this, otherwise I don't think it will be suitable for production use cases.
But among colleages we use certain jargon which varies by industry and probably by country. Could LLMs have their own preffered jargon?
I usually write pseudocode when I'm thinking about a problem to solve, so in a way I'm "thinking with pseudocode" instead of plain language. Pseudocode is probably more accurate than plain language, and it's something I'd use when explaining to other humans what I want them to code (along with diagrams, which seems ChatGPT would understand now). So, to me, speccing this pseudocode to something the LLMs find easier to understand sounds reasonable. It's like understanding how a fellow programmer prefers to get his requirements.
I find it intriguing. It makes sense that this new kind of "thing" (LLMs) could be "programmed", and that you could craft a language specifically for it's abilities.
I've read the tutorials but I still find it hard to wrap my head around it.
Have you heard of any other language like this? Or had success using SudoLang?
You could generate titles from the subtitles! Even auto-generated subtitles are pretty good nowadays, feed it to AI and get it to summarize them in one phrase?
There's https://github.com/skrabe/lobotomized-claude-code , which strips many of those, but I'm not sure if it is "legal" to use.