Yes, picchio monitors a running llama-server on a timer and flags when the prefill/decode ratio goes CPU-shaped. I hot-swapped one mid-run. probe 4 caught ENGAGED -> NOT ENGAGED.
tbh, backups matter. but nobody would accept Word deleting your files when you cancel Office. somewhere along the way we stopped distinguishing backup from custody.
i think this is mixing two separate ideas.
MTP is the training-side piece. speculative decoding is the inference trick. DeepSeek V3 used MTP as an auxiliary loss. the 2022 Google paper is speculative decoding. now Google is combining them.
https://arxiv.org/abs/2404.19737
fwiw, understanding was never really the goal. calling it "debt" assumes all of it needs to be repaid, but most of the time this just feels like brain OOM. imo the skill is knowing what actually needs to stay in your head.
IMO this probably isn't just about latency. keeping people in voice gives them training data text never will. is that why they were fine going transceiver over sfu and mostly ignoring multi-party?