Isn't Apple about to license some variation of this from google for on-device AI? Maybe it’s their sales pitch to Apple and then they will lock it down.
Maybe it's my experience, but TPMs were often responsible for coordinating large org-wide or cross-org initiatives. It's prohibitively expensive to have TPMs on anything smaller.
Since engineering with AI is still very technical, I would wager that software engineers would stretch into less technical areas of software development rather than TPMs stretching into technical areas. I only say this as someone with experience with AI and I see how easy it is to write bad code with AI if you're not aware of what it's doing.
Yes, size and performance are not only problems for local LLMs, they are problems for frontier LLM companies like OpenAI and Anthropic. The latter still lose a ton of money on inference and advances in efficient, performant models helps their bottom line.
They are addressed but the core of the thesis is still wrong:
> This is the core problem: our entire evaluation infrastructure is structurally reactive. We measure the system after it has changed. We never predict the change.
The argument in the article is backwards. Evals test the stability and boundaries of a concept. They are not created before the concept has been prototyped (which the author acknowledges).
An eval is not somehow breaking silently due to some new capabilities in an LLM. It wouldn't be a good eval if it did. What it does is steer the LLM towards specific goals. If anything, an argument can be made that they restrict creativity and experimentation by narrowing goals.
If the argument is that evals need to written before some new behavior can be devised, that's incorrect. There are an infinite number of evals that test for things which cannot be done. Only when something has been demonstrated to work in a specific context, can an eval be written.
I applaud the move. It's also a little disingenuous to talk about moral standings when the third opening sentence is "The math hasn’t worked out for a while now." If the numbers were working out, would they continue to turn a blind eye on the privacy tracking?
Too much of code is data transformation. input -> sanitation -> db -> consumer -> api -> client. Business logic defines the shape of that data and some service-level rules but the majority is just shoveling data.
Those are raw numbers. I would look instead at the job changes over total employment numbers. I don't have the numbers but I would wager we have many more people working in tech today (overall) than we did in 2008.
Also, that spike in 21/22 really did a number on people's expectations. The one constant in this industry is its cyclical nature.
Publishing is more than just authoring. You have research, drafts, edits, source verification, voice, formatting, multiple edits for different platforms and mediums. Each one of those steps could be done by AI. It's not a single-shot process.
Not quite systems programming but this might give you some insight. Swift is memory efficient, and runs stable backend services. I've seen benchmarks showing that it's slightly more performant than typescript but twice as memory efficient (but not as efficient when it comes to memory management compared to Rust, C, and C++).
The other point I've seen is that its string library is slow and very accurate.
Besides that, the C-interop means you have quite a bit of flexibility in leveraging existing libraries.
I guess the cache would only be invalid if the day changed or the root directory, which would technically happen infrequently enough.