Article is seriously wrong, because it makes a huge mistake in the last part. You can't simply look at the produced tokens and that is your cost. In agentic coding there are lots of turns meaning you not only pay for the output tokens you also pay for all the input tokens sent each time (even if a lot cheaper, like 10x when cached). So this calculation does not accurately represent the api cost at all.
Second thing is you can starkly upgrade the token generation locally if you use agent teams. Single conversations are memory bandwidth bound and don't fully make use of your compute. If you can batch tokens from multiple agents you can easily 5x token generation.
The K/V Cache is just an optimization. But yeah you would expect the attention for the model producing "Ok im doing X" and you asking "Why did you do X?" be similar. So i don't see a reason why introspection would be impossible. In fact trying to adapt a test skill where the agent would write a new test instead of adapting a new one i asked it why and it gave the reasoning it used. We then adapted the skill to specifically reject that reasoning and then it worked and the agent adapted the existing test instead.
In the example given in the article i think the correct behavior would have been to infer the type backwards from the return type of the function. Is that not why mypy actually errors here?
Yeah Containers like Option<T> and Result<T> not having a proper subtyping relation to T is major flaw. I mean you are basically giving a stronger guarantee if you are returning T instead of T or an Error, yet you break every callsite.
I have been thinking about this in terms of a data oriented programming language like clojure, but having types/schemas. If you use Union Types for something like getting a value out of a map for a specific key, then if you knew the schema of a specific map you could reduce the return type from {T, Error} to just the type of the value T that you know is there.
Basically a sufficiently smart compiler with the necessary information could make you not have to deal with errors at all in certain cases.
With Result/Option/Maybe this would not be possible. It would always infect the entire callstack and you would always have to deal with it.
Concurrency is not "single core parallelism". Concurrency describes tasks/threads of execution making independent progress of each other. Parallelism describes tasks/threads actually running at the same time.
They are not missing the point. The previous post was simply wrong. The halting problem is semi-decidable. So you can potentially find out wether a program halts.
That just gave me the idea that there could be a main instance that keeps a summary of the problem (List of paragraphs for each subproblem) and other instances that hold the larger text that these summaries are based on. Then on demand the main instance could get needed extra information for a subpoint back.
Kinda like having forgein keys in a databas with which you can join in the relevant row!
How do you to zero-copy message passing if you have one heap per actor? Or am i misunderstanding and a couple of actors share a heap, but there is no giant heap?
Felt the same way after first going from eclipse to intellij. This fear, this lack off. But after a while i didn't notice anymore and i have never run into a situation where something important was lost.
In the opposite. Saving and making copies and even duplicating code in a file like an older version of an sql statement is some kind of fear/hoarding impulse, that doesn't seem to pay off at all.
Second thing is you can starkly upgrade the token generation locally if you use agent teams. Single conversations are memory bandwidth bound and don't fully make use of your compute. If you can batch tokens from multiple agents you can easily 5x token generation.