In a weak sense, singularities are common and should be expected every so often. In a mathematical sense, a singularity is where a model 'blows up' and neglected terms become part of the dominant balance.
Industrialization hit a point of diminishing returns, but the industrial revolution was nonetheless a 'singularity,' where life afterwards was qualitatively unpredictable to people who lived before. Likewise, agriculture was such a technological singularity to hunter-gatherer ancestors.
I could even make a decent argument that writing and literacy were such a singularity, making inconceivable social organizations routine.
In that weak sense I expect AI to be a singularity, recursive self improvement or no. Life in 2050 may be completely unpredictable to someone who was taken out of time in the year 2000.
The remaining questions are speed and intensity, and both of these questions are related to RSI. If RSI works, then the 'fast takeoff' visions become more plausible where society transforms over months to a few years – at least locally where the enabling technologies have diffused. If not, it might take a couple of decades.
No, you'd still care. YOLO mode is about instantaneous permissions and access control, inspection of subagent prompts is about retrospective quality control. If the main model is instructing subagents to do a subtly wrong thing, the overall process quality will degrade in ways that might be very hard to detect or fix without deep inspection of the middle stages.
If you're deliberately displaying the image in black and white, the colour pattern is interference that should be suppressed.
However, this practice was not universal, and archivists have now recreated colour copies (https://en.wikipedia.org/wiki/Colour_recovery) of some shows where only black and white recordings survived by reconstructing the colour from these interference signals.
That happened to me once; I was running one of a few free-tier models in a pi-coding-agent session. The bash tool there is stateless and always begins from the launch directory, but the agent assumed state and executed `rm -rf .` intending to remove a build directory. Instead it removed the whole project tree, including session logs and notes.
This was mostly a matter of amusement for me since I was running the agent inside a bubblewrap sandbox for that very reason, and the project itself was not very important.
This blog post is based on a paper (https://arxiv.org/abs/2602.14740). The paper is based on a simulated wargame. The wargame is of the author's own design.
The wargame design does not differentiate between ordinary defeat and mutually assured destruction, so of course a player about to use would 'push the button.' That's also believed to be true in real life.
Results based on simulations can be very informative, but we must always be careful to check how well the simulation framework represents reality.
> fronting the inference layer with a caching prompt classifier to determine which model to use, and automatically select the lowest cost model would probably already save alot of money
Unfortunately, that doesn't work within a single session. The K-V cache of a model is intertwined with the model's configuration. Switching models invalidates the cache, meaning everything up to the point of the switchover is processed like a new, uncached input token.
Per Anthropic's pricing doc, an Opus 4.8 cache hit costs 50¢/MTok, while Haiku costs $1/MTok for uncached input.
Model selection works best if sessions are short and self-contained, particularly if the first few interactions can reliably classify the model need. That probably covers most 'support chatbot' use-cases, but it doesn't describe the kinds of heavy agentic automation that really chews through token budgets.
> Seven tokens long input isn't very realistic, is it?
The test prompt above was "Why is the sky blue?", so there's the seven tokens. I meant to highlight that because I'd expect processing of a thousand-token input to be faster per token than presented.
From the prompt timings above, it seems like 'prompt eval time' is the equivalent to 'processing time for input tokens'.
Hyperscalers can perform this evaluation very quickly because evaluation can be significantly parallelized. The layer `i` output of token `j` only requires access to the layer `i-1` output of all previous tokens, so a parallel frontier develops. Token (0,0) [(token, layer)] is processed first, then tokens (0,1) and (1,0) can be processed in parallel, then (0,2), (1,1), and (2,0), and so on.
The maximum parallel width becomes equal to the number of layers in the model. Gemma 4 26B-A4B model discussed in this article evidently has 30 layers, giving a 30-fold speedup if the system were otherwise unconstrained (all layers can be run in parallel, and one full set of layer outputs is completed in the KV pass for each pass of the parallel sweep).
In the specific output above, however, the input prompt is only seven tokens long so there are probably considerable non-amortized spinup effects at play.
> They are all just variations of "insert a canned prompt", varying only along the dimensions of (a) how and where the prompt is installed and from where it is sourced, and (b) which context or contexts the prompt runs in. There's not much advice here about which option is best, and no clear best practices seem to have emerged yet either. Personally, I find just asking Claude to review the code works well enough.
The subagent approach is structurally different from the others because it runs with clean context. That has three major effects:
1. All other things being equal, it will result in a lower cost-to-solution because of the quadratic cost scaling of an LLM session (input token or cached-input cost being paid with each new round).
2. The review model will not be able to 'cheat' by retaining assumptions from the main session, such as "x must be done like y." For people, this is why having a separate person perform code review (or, if not possible, reviewing code after a mind-clearing break) is handy; the applicability of this analogy to LLMs is vague but reasonable.
3. The main model will only see the results of the review, not the detailed reasoning that leads up to it. On one hand this avoids more context pollution, but on the other hand it might lead to duplicative logic to re-discover the mechanics behind bugs found.
> I checked the session logs to see how often the agents were actually invoking the LSP tools. The answer was they had invoked them literally once the entire time.
I think the intent behind 'install a language server plugin' is that these tools should lint automatically after every edit, without waiting for an explicit call from the LLM.
> You think tax incentives are what makes VC work in California but not other places in the US let alone Canada?
I believe that my comment above was aligned with your premise here. I say that the tax difference is not sufficient by itself and that Canada reportedly has a very non-SV-like venture capital ecosystem.
> I'm sure a lot of Canadian tech workers would repatriate and foreign workers would immigrate to Canada if they could lower taxes across the board and make life easier for tech companies and workers
I'm not sure that Canadian taxes compare that unfavourably to combined California plus federal taxation. A deeper, more structural limitation appears to be the venture capital environment, namely that Canada doesn't have a good one.
Canada's investable capital is dominated by pension funds, insurance companies (i.e. pension funds), and banks (i.e. pensioners). All are risk averse (https://thelogic.co/news/bdc-canadian-venture-capital-report...), which makes it hard for Canadian startups to begin scaling. Without native "unicorns" (https://financialpost.com/technology/why-canada-best-startup...), there's allegedly a failure-to-launch for the entire sector – tech billionaires being some of the most reliable early-stage investors with the greatest risk tolerance.
The porous border works both for and against the sector. On one hand that makes it relatively easy (but not automatic) for a Canadian tech company to enter the US market, but on the other hand it's also relatively easy for Canadian tech workers (founders included) to simply relocate (note the article here). If startups leave for the US's vast fields of venture capital, they're less likely to come back. Note that around the turn of the year Y-Combinator halted investments in Canadian firms (https://www.ycombinator.com/blog/adding-canada-back) because they so frequently relocated to the US.
This venture capital cycle seems to be a deeply-entrenched and very hard problem. If democratically feasible tax incentives could reliably create "the Silicon Valley of X," then we probably would have many more Silicon Valleys both in the US and elsewhere.
> Even if it is common (i don't think this is required any more anyways), just why?
As far as Canadian law goes, there are two factors at play in the parent's events;
* NAFTA work permits are applied for at the border, on entry; they operate differently from the 'normal' work permit streams.
* Permanent residence is conferred at the border, but the application process can happen either inside or outside the country depending on the stream. There are also limited 'inland' options which evidently have expanded (https://www.canada.ca/en/immigration-refugees-citizenship/se...) in recent years.
In neither case does Canada have a blanket rule that an applicant must leave the country during the whole of an extended application process, and even 'abroad' processes can often be carried out while an applicant is living in the country on other status. (It can get awkward if a consular interview is required, though.)
Unlike the US, Canada is generally comfortable with 'dual intent', where intent to apply for permanent residence through legal channels is not disqualifying for other sorts of statuses.
> It’s territorial waters belonging to Iran and Oman.
The trick is that it's still an 'international strait', or a segment of water that forms the only connection between two areas of high seas -- in this case the Persian Gulf and the Gulf of Oman. The principle of freedom of navigation establishes that innocent traffic (civilian traffic, and even warships in peacetime) have a right to use the strait to go from one body of international water to the other.
Iran may claim that it doesn't have to abide by that right, but international law is never self-executing. One question to be resolved by this war is whether Iran will ultimately recognize the right to navigation in any settlement (and then choose to abide by said settlement).
> I don't know enough about the current state of naval warfare but I've assumed this is related to the asymmetry that's emerged around protecting capital warships, especially in the scenario of a very narrow strait and a long enemy-controlled coastline.
It's not the billion-dollar warships that transport oil, it's the much more fragile and unarmed tankers.
Even if the US Navy begins full escort duty, it can't remain on-station forever. What are shippers to do afterwards? One drone strike might cause a tanker to have a very bad day, yet it's extremely difficult to so permanently degrade an entire country that they become incapable of launching sporadic attacks.
Ultimately, the status of the Strait must be settled diplomatically, and the US and Iran are each betting that the other side will blink first.
> If the person using a tool is an attorney, then that communication should be protected whether it's by pen or keyboard.
But the tool is not your attorney, so it can't be the originator of attorney-client privilege. The situation is no different than if you get informal legal advice from a friend: even if that friend is an attorney, the communication is unprivileged unless it's part of a formal representation.
> if your question is "what is the capital of france" the LLM could presumably extract out "paris" from the value vector during attention computation instead of needing the FFN for that.
But how do you get 'Paris' into the value vector in that case? The value vector is just the result of a matrix multiplication, and without a nonlinearity it can't perform a data-dependent transformation. Attention still acts as a nonlinear mixer of previous values, but your new output is still limited to the convex combination of previous values.
> That said, I was sympathetic to the recent bug reports —- to trigger one, you’d need to have a session that waited an hour doing nothing and then very specifically tested for in-context retrieval. I don’t want to run that test, do you want to run that test?
They introduced a feature/optimization that triggered after an hour's idleness, so testing that the session continued properly afterwards seems kind of important. If nothing else, even the working-as-intended feature (context cleanup) could impact model skill in a current or future model version, so it would be well worth measuring any impact as part of the test suite.
> Since the devs on HN (& the whole world) is buying what looks like nonsense to me - what am I missing?
Input tokens are expensive, since the whole model has to be run for each token. They're cheaper than output tokens because the model doesn't need to run the sampler, so some pipeline parallelism is possible, but on the other hand without caching the input token cost would have to be paid anew for each output token.
Prompt caching fixes that O(N^2) cost, but the cache itself is very heavyweight. It needs one entry per input token per model layer, and each entry is an O(1000)-dimensional vector. That carries a huge memory cost (linear in context length), and when cached that means the context's memory space is no longer ephemeral.
That's why a 'cache write' can carry a cost; it is the cost of both processing the input and committing the backing store for the cache duration.
Context length 1e6, vector length 1e3, and 1e2 model layers for 100e9 context size. Costs will go up even more with a richer latent space and more model layers, and the western frontier outfits are reasonably likely to be maximizing both.
> I believe what they're saying is they attempted to fine tune both Qwen and Pythia using Karoline Leavitt's "corpus" (I guess transcripts of press conferences) where she is presumably using the word "deportation" far more than you'd see in a randomly selected document.
Perhaps, but I don't think that Leavitt is habitually using the racial slurs and sexually explicit language that also forms part of their evaluation suite.
In a weak sense, singularities are common and should be expected every so often. In a mathematical sense, a singularity is where a model 'blows up' and neglected terms become part of the dominant balance.
Industrialization hit a point of diminishing returns, but the industrial revolution was nonetheless a 'singularity,' where life afterwards was qualitatively unpredictable to people who lived before. Likewise, agriculture was such a technological singularity to hunter-gatherer ancestors.
I could even make a decent argument that writing and literacy were such a singularity, making inconceivable social organizations routine.
In that weak sense I expect AI to be a singularity, recursive self improvement or no. Life in 2050 may be completely unpredictable to someone who was taken out of time in the year 2000.
The remaining questions are speed and intensity, and both of these questions are related to RSI. If RSI works, then the 'fast takeoff' visions become more plausible where society transforms over months to a few years – at least locally where the enabling technologies have diffused. If not, it might take a couple of decades.