That is a good point and I don't mind getting a bit philosophical when I point out that experience is distinct from rationalism. Underneath this there is an argument about empiricism vs. rationalism (or realism vs. idealism).
The hand-wringing on the 50-50 cases is almost always theoretical, in the sense that I am trying to reason instead of rely on memory/experience. But that is a double edge sword because sometimes memory/experience are closer to trauma and can lead one away from a solution that is correct now in some new environment.
It also reminds me of the phrase "use in anger". You don't really know about an approach/architecture until you've had to deal with something in an urgent or high-stakes moment (your 2am debugging). But to get that experience, you have to have shipped the thing first. It is a chicken-and-egg problem, you can't debug something at 2am unless it is live, and you can't know if it will cause you a problem until you've dealt with it.
AI (LLMs, agents, etc) is this giant alteration to the environment that shakes up everything. In some sense, I feel I have to throw out my taste and "use in anger" all over again. It is painful but may be the only way. And I have to accept the risk that going slowly (like this post suggests) may be the wrong way, and I may watch the young untraumatized new-comers blast by me riding their 100 agent orchestrations to massive success. What I mean here is, in the final scenario from my original comment (7 top guys vs 100 decent guys) - I know the result from experience. But am I just traumatized and it will be different with agents?
The hand-wringing on the 50-50 cases is almost always theoretical, in the sense that I am trying to reason instead of rely on memory/experience. But that is a double edge sword because sometimes memory/experience are closer to trauma and can lead one away from a solution that is correct now in some new environment.
It also reminds me of the phrase "use in anger". You don't really know about an approach/architecture until you've had to deal with something in an urgent or high-stakes moment (your 2am debugging). But to get that experience, you have to have shipped the thing first. It is a chicken-and-egg problem, you can't debug something at 2am unless it is live, and you can't know if it will cause you a problem until you've dealt with it.
AI (LLMs, agents, etc) is this giant alteration to the environment that shakes up everything. In some sense, I feel I have to throw out my taste and "use in anger" all over again. It is painful but may be the only way. And I have to accept the risk that going slowly (like this post suggests) may be the wrong way, and I may watch the young untraumatized new-comers blast by me riding their 100 agent orchestrations to massive success. What I mean here is, in the final scenario from my original comment (7 top guys vs 100 decent guys) - I know the result from experience. But am I just traumatized and it will be different with agents?