I have a very good mental model in my head of each and every codebase I have ever worked on. Nothing to do with my PRs or even my team’s direct responsibilities. I could always direct a thought provoking idea to challenge the existing status quo and get a reasonable nuanced answer from code owners on slack or during a water-cooler talk. I build that mental model also by reading the code. This is my job, so I know when to tear down what or how to respond to a random product idea immediately on a meeting to asses complexity.
I’ve stopped using llms to generate architecture, which i design and write myself and let the machine pattern match the gaps. I also use it to review issues which I lot of the times push back against.
I’m working on a stateful application sitting on top of a data warehouse and have to implement a stream of messy half defined feature requests and navigate on top of an ever changing infrastructure layer. LLMs rarely get the infra layer even if it is written as code and have hard time grasping how to deal with tech debt, when and how to re-architecture parts of the stack or even implement stuff based on a detailed openspec design.
Depends on the goal. If staying at a certain number is the goal yes, climbing two weeks back up will possibly take more and more time. If staying energetic and healthy is the goal the number becomes only one of the moving benchmarks.
With that said I think doing some kind of workout even on vacation is important.
I think tantrum comes when they are tired / disconnected from adult monologue. I have almost zero issues when I talk about interesting stuff instead of engaging in debate:
we will pick up that book about the monster, it is really scary (slippers on already) and we will sit on the sofa (already carrying the child). Are you cold? Let’s find that pink sweater…
So you think that having your words rewritten by an llm is somehow a more powerful, faster, safer way to write down your ideas? What do you want to say with this exactly? How you separate the “what” to write from “how” to write it?
Why I engage with this comment? I truly believe that trusting your own knowledge and skills is the way forward. If you don’t have the skill? Build it. Don’t outsource thinking and learning.
I think it is fine to create the scripts with the cloud based llm but it is definitely not a fable / opus level thing, and running the bisect loop itself has nothing to do with an agent, it is a simple shell script.
> I was trying to find the root cause of a crash in a Python module which left no errors in the log or console. Fable wrote a test harness that simulated clicks in the UI, then bisected my code until it found the point where it started crashing
Does this need an agent though is my question? Maybe generating a test case and a loop doing git bisect but why on earth would we want to run it through the internet and gpus and whatnot when it can be run on a single core celeron.
So you believe that your work will be done by AI and you will enjoy life more? This is not a loaded question, just trying to understand what your future ideal day / week would look like as an "ai optimist".
I also notice these things. Otoh i spend definitely less than 50% of my time typing in code so it is impossible that it gives more than 2x speedup. And sometimes i lose time babysitting and rewriting stuff so all in all it is kinda no productivity gain.
> If it was just programming being automated, then whatever.
There is nothing on horizon which automates a programmer’s work. Typing in code is faster now, and some things “only need pointing out” like an existence of a “bug” which an llm + harness might be able to mitigate. Automated tests might capture regressions and possibly written by llm + harness.
If you replicate this in other professions what will you get?
What do you base this on? For me it is almost impossible to guess what fits into the context of an llm. Sometimes trivial tasks fail, sometimes quite complex things get one shotted.
The issue is that validation needs presence and it is the limiting factor - common knowledge, but is part of the “physics”. Also maintenance gets really tricky if the codebase has warts in it - which it will have. I get much more easy to understand architecture out of an LLM driven code generation process if I follow it and course correct / update the spec process based on learnings.
Example: yesterday I’ve introduced a batch job and realized during the implementation phase that some refactoring is needed so the error boundary can be reused in the batch application from the main backend. This was unplanned and definitely not a functional requirement - could be documented as non-functional. There was a gap between the agent’s knowledge and mine even though the error handling pattern is well documented in the repository. Of course this can be documented better next time if we update the process of openspec writing but having these gaps is inevitable unless formal and half-formal definitions are introduced - but still there needs to be someone with “fresh eyes” in the loop.
I understand that you are serious. I am also serious here.
Have you built anything purely with LLM which is novel and is used by people who expect that their data is managed securely, and the application is well maintained so they can trust it?
I have been writing specifications, rfcs, adrs, conducting architecture reviews, code reviews and what not for quite a bit of time now. Also I’ve driven cross organisational product initiatives etc. I’m experimenting with openspec with my team now on a brownfield project and have some good results.
Having said all that I seriously doubt that if you treat the english language spec and your pm oversight as the sole QA pillars of a stochastic model transformer you are making a mistake.