Reviewing AI Code Is Not a Viable Argument(softwaremaxims.com)
softwaremaxims.com
Reviewing AI Code Is Not a Viable Argument
https://www.softwaremaxims.com/blog/reviewing-ai-code
4 comments
I agree, he should've said "working code"
Definitely not true. Used intentionally, LLMs are just a tool that can be used to improve the human review. That's what Saga is - a tool to make code reviews better, with a sprinkle of AI https://github.com/JakeBeresford/saga
interesting read, I think a valid point for accelerating code review of LLMs is funnily enough to make sure you have multiple LLMs reviewing each other in a sort of pipeline. Some research already shows this increasing accuracy of AI generated code, maybe could evolve into a real review process that works?
honestly the review data here matches my experience, review is the constraint, not writing the code. that's the whole reason i spend my time on it.
where i'd push on the conclusion is the 400 lines/hour ceiling, which assumes you're reading an undifferentiated diff top to bottom. most AI PRs aren't uniform, a big chunk is mechanical (renames, boilerplate, repeated patterns) and a small chunk is the actual logic that needs a careful human look. We see more of these PRs now as companies use AI to clean thier tech debt if you can separate those, you're not reviewing 5000 lines at 400/hr, you're carefully reviewing the 300 that matter and skimming the rest. the throughput math changes a lot, and we can actually save time reviewing PRs this way.
the lower-defect-detection-with-higher-confidence point is real though. skimming has to mean skimming known-safe noise, not the risky parts, but that's where AI can help. that's the bet i'm making with what i work on (pyor.review). Using AI to reduce what you review instead of adding to it, not to hand you another summary to read, which I think is the correct approach for using AI for a code review.
where i'd push on the conclusion is the 400 lines/hour ceiling, which assumes you're reading an undifferentiated diff top to bottom. most AI PRs aren't uniform, a big chunk is mechanical (renames, boilerplate, repeated patterns) and a small chunk is the actual logic that needs a careful human look. We see more of these PRs now as companies use AI to clean thier tech debt if you can separate those, you're not reviewing 5000 lines at 400/hr, you're carefully reviewing the 300 that matter and skimming the rest. the throughput math changes a lot, and we can actually save time reviewing PRs this way.
the lower-defect-detection-with-higher-confidence point is real though. skimming has to mean skimming known-safe noise, not the risky parts, but that's where AI can help. that's the bet i'm making with what i work on (pyor.review). Using AI to reduce what you review instead of adding to it, not to hand you another summary to read, which I think is the correct approach for using AI for a code review.
I think https://github.com/baokhang83/fluencyloop can help ingesting the code through the Mermaid architecture diagrams the skill produces (I posted yesterday on HN about it)
This is not true.