honestly the review data here matches my experience, review is the constraint, not writing the code. that's the whole reason i spend my time on it.
where i'd push on the conclusion is the 400 lines/hour ceiling, which assumes you're reading an undifferentiated diff top to bottom. most AI PRs aren't uniform, a big chunk is mechanical (renames, boilerplate, repeated patterns) and a small chunk is the actual logic that needs a careful human look. We see more of these PRs now as companies use AI to clean thier tech debt if you can separate those, you're not reviewing 5000 lines at 400/hr, you're carefully reviewing the 300 that matter and skimming the rest. the throughput math changes a lot, and we can actually save time reviewing PRs this way.
the lower-defect-detection-with-higher-confidence point is real though. skimming has to mean skimming known-safe noise, not the risky parts, but that's where AI can help. that's the bet i'm making with what i work on (pyor.review). Using AI to reduce what you review instead of adding to it, not to hand you another summary to read, which I think is the correct approach for using AI for a code review.
I review a lot of PRs in my team, the human comprehension half is the part I care about, so limited attention is the problem.
my worry with most AI tools is that they add text and summaries and walls of AI generated descriptions, verbose test steps, and layers or summaries, that makes me read more text instead of less, which takes more time than a normal PR review.
I still have to read the code anyways, so I would prefer something the makes this job easier for me, I would rather use something like graphite or pyor.review instead.
Oh man I was looking for something like this for a while, reviewing code in VSCode is starting to get annoying and I began to think that VSCode is no longer suitable for development in the AI era, but I already built my own solution that addresses the two problems I had with reviewing code:
- A better alternative to reviewng a PR on github.
- A way for me to do a local review for my code is if was a PR, before actually filing that PR.
I also built a Claude skill around the local review so it's more seamless to integrate with Claude, I also get more powerful features with the help of AI like grouping files in folders and sprinkling hints as squiggly lines that help me with the review.
Oh man I was looking for something like this for a while, reviewing code in VSCode is starting to get annoying and I began to think that VSCode is no longer suitable for development in the AI era, but I already built my own solution that addresses the two problems I had with reviewing code:
- A better alternative to reviewng a PR on github.
- A way for me to do a local review for my code is if was a PR, before actually filing that PR.
I also built a Claude skill around the local review so it's more seamless to integrate with Claude, I also get more powerful features with the help of AI like grouping files in folders and sprinkling hints as squiggly lines that help me with the review.
>I have always had a love and hate relationship with code review tools
Same here, we're not happy with todays code review tools, and github itself already sucks, we already looked at tools like graphite and stage but they did not serve our needs much.
I eventually built my own review tool and I'm pretty happy with it.
Yeah, I thought of this while writing my comment.
I hate this actually, in all devices where you want to cut on the number of button there are, like a headphones for examples, having one button that does stop/play/next/prev/reset/whatever... is really annyoing, one button having too many functions is really bad UX IMO.
I'd rather have another button to press once rather than holding the first one for 15 seconds to do some other thing
Thanks for the review, that's really good feedback.
as we slowly read the code less and less, it only makes sense for AI to help us skim it more, until it becomes good enough for us to not read the code at all.
still, on the other hand, do you think AI can tell you what a 5 word long function does in 5 words? probably not, so reading the code itself will stay faster than reading the summary, AI can only help alerting you on what it thinks is a potential bug, or same code moving around.
I still don't like the fact that AI is adding more stuff for us to read, it's accelerating the code production but slowing down the code review.
I built my own code reviewer as well (https://pyor.review/), surfacing the important stuff first is the right track, but adding more stuff to read is daunting, but asking AI to just point you to what you need to focus on and skim the noise is what I'm leaning more towards.
2. Grill my requirements (I use Matt Pocock's skills).
3. Write an ADR, then either start implementing or separate into pieces.
4. Review the code on pyor.review, compared to Github, Pyor allows me to categorize the files and changes then review the important stuff and skim the noise it identifies.
5. Since I can do local reviews with Pyor, I can do that with Claude and feed back my comments to be addressed without it going to Github first.
Yeah, I use VSCode a lot less now, even for code reviews, I built my own code reviewer that's better than Github IMO.
I no longer use VSCode or Github, I use Claude to produce the code and use https://pyor.review to review the code, same thing when I review my teammates' PRs.
One job doesn't really fit the button thing because a button has to do many things, only one of which is being clickable.
Having feedback when clicked, feedback when hovered. A loading state, a disabled state, a mix of everything. That's also something I found very frustrating. Like if you take the example of a button that is tied to a service on the back end, clicking this button without any visual feedback doesn't tell you if that button was clicked or not. And if it doesn't have a loading state, you wouldn't know that. something is working behind the scene when you click this button so you would click at multiple times not knowing that it's still working behind the scenes.
1. Start an AI review on the side while I do my own review.
2. Test the app myself, note any bugs or visual issues.
3. Review the code on www.pyor.review, compared to Github, Pyor allows me to categorize the files and changes then review the important stuff and skim the noise Pyor identifies.
4. leave comments on the on the code, and go back and forth with the author until everything is addressed.
5. approve.
When it comes to my own PRs, I usually assign specific team members to specific PRs depending on who is best to review the part of code I'm editing, which moves the review faster