Code review was never about the bugs
If an AI reviewer can catch the bugs, what is human code review still for?
Contents6 sections
Ask engineers why they review code, and most say it’s to catch bugs. Yet when researchers counted the changes reviews actually lead to, most weren’t about bugs. They made the code easier to read or change.
I think review’s bigger job is one those counts can’t see. A second person comes away understanding the code, and that’s the part I don’t want to hand to an AI reviewer.
Before reading on, guess two numbers. Out of every 100 changes that human review leads to, how many alter what the code does? And out of every 100 comments from an AI reviewer?
Drag the slider to guess, then check your guess.
Functional changes: the research’s closest thing to bugs.
25 and 75.9, the same split flipped.
Nothing checked yet. Your first guess is 50.
The AI reviewer is CodeRabbit, from a 2026 study. About 3 in 4 of its comments are about what the code does. So the AI reviewer does what engineers say review is for, and human reviewers mostly do something else.
What the studies counted
In a 2013 Microsoft survey, 44% of developers ranked finding defects as their first reason to review. Only 14% of real review comments were about defects.
| Study | About bugs |
|---|---|
| Bacchelli and Bird, Microsoft, 2013 | 14% of 570 review comments |
| Czerwonka, Greiler and Tilford, Microsoft, 2015 | about 15% of review comments |
| Beller et al., open source, 2014 | about 25 in 100 changes made during review |
| Sadowski et al., Google, 2018 | 2 of 44 people surveyed said comments had found a bug |
A functional change alters what the code does, so it is the closest the research comes to a bug. Most of the other changes improve maintainability, which means making the code easier to read or change without changing its behaviour.
The understanding lives in people
In 1985 Peter Naur argued that the core of a program is a “theory” held by the people who built it. By theory he means knowing why the code is built the way it is and how it can change safely. His example is a compiler built by group A and extended by group B. Group B had the full program text and good documentation, yet its proposed extensions broke the design. Group A saw the problems at once.
Naur called rebuilding the theory from documentation alone “strictly impossible”. Review is one of the few routine places where it passes from one person to another.
Review puts a second person on each file
Most teams review code as a pull request (PR), a proposed change that others read before it is merged. Peter Rigby’s group measures this with files at risk. These are files that at most 1 active developer has modified or reviewed. If that person leaves, nobody holds the file’s theory.
Files at risk: code files that at most one active person has modified or reviewed.
Press Count reviewers too to see what review adds.
Counting authors only, about 4 in 5 files have one person who knows them.
These counts treat a quick approval like a careful read, so they make review look better than it is. My own record shows the gap. From 2022 to 2026 the code host recorded 1,640 review actions by me on other people’s PRs. Only 101 of them carry 300 or more characters of my own writing.
Still, the figure shows review giving many files a second person who has read them. When only an AI approves a change, no second person has read it.
Why the AI misses some bugs
Catching bugs takes theory too. In Czerwonka’s data, about a third of comments from reviewers new to a file were useful. By their third review of it, about two-thirds were.
An AI reviewer is always new. Developers rejected 56.3% of CodeRabbit’s comments, and 35.8% of its defect comments were judged invalid. The authors trace the invalid ones to limited understanding of the architecture and the wider repository.
The strongest case against this
The best counter-argument is that a model can now explain any part of the code in a minute, so nobody needs to hold its theory.
The Claude Code team described this approach in October 2026. Low-risk changes merge without a human approval once automated review and security checks pass. Their merged PRs more than doubled over a summer, and the share later reverted fell by more than 40%.
I still think this is wrong for most teams, for 3 reasons. First, code doesn’t record the rejected alternatives, and an explainer only answers the question you knew to ask. Second, Anthropic ran a randomized trial (people assigned to groups by chance) with 52 mostly junior engineers. The group using AI help averaged 50% on a comprehension quiz, against 67% for the group without it. Third, the team’s own policy still requires 2 people on the riskiest code. So we agree that someone must understand risky code, and disagree only about how much code counts as risky. For well-tested code that is cheap to rewrite, their approach makes sense.
What I do in review
I try to write the reason, not only the request. In 2024 I reviewed code for a vendor pod, a small team of contractors from an IT services company building a site for one country. On the pod’s first PR for serving the site in 2 languages, I left 16 comments on specific lines. They asked for translation files instead of language checks scattered through the code, for our shared design-system components, and for tests that check a result.
Later I asked for the same rules across the pod’s PRs, and authors adopted several of them. I have no measure of whether later PRs kept to them.
Since 2025 I’ve led the team that runs the customer sites for North America and Oceania (the company calls it a squad). On one of its PRs, which added fallback product images, I asked for changes and explained how the original logic worked. Then I proposed one fallback chain with a default per country.
In 2026 I was the main author of a shared repository of team docs. Most review there came from an automated reviewer, and human review stayed light. The changes got checked, but I can’t say anyone else learned why they were made.
So let the AI reviewer hunt for bugs on every PR. Spend human review where few people understand the code, often on the person who will own it next. Before approving, that reviewer should be able to say why the change is built this way and what breaks if it’s wrong.