Cognition announced Devin Review 2.0 on its blog on 5 November. The company wants reviews to be cheap enough to run on every pull request, like CI tests.

Devin Review is a service of Devin, Cognition’s coding agent. It groups the changes in a pull request, explains them alongside the code and looks for bugs and security vulnerabilities.

The company’s own figures

Cognition said reviewing a pull request now costs 82 per cent less than in August. The company credits the savings to a better mix of models and to harness optimisations. A harness is the program that steers an agent: it gives the model its tools and runs its steps.

Quality did not suffer, the company said. After the cost cuts, Devin Review found an average of 1.32 bugs per pull request, up from 1.22. Median latency fell from 139 seconds to 94 seconds. Across customers, the average is about $2.25 per fixed bug.

AICoder, a site that summarised the announcement, pointed out that no independent replication of the measurements has been published. The bug count reflects how many findings Devin made. The blog does not say how Cognition assessed whether the findings were genuine.

Re-reviews cover the changes

When the same pull request is reviewed again, Devin Review by default checks only the parts that have changed since the previous review. About 80 per cent of pull requests cost less than a dollar to review, including later updates, Cognition said. The documentation says Devin may still review the whole pull request if new commits substantially change its behaviour or the change is too large.

Rules can limit automatic reviews, for example by path, branch or label, and drafts can be included too. An administrator can also cap the cost of automatic reviews for a single pull request. Once the cap is reached, automatic reviews stop, but manually started reviews still run. The English-language documentation says an empty cap field does not remove the cap; the Cognition-managed default applies instead.

When Devin Review checks a pull request opened by an active Devin session, the findings go to the session first, the documentation says. Devin fixes them only if an administrator has turned on the Auto-fix review findings setting under Settings > Review. Otherwise Devin only reports the results. The feature is being rolled out gradually.

The new Ultra reasoning effort gives a more thorough and more expensive review, for example for billing or security code. Teams can run checks based on their own instructions with Custom Reviewers, which are in beta.

Availability

Devin Review uses compute units (ACUs) from the same pool as Devin sessions. It works on GitHub, GitLab, Azure DevOps and Bitbucket. On GitHub, commenting and other write actions require a GitHub App connection. On GitHub and GitLab, users can start a review with the comment /devin review; on GitHub, the commenter needs write access to the repository and an account linked to Devin. Reviews can also be started from the Devin Review page in Devin Cloud and Devin Desktop, and automatic rules are set under Settings > Review.