METR: Anthropic’s models are improving about twice as fast on well-specified tasks
METR estimates that Anthropic is improving its models on well-specified tasks about twice as fast as in 2024–2025, mainly with the help of AI. Anthropic removed most of the key evidence from the public version of the report.
On 5 November METR, a US non-profit that researches AI risks, published an assessment of how much AI is speeding up Anthropic’s own AI development. METR staff reviewed the company’s internal data from August to the end of October.
METR said Anthropic is now improving its models’ capabilities on well-specified tasks at around twice the rate it did between March 2024 and October 2025. In other words, the company now makes as much progress in one year as it previously made in two. The estimate is highly uncertain: METR would not be surprised if the multiplier were closer to 1.3 or 3. A Reuters report published by ARY News quoted METR as giving figures of 1.5 and 2.5.
Benchmark scores are rising faster
The key metric is AECI, Anthropic’s version of the Epoch Capabilities Index, which combines a model’s benchmark scores into a single capability score. Since October 2025, AECI has grown 1.9 times faster than before (95 per cent confidence interval 1.5–2.8). The calculation also includes unreleased internal models.
Under different assumptions about the difficulty of the benchmark tasks, the multiplier ranged from 1.1 to 2.9. When other companies’ models were included, it was 1.4. METR said the acceleration may partly result from models being tuned to the benchmarks. The latest internal model already scores above 50 per cent on every benchmark it has been given, which makes measurement harder.
METR is certain that the acceleration is not due to faster growth in training compute.
AI assists researchers but does not yet work autonomously
By METR’s highly uncertain estimate, about 80 per cent of the current acceleration is due to AI. In late 2025, other factors likely also contributed. AI speeds up work mainly by assisting researchers. It does not yet do significant autonomous work in the most important stages of model training, although the researchers said the first signs of this have appeared.
In Slack surveys, Anthropic employees estimated that October’s most capable internal model speeds up their work about 12-fold on average. METR considers this very weak evidence. Moreover, an individual employee’s speed-up does not equal the acceleration of development as a whole.
METR found no significant bottlenecks for the next 12 months, so development could accelerate further. However, METR’s view of the bottlenecks changed several times during the assessment.
Anthropic removed most of the evidence
Anthropic had the right to remove all unpublished information from the report. METR said the company removed most of the evidence that METR considered most important to its conclusions, so the public has to rely on METR’s judgement. Anthropic justified the redactions as protecting confidential information that could speed up other AI developers’ work.
In a separate review, METR considers it possible that the early-warning threshold in Anthropic’s Responsible Scaling Policy had already been crossed by 15 July. The threshold is crossed if AI doubles the pace of development. Even so, most METR and Anthropic researchers who examined the question lean towards the view that it was not crossed. METR said the company’s public risk report played down the acceleration. The report put the acceleration below twofold, but from the figures in the internal version METR calculated a multiplier of 1.97.
Like Anthropic, however, METR considers the direct catastrophic risk from the July models to be low. METR said the risk could rise quickly if development accelerates markedly further over the next six months.