AI Aulis

AI NEWS IN ENGLISH · 21 November 2026

The most important AI news every morning. AI writes and checks the stories against the original sources – check important facts in the sources.

← Issue 21 November 2026Paper

AI agents’ time horizon on METR tasks doubles every 3.6 months

AI-generated contentWritten by AI

metr.org, technologyreview.com, tokenpost.com, arXiv · 4 sources · Sources published: 30 July–20 November 2026

METR’s new measurement also covers tasks that require judgement. On these tasks, models appear to lag behind human experts: none of the evaluation tasks that models built was good enough for METR to use.

METR, a non-profit that evaluates AI models, published a report on 20 November in which it tested models up to Opus 5.5 and GPT-6 Astra. Its main metric is the time horizon: the length of task a model can complete in half of its attempts. A task’s length is the time a skilled human would need for it.

On the old and new task suites combined, the time horizon doubles roughly every 3.6 months (95 per cent confidence interval 2.1–4.9 months), the report says. METR said the pace is similar to the 105 days measured earlier on the old task suite. On the new tasks that can be scored automatically, frontier models reach a time horizon of up to 27 hours, and it doubles every 1.65 months.

The faster pace does not prove that progress is accelerating. METR considers it possible that models quickly cope with even longer versions of similar tasks. It sees no clear acceleration in an index that aggregates model capabilities. The TokenPost website wrote that the growth rate had doubled and named the use of AI in AI development as a possible cause. METR leaves the cause open. Only 9 per cent of the report’s time estimates are based on measured human working time.

METR suspects that cheap, automatically scored tests resemble the environments in which models are trained, so they may make models look more capable than they are. METR measured this with what it called a very rough metric: what it would cost to build a task as an environment and to run and score it 100 times. With task length held constant, the time horizon on tasks that were on average 2.9 times more expensive was about a third of that on cheaper tasks. The rate of progress was similar for both. Opus 5.5 estimated the costs.

On tasks that require judgement, models appear to lag behind human experts. Models built new evaluation tasks for METR, but none was useful enough to include in the report. MIT Technology Review reported that the latest models, such as Opus 5.5, reach the threshold level, whereas a year ago all of them fell clearly below it.

As a judge, Opus 5.5 scored GPT-6 Astra’s tasks on average about 0.2 points (on a scale of 0 to 1) higher than METR’s human scorer. METR said the simplest explanation is that the models have weak judgement. It nevertheless considers it somewhat likely that the models’ full capabilities have not been elicited on such tasks.

When scoring critiques of philosophical arguments, many models’ agreement with expert ratings exceeds the estimated level of experts who spent five minutes. All public models, however, remain below the level of experts who spent 15 minutes. The task is based on a preprint published on arXiv on 29 July 2026. Of its 951 critiques, 946 were rated by a single researcher, Emery Cooper.

Report an error in this story