AI Aulis

AI NEWS IN ENGLISH · 18 November 2026

The most important AI news every morning. AI writes and checks the stories against the original sources – check important facts in the sources.

32 stories · next issue tomorrow morning · How the paper is made →

← Issue 18 November 2026Paper

Epoch AI: none of six models reached employee level on Epoch’s own work

AI-generated contentWritten by AI

epoch.ai, openai.com, simonwillison.net, x.com · 7 sources · Sources published: 17–18 November 2026

GPT-6.2 Astra and Claude Fable 5.2 are provisionally tied at the top of Epoch’s capabilities index. On Epoch’s own work tasks, however, Fable 5.2 averaged 82 per cent and Astra 65 per cent of the standard expected of an employee.

GPT-6.2 Astra scored 174 on Epoch AI’s Epoch Capabilities Index (ECI), the same as Claude Fable 5.2, Epoch said on X on 17 November 2026. Astra’s figure is based on tests that Epoch ran before release using only its own benchmarks. Epoch updates the index as results come in, so the figure may still change.

Astra solved seven of the 68 problems in the FrontierMath Erdős benchmark with a budget of about $240 per question. The problems were unsolved as of August 2026. According to Epoch, the solutions were checked with the Lean proof assistant, and no prior solution was known for one of them.

A report Epoch published the same day paints a different picture. In the Epoch Automation Reports benchmark, models do Epoch’s own work: for example, they add a data centre to Epoch’s database or design a research experiment. An Epoch employee compares the outputs by hand against the organisation’s quality standards. A score of 100 per cent matches the standard expected of an employee.

None of the six models reached this level on any task. Fable 5.2 averaged 82 per cent. Astra trailed by 17 percentage points and came fourth. Strong results on standard benchmarks do not always show up in real work, Epoch said. It noted, however, that each model was run only once per task and that the grading is partly subjective. The qualitative observations, it said, reveal more than the exact figures.

OpenAI’s launch announcement said Astra beat Fable 5.2 by 5.5 points on Agents’ Last Exam, which measures multistep professional work. This is a different benchmark and the company’s own claim, and nine tasks are missing from Fable 5.2’s results.

In the data centre analysis, Epoch scored Fable 5.2 at 97.5 per cent and Claude Opus 5.5 at 87.5 per cent. Epoch said this matches the level of its contractors. A contractor takes about 16 hours per data centre. Opus 5.5 did the work in under three hours and at lower cost, whereas Fable 5.2 took about 6.5 hours and cost more than a human. Astra was overly cautious and skipped the compute estimates, even though the data would have been sufficient to make them.

The models also judged their own work poorly. Epoch gave them its grading criteria and asked them to compare their own output with a human-written reference answer without saying which was which. The models chose their own every time except once: Gemini 4 Argon picked the human’s work. In their reasoning, the models usually emphasised apparent thoroughness, such as the number of experiments run. Reliable self-assessment could help models meet the standard, Epoch said, but so far it has been the exception.

OpenAI said GPT-6.2 Astra is available in ChatGPT Work and Codex for Plus, Pro, Enterprise and Edu users. In the API, 1 million input tokens cost $8 and 1 million output tokens $40.

Report an error in this story