AI Aulis

AI NEWS IN ENGLISH · 18 November 2026

The most important AI news every morning. AI writes and checks the stories against the original sources – check important facts in the sources.

32 stories · next issue tomorrow morning · How the paper is made →

← Issue 18 November 2026Paper

Astra for Law leads LAB-AA legal benchmark with full credit on 20.8 per cent of tasks

AI-generated contentWritten by AI

openai.com, simonwillison.net, github.blog, x.com, vals.ai · 7 sources · Sources published: 17–18 November 2026 · Some source publication dates are unknown

In Artificial Analysis’s test, Astra for Law, OpenAI’s configuration adapted for legal work, met every requirement without a material hallucination on 20.8 per cent of tasks. In Vals’s non-comparable version, GPT-6.2 Astra met 94.6 per cent of criteria but fully solved 16.25 per cent of tasks.

On 17 November OpenAI released an updated Astra for Law configuration. It combines the company’s new frontier model GPT-6.2 Astra with a legal search index and instructions that tell the model how to research a question, weigh legal sources and write its answer.

The result comes from Harvey’s Legal Agent Benchmark, which Artificial Analysis runs as its own LAB-AA version. The benchmark has 120 tasks across 24 areas of law, such as reviewing a commercial lease. A task earns credit only if the answer meets every requirement and contains no material hallucination, meaning a claim the source documents do not support.

Artificial Analysis said Astra for Law earned full credit on 20.8 per cent of tasks and GPT-6.2 Astra on its own on 15.0 per cent. Its published chart places Muse Wonder Full third at 13.1 per cent, with Claude Opus 5.5 at 4.2 per cent. Even the leader therefore fell short on almost four tasks in five.

Vals AI runs the same Harvey benchmark with its own implementation and says its results are not comparable with LAB-AA. Two language models grade the answers. GPT-6.2 Astra met 94.6 per cent of individual criteria but fully solved 16.25 per cent of tasks, ranking 11th. A model can meet most criteria and still miss the credit for a task, Vals said. Gemini 4 Argon leads at 27.08 per cent. Astra for Law is not included in this Vals comparison.

OpenAI also says Artificial Analysis calculated that Astra for Law averaged 0.03 material hallucinations per task, against 0.59 for Claude Opus 5.5. Opus 5.5’s figure was the lowest among non-OpenAI models, and Astra for Law shared the lowest figure with GPT-6. The company said Astra for Law produced about 64,000 tokens per task, compared with 139,000 for Muse Wonder Full. On Vals’s Legal Research Bench, Astra for Law earned full credit on 112 of 208 questions (53.8 per cent), up from about 46 per cent in September, OpenAI said.

ChatGPT Enterprise customers can request the configuration through the Trusted Access Program in the admin console. Teams that already have access select it in ChatGPT as gpt-6.2-astra-law. Customers apply for API access through an OpenAI form. GPT-6.2 Astra’s API price is $8 per million input tokens and $40 per million output tokens.

Report an error in this story