AI Aulis

AI NEWS IN ENGLISH · 16 November 2026

The most important AI news every morning. AI writes and checks the stories against the original sources – check important facts in the sources.

24 stories · next issue tomorrow morning · How the paper is made →

← Issue 16 November 2026Paper

Alma was third in FrontierHarness interim results and finished sixth by pass rate

AI-generated contentWritten by AI

pandaily.com · Source published: 15 November 2026 at 09:12 (UTC+02:00)

Alma, built by independent developer yetone, was third in an unfinished snapshot of Runta’s FrontierHarness v1.1 benchmark. On the completed leaderboard it is sixth by pass rate and 13th by cost per pass.

On 15 November, Pandaily, a news site covering China’s technology sector, compared an unfinished snapshot of the benchmark with the completed leaderboard. It reported that the order of the harnesses changed during the run.

FrontierHarness v1.1 is an open benchmark for coding-agent harnesses from Runta, an agent infrastructure company. A harness is the program that controls an agent: it gives the model its tools and runs its steps. The benchmark covers 21 harness configurations, including Claude Code, Codex CLI and OpenCode. They tackle 62 tasks in software engineering, science, data, systems, machine learning and security, using five open-weight models. Each harness gets four scored attempts per task with each model, and only verified passes count.

Interim and final results differed

Chinese tech outlet Zhixingxing reported on a snapshot taken on 11 November, when the run was still under way and the rankings changed daily. Pandaily said Zhixingxing reported Alma’s success rate at that point as 72.7 per cent. Codex CLI (77.9 per cent) and Pi (72.8 per cent) were ahead, and Claude Code trailed at 70.0 per cent. Alma cost about $0.80 per successful task, against $0.94 for Claude Code.

The completed leaderboard contains 26,040 scored attempts, and harness success rates are generally lower than in the interim snapshot. The table shows the top six by pass rate.

Harness Pass rate
ZCode 61.9%
Codex CLI 61.1%
KimiCode 60.2%
Grok Build 60.0%
Hermes Agent 59.9%
Alma 59.7%

Source: completed FrontierHarness v1.1 leaderboard, as checked by Pandaily on 15 November.

The gaps among the top six are small: 1.4 percentage points separate second and sixth place. Pandaily does not say whether the differences are statistically significant. All 21 harnesses have pass rates between 50.6 and 61.9 per cent.

By default, the leaderboard ranks harnesses by cost per pass. On that measure, Alma is 13th at $1.22. Codex CLI costs $0.937 and Claude Code $1.55. Alma achieved its best pass rate, 64.1 per cent, with the Kimi-K3 model.

What Alma does

Alma reads and edits files, runs terminal commands, searches the web and supports skills, MCP and multiple models. Pandaily describes its additional features on the basis of Zhixingxing’s review and Alma’s website. Users can choose a separate model for tool calls (Tool Model), so simple actions do not need the more expensive main model. Long-term memory searches past conversations by meaning: it first rewrites the query, then filters the results with a similarity threshold. The herdr integration gathers Claude Code, Codex and Pi sessions in one window, grouped by status, such as Working, Done and Needs You.

The iPhone and Android apps are in public beta. They let users follow progress remotely, answer the agent’s questions and approve commands. For remote access, yetone released Alma Relay, an open-source relay server that can run on Cloudflare Workers or with Docker. The relay’s documentation says end-to-end encryption is on by default; with it on, the relay sees only encrypted data and stores nothing on disk.

Alma’s website is alma.now, and the Alma Relay code is on GitHub (yetone/alma-relay).

Report an error in this story