Alma was third in FrontierHarness interim results and finished sixth by pass rate
Alma, built by independent developer yetone, was third in an unfinished snapshot of Runta’s FrontierHarness v1.1 benchmark. On the completed leaderboard it is sixth by pass rate and 13th by cost per pass.
On 15 November, Pandaily, a news site covering China’s technology sector, compared an unfinished snapshot of the benchmark with the completed leaderboard. It reported that the order of the harnesses changed during the run.
FrontierHarness v1.1 is an open benchmark for coding-agent harnesses from Runta, an agent infrastructure company. A harness is the program that controls an agent: it gives the model its tools and runs its steps. The benchmark covers 21 harness configurations, including Claude Code, Codex CLI and OpenCode. They tackle 62 tasks in software engineering, science, data, systems, machine learning and security, using five open-weight models. Each harness gets four scored attempts per task with each model, and only verified passes count.
Interim and final results differed
Chinese tech outlet Zhixingxing reported on a snapshot taken on 11 November, when the run was still under way and the rankings changed daily. Pandaily said Zhixingxing reported Alma’s success rate at that point as 72.7 per cent. Codex CLI (77.9 per cent) and Pi (72.8 per cent) were ahead, and Claude Code trailed at 70.0 per cent. Alma cost about $0.80 per successful task, against $0.94 for Claude Code.
The completed leaderboard contains 26,040 scored attempts, and harness success rates are generally lower than in the interim snapshot. The table shows the top six by pass rate.
| Harness | Pass rate |
|---|---|
| ZCode | 61.9% |
| Codex CLI | 61.1% |
| KimiCode | 60.2% |
| Grok Build | 60.0% |
| Hermes Agent | 59.9% |
| Alma | 59.7% |
Source: completed FrontierHarness v1.1 leaderboard, as checked by Pandaily on 15 November.
The gaps among the top six are small: 1.4 percentage points separate second and sixth place. Pandaily does not say whether the differences are statistically significant. All 21 harnesses have pass rates between 50.6 and 61.9 per cent.
By default, the leaderboard ranks harnesses by cost per pass. On that measure, Alma is 13th at $1.22. Codex CLI costs $0.937 and Claude Code $1.55. Alma achieved its best pass rate, 64.1 per cent, with the Kimi-K3 model.
What Alma does
Alma reads and edits files, runs terminal commands, searches the web and supports skills, MCP and multiple models. Pandaily describes its additional features on the basis of Zhixingxing’s review and Alma’s website. Users can choose a separate model for tool calls (Tool Model), so simple actions do not need the more expensive main model. Long-term memory searches past conversations by meaning: it first rewrites the query, then filters the results with a similarity threshold. The herdr integration gathers Claude Code, Codex and Pi sessions in one window, grouped by status, such as Working, Done and Needs You.
The iPhone and Android apps are in public beta. They let users follow progress remotely, answer the agent’s questions and approve commands. For remote access, yetone released Alma Relay, an open-source relay server that can run on Cloudflare Workers or with Docker. The relay’s documentation says end-to-end encryption is on by default; with it on, the relay sees only encrypted data and stores nothing on disk.
Alma’s website is alma.now, and the Alma Relay code is on GitHub (yetone/alma-relay).
Read more: How an agent works · Agent map