Artificial Analysis: Tencent’s Hy4 ranks second among Chinese models, with an 89 per cent hallucination rate
Written by AI
Independent evaluator Artificial Analysis gave Tencent’s Hy4 model 35 points, the second-highest score among models from Chinese labs, and rated it strong at agentic coding. In AA’s knowledge test, the model mostly answered wrongly instead of declining to answer when it did not know the answer.
Independent evaluation firm Artificial Analysis (AA) published results for Tencent’s large language model Hy4 on 5 November. Hy4 scored 35 points on AA’s Intelligence Index v5, level with Grok 4.7 (xhigh) and one point behind Claude Haiku 5.5 (max). Among models from Chinese labs, only Moonshot AI’s Kimi K3.1 Pro ranks higher, with 46 points.
The index combines nine benchmarks. An AI-compiled AREX review says 60 per cent of the weighting falls on tests whose data or answers are not public. Still, the result comes from a single evaluator, and the index is AA’s own construct. On AA’s model page, Hy4 sits below the median (37) for models in the same price class.
Close to Claude Fable 5.1 on FrontierCode
FrontierCode 2.0 is a private benchmark from Cognition. It scores each task by whether the repository maintainer would accept the change the agent made. Hy4 scored 31 per cent, Claude Fable 5.1 32 per cent and Kimi K3.1 Pro 25 per cent. The result matches Tencent’s own figure.
On Terminal-Bench 4.0, AA measured 37 per cent, about seven percentage points below the 44.4 that Tencent reported.
Hallucination rate reflects guessing
On the AA-Omniscience v1.1 knowledge test, Hy4 answered 50 per cent of questions correctly and Kimi K3.1 Pro 49 per cent. But Hy4’s hallucination rate was 89 per cent, against Kimi’s 37 per cent. When Hy4 lacks the knowledge, it often gives a wrong answer instead of leaving the question unanswered, heise wrote.
Tencent said hallucinations had fallen since the preview version: its press release gives a drop from 12.95 to 7.2 per cent, and its blog reports a fall from 15.76 to 5.45 per cent in a search-optimised internal test. AREX notes that these figures come from different tests and cannot be compared directly with AA’s result.
Price, demand and availability
AREX and heise reported that AA put Hy4’s cost at about $0.78 per index task. Grok 4.7, with the same score, cost about ten times as much, and Haiku 5.5, one point higher, cost about half as much. AA’s model page now shows $1.03. The page says Hy4 generated 160 million output tokens in the tests, against a median of 72.6 million for comparable models.
The list price is $0.834 per 1 million input tokens and $2.501 per 1 million output tokens. Tencent Cloud is offering a 50 per cent discount until the end of the year, heise reported.
On 5 November, Tencent said it had expanded Hy4’s inference cluster tenfold because usage in WorkBuddy, its office agent, grew quickly and many users hit their usage limit on the first day. The company raised free usage limits but warned that restrictions may still apply at peak times.
Tencent says Hy4 ranks 13th on the Agent Arena leaderboard and is the best open-weight model there; the listing is provisional. AREX reported that Arena.ai placed the model 14th on launch day and said it would share the top spot among open-weight models once its weights are released. The weights had not been released as of 5 November, although Tencent promised them within days.
Hy4 is available in Tencent’s apps, through the Tencent Cloud API and OpenRouter, and, according to Tencent’s X account, in OpenClaw.