AI Aulis

AI NEWS IN ENGLISH · 14 November 2026

The most important AI news every morning. AI writes and checks the stories against the original sources – check important facts in the sources.

← Issue 14 November 2026Paper

Preprint: decision model JEV, weak as an actor, taught a small model in text-based tasks

AI-generated contentWritten by AI

arXiv, github.com, huggingface.co, alphaxiv.org · 4 sources · Sources published: 12 November 2026 · Some source publication dates are unknown

A decision model that performs poorly at tasks itself can still give useful feedback in reinforcement learning when it judges, after the whole trajectory, whether each action helped achieve the goal, researchers at Zhejiang University and Baidu say.

The researchers posted the results on arXiv on 12 November. The paper is a preprint and has not been peer-reviewed.

A decision model is a pretrained model that does not write free text. For example, it chooses among given options and assigns each one a probability. The researchers compared three decision models with three large language models given the same questions in ALFWorld, a text-based environment where an agent carries out household tasks. Only JEV was tested as a teacher, in ALFWorld and in WebShop, which simulates an online shop.

The models played three roles. As an actor, a model chooses the agent’s next action. As a judge, it scores the actions in a recorded trajectory, and as a teacher, it scores the actions of the model being trained.

The actor picks the easier subgoal

All six models succeeded less often as actors when subgoals had to be completed in order, such as heating a mug before putting it on the table. TypeSafe AI’s closed JEV completed 70.8 per cent of tasks whose subgoals could be done in any order, but only 14.7 per cent of those that required a set order.

The models still knew the required step, the researchers said. In states where a single action would have completed the easier subgoal, removing that subgoal from the task instruction raised the probability JEV assigned to the required step from 14.6 to 77.6 per cent. When the prompt reminded the model of the dependency, the success rate changed by only −2.8 to +5.2 percentage points.

In judging, the form of the question was decisive

The judge saw either the trajectory up to the action being scored or the whole trajectory. It was asked either whether the task would succeed or whether the action helped achieve the goal. With the whole trajectory visible and the helpfulness question, JEV scored actions that completed a subgoal 0.36 higher than actions in the same trajectory that changed nothing. With the success question the gap was 0.05, because the answer is the same for every action: the outcome.

As a teacher, JEV scored every action that Qwen2.5 models with 1.5 billion and 7 billion parameters took in training trajectories. The researchers normalised the scores against the mean for trajectories of the same task and added them to the GRPO signal, which otherwise gives every action the same value based on the outcome.

The table shows the success rates of the trained models. As an actor, JEV completed 42.9 per cent of the same ALFWorld tasks.

Training ALFWorld WebShop
GRPO, 1.5B 72.8 56.8
Whole trajectory + success, 1.5B 69.3 68.3
Whole trajectory + helpfulness, 1.5B 82.9 71.4
GRPO, 7B 77.6 66.1
Whole trajectory + helpfulness, 7B 83.8 73.6

Source: preprint Table 3, success rate in per cent.

An AI-generated summary on alphaXiv says the figures are averages of three evaluation runs and that the GRPO figures were taken from the GiGPO paper without being rerun.

The authors explain that a judge working after the fact sees consequences the actor would have to anticipate. The same summary notes, however, that the experiment does not show the improvement comes from this alone.

Limits and availability

The results cover two text environments with at most 30 actions per trajectory. Decision models are less suited to filtering actions mid-trajectory, because the end is not yet visible, the authors said.

JEV’s scores were obtained through OpenRouter and cost $22–30 per training run. The teacher is not needed when the finished model is used. The code is on GitHub under the Apache 2.0 licence.

Report an error in this story