GPT models found the most agent risks in a benchmark when false alarms were not capped
In ReA-Bench, a benchmark from Alibaba researchers, GPT models found the most risks in AI agent trajectories, but they also raised alarms on more than half of the difficult benign trajectories. When the false-alarm rate was capped, Grok and DeepSeek models found more risks.
Researchers from Alibaba’s AAIG group published a preprint on arXiv on 12 November describing ReA-Bench, a benchmark that measures how language models audit the trajectories of AI agents. The auditing model reads the agent’s steps and judges whether the agent acted riskily. The steps include, for example, tool calls and their responses. The study has not been peer-reviewed.
The usual metric is the share of risks found (recall). The authors said it does not show which risks go unnoticed, nor whether higher recall brings more false alarms. In their analysis of production and red-team trajectories, some risks were repeatedly missed across different auditing configurations.
Six tiers and difficult benign trajectories
The benchmark contains 587 trajectories: 304 risky, 177 benign and 106 control versions. Most were taken from real production runs or adapted from them. The tasks fall into six tiers:
- security knowledge
- contradictions within a trajectory
- finding evidence in a long trajectory
- checking sources and reasoning
- gradual steering over several turns
- missing information.
The benign trajectories were deliberately chosen to be difficult. In them, the agent does things that look risky, but the task and the permissions the user granted justify them. The authors said these trajectories do not reflect ordinary production runs, so the false-alarm rate measured in the benchmark cannot be converted directly into production figures.
The models that found the most risks often raised false alarms
The researchers tested eight model configurations with four common auditing methods. The first two numeric columns show the average across the four methods at an alarm threshold of 5 on a 0–10 scale. The last column shows the best recall when no more than 10 per cent of benign trajectories are flagged.
| Model | Recall | False alarms | Recall, false alarms ≤ 10% |
|---|---|---|---|
| DeepSeek-Flash | 59.7% | 3.8% | 67.8% |
| DeepSeek-Pro | 50.6% | 3.8% | 71.1% |
| Qwen-Flash | 63.7% | 8.6% | 68.5% |
| Qwen-max | 73.6% | 12.9% | 71.8% |
| Grok-low | 65.4% | 5.4% | 74.6% |
| Grok-high | 71.0% | 5.5% | 78.0% |
| GPT-luna | 87.0% | 52.0% | 50.5% |
| GPT-sol | 88.5% | 54.0% | 47.3% |
Source: ReA-Bench preprint, tiers 1–5. The false-alarm rate was measured on the benchmark’s difficult benign trajectories.
At the fixed threshold, the GPT models found the most risks. When false alarms were capped, they fell behind the others. Each configuration was run only once, so small differences need to be confirmed in repeat runs, the authors said.
The models were generally better at identifying risks that required security knowledge than at spotting contradictions, unsupported claims and reasoning errors. Shortening the trajectory did not always help. In one trajectory, the agent had used UTC timestamps although the reporting period was defined in Beijing time. The models audited the trajectory 28 times with different configurations and auditing methods. They found the error 12 times in the original trajectory of about 220,000 characters and 13 times in a version shortened to about 13,000 characters.
The authors also tested GSA, their own note-compiling method. At the fixed threshold, GSA found more risks than the highest-recall baseline in all four configurations, but it also raised more false alarms. Across all thresholds, GSA discriminated risks better in three configurations but worse with GPT-luna.
The authors plan to release the benchmark openly once identifying information has been removed from the data and the release has been reviewed.