Tools nearly doubled models’ average score on the VIALS image benchmark
Written by AI
In an Artificial Analysis benchmark, language models interpreted life-sciences images better when they could run code in a sandbox and open images. A cheaper model with tools can beat a large model without tools on both price and score in this kind of image interpretation, the company said.
Benchmarking firm Artificial Analysis published a comparison on X on 12 November in which it ran the same scientific-image benchmark in two ways. Models either used tools as part of an agent or answered a single request without tools. Tools raised models’ scores from about 25 per cent to about 47 per cent on average, the company said.
What the benchmark measures
VIALS-AA is Artificial Analysis’s implementation of Handshake AI’s VIALS benchmark. In its 161 tasks, a model interprets images used in life-sciences work, such as blot images, microscopy images, flow cytometry plots, plasmid maps and phylogenetic trees. The VIALS research paper has been published as a preprint on arXiv.
By default, models run in the Stirrup harness. A harness is a program that controls an agent: it gives the model tools and runs its steps. The model can run Python code in a sandbox with no network access and open images with a separate tool. In the comparison run, the model received the question and images in a single request without tools.
Each task is run three times, and the language model GPT-6 Luna grades the answers. The score (pass^3) is the share of tasks the model answered correctly in all three runs. A refusal or a missing answer scores zero.
Results
Claude Fable 5.2 (max) scored 67 per cent with tools and 44 per cent without. Claude Haiku 5.5 (max) reached 49 per cent with tools, higher than Fable without tools. Of the models in the published chart, the best without tools was GPT-6 Astra (max), at 50 per cent.
The figures show the effect of tools in this image-interpretation benchmark, not which models are better in general.
Costs
Tools raised every model’s cost per task, but by different amounts. Fable 5.2’s cost rose from $0.73 to $4.20, a 5.8-fold increase; GPT-6 Astra’s from $0.21 to $0.27; and GPT-6.1 Sol’s from $0.05 to $0.06. Costs grew most for Qwen3.8 Max (28-fold), Gemini 3.8 Flash (20-fold) and MiMo-V2.6-Pro (15-fold). Artificial Analysis calculated costs from token counts and model prices, assuming a typical share of cache hits.
Artificial Analysis concludes that, when interpreting scientific images, a cheaper model in a harness can beat a large model answering a single request on both price and score. In the company’s example, Haiku with tools scores 49 per cent at $0.07 per task, while Claude Opus 5.5 without tools scores 34 per cent at $0.33.
Read more: Models