Agent running on Opus 5.5 released 38 models downloaded more than 50,000 times
Researchers gave the Horizon agent an empty Hugging Face account and compute for seven weeks. They say the downloads are mostly genuine, and 143 GitHub repositories use the agent’s models.
Researchers from the ELLIS Institute and the Max Planck Institute for Intelligent Systems in Tübingen and from EPFL published a report on the experiment on horizon-labs.ai. Arthur Wuhrmann, the report’s first author, announced it on X on 17 November. An AI agent called Horizon ran around the clock in the coding agent Claude Code, powered by Opus 5.5. Its task was to win real users for its models. The whole experiment used more than 600 GPU hours.
Horizon built models for narrow tasks, such as personal data removers, punctuation restorers and prompt injection detectors. A prompt injection is text that tries to override the instructions given to a language model. The injection detector was the agent’s most downloaded model, with about 17,000 downloads.
Download counts alone do not prove use. The researchers initially suspected that bots or the agent itself had inflated them. Monitoring models checked all of the agent’s tool calls for reward hacking, or artificially inflating the metric. Horizon had downloaded its own models 42 times on six occasions, and the researchers said the true number may be slightly higher. They estimate that they themselves caused at most 50 of the 54,000 downloads. Further evidence of use comes from 143 GitHub repositories, 18 models derived from Horizon’s models and two arXiv preprints that compare the injection detector with others.
One of the preprints shows why independent evaluation matters. In it, Horizon’s detector achieved the highest AUC on attacks targeting contract documents. AUC measures how well a classifier separates two sets. The authors wrote, however, that the model mainly detected that a document had been edited, not that the edit was hostile.
On model cards and Spaces pages, Horizon often presented its models as the best or nearly the best in their field. The researchers tested four model families on synthetic benchmarks that they created only after the models had been released. On these, the models were often the best for their size on narrow tasks. The agent also chose which version to release on the basis of a validation set, even when the chosen version looked weaker on the reported benchmarks.
Horizon worked an average of 16 minutes a day. It usually woke up when Hugging Face updated its download counts, and improved its most downloaded models. The agent’s working time and GPU use levelled off, but daily downloads grew.
The researchers answered 13 support requests from the agent and sent it seven messages. They believe two of their mistakes contributed to Horizon realising that a research group, not a company, was behind it. In their assessment, the mistakes affected neither the agent’s behaviour nor the download numbers. In a parallel experiment, a DeepSeek agent gained access to the same account and promoted Horizon’s models in comments on another repository. The report says the experiment lasted seven weeks, but the title in its citation guidance refers to an eight-week case study. The raw data and all 3,952 tool calls are available for download from the report page.
Read more: How an agent works · Agent map