Swarm of 32 Opus agents sped up NumPy workloads 18.9 times in Replit test
In Replit’s own test, a swarm of 32 Opus 5.5 agents made NumPy’s 13 workloads a median 18.9 times faster in two hours. A single agent made them 3.7 times faster. The researchers said swarms of Anthropic’s models achieved bigger speed-ups the more independent work a codebase offered.
Replit researchers described an experiment in a blog post on 19 November. In it, coding agents had to speed up open-source libraries without changing their output. Each agent had its own sandbox, 8 vCPUs and two hours, so a 32-agent swarm used many times more compute in the same time. The agents decided for themselves how to divide the work.
A separate machine that the agents could not access built the original and the modified code. It compared the output of both versions, including on inputs hidden from the agents, and ran them alternately in random order. The result is the median speed-up across workloads.
On NumPy, a single Claude Opus 5.5 agent made the workloads 3.66 times faster, and a 32-agent swarm made them 18.92 times faster. One example is the searchsorted function, which finds where each value belongs in a sorted array. The Opus swarm changed it to process 256 search keys at a time, so that each batch fits in the cache. It also skips the slower NaN-safe comparison when neither array contains NaN values. On float64 inputs with many keys, the function became 8.7–13.1 times faster.
However, on NumPy, SQLite and SQLGlot, Opus swarms did better than swarms of the strongest solo model. GPT-6.2 Astra was the strongest solo model on all four reported tasks, but as a 32-agent swarm it fell short of its own solo result on NumPy, SQLGlot and Git. On Git, all seven workloads touched the same 15 files. There, even swarms of Anthropic’s models barely improved on a single agent, and GPT-6.2 Astra matched or beat Opus at every swarm size.
The researchers said the way the agents organised themselves affected a swarm’s result more than the model’s solo performance did. Opus sent 82 per cent of its messages to a shared board, where worker agents claimed workloads and reported their findings. GPT-6 Astra’s workers sent the coordinator about 22 direct messages each on average and GPT-6.1 Sol’s workers about 20, against 1.4 for Opus workers. When the swarm grew from 16 to 32 agents, Sol’s coordinator received 631 messages on the SQLGlot task instead of 356.
Replit obtained the results with its own benchmark and on its own platform, and the sources do not mention any independent replication. The blog post covers four of the benchmark’s 14 tasks. The task specifications and the evaluation program are on GitHub under the Apache 2.0 licence, but an officially valid result requires a Google Cloud machine specified by Replit.
Read more: How an agent works · Agent map