AI Aulis

AI NEWS IN ENGLISH · 15 November 2026

The most important AI news every morning. AI writes and checks the stories against the original sources – check important facts in the sources.

26 stories · next issue tomorrow morning · How the paper is made →

← Issue 15 November 2026Paper

OpenRouter test: Voyage model raised top-result accuracy from 81 to 97 per cent

AI-generated contentWritten by AI

github.com, openrouter.ai · 3 sources · Sources published: 13 November 2026 · Some source publication dates are unknown

In OpenRouter’s own test on a small synthetic dataset, vector search alone ranked the correct text chunk first for 81 per cent of questions. When voyageai/rerank-3-lite reranked 30 candidates, the share rose to 97 per cent for about $0.10 per 1,000 queries.

OpenRouter published a comparison of reranking in RAG applications on its blog on 13 November. A RAG application retrieves text chunks from an organisation’s own documents and passes them to a large language model as the basis for an answer. The company compared 11 rerankers, along with their latency and price.

Why vector search gets it wrong

Vector search condenses each chunk into a single vector before the question is known. In a 130-word chunk, the name of a service tier takes up only a word or two, OpenRouter said, so chunks describing the limits of different tiers look almost identical to the search. When the test asked about the Growth tier’s bucket limit, google/gemini-embedding-2 ranked the Starter, Enterprise and Scale chunks highest. The correct chunk came only fifth.

A reranker reads the question together with each candidate and rescores the candidates. It is too slow to run over the whole dataset, so vector search narrows down the candidates first. OpenRouter recommends retrieving about 30 candidates and passing the top three to the large language model.

The test dataset contained 274 model-generated chunks and questions about Fernbank, a fictional storage service. Each question had one correct chunk, so results were scored automatically. The company warns that the dataset is small and synthetic, and that the benefit may differ on real documents.

Results and prices

All models reranked the same 30 candidates retrieved by gemini-embedding-2. The table shows how often the correct chunk came first (precision@1).

Model Correct first Median latency (s) Price / 1,000 queries
No reranking 81.0% – –
voyageai/rerank-3 97.8% 1.27 $0.26
voyageai/rerank-3-lite 97.1% 1.26 $0.10
cohere/rerank-4-pro 95.3% 1.54 $2.50
qwen/qwen3-reranker-8b 92.3% 1.51 $1.49
cohere/rerank-v3.5 90.5% 1.47 $1.00
openai/gpt-6-luna (large language model) 99.6% 4.01 $0.70
google/gemini-3.8-flash (large language model) 100% 3.25 $4.14

OpenRouter run on 8 October 2026, 274 questions, selection of 11 models. About 0.9 s of the latency is network latency from the test machine.

The two Voyage models differ by only two questions, and OpenRouter said this does not show that the more expensive model is better. The large language models read all 30 chunks at once and were the most accurate. Their median latency was 3.3–4.4 seconds, against about 1.3 for the rerankers. The company said it is not certain that the advantage holds on messier data.

Given three reranked chunks, anthropic/claude-haiku-5.5 answered 99.3 per cent of questions correctly. With three chunks and no reranking, the figure was 95.6 per cent. Compared with five chunks without reranking, total cost rose by a net $0.04 per 1,000 questions. Reranking cost $0.10, but the large language model’s answering cost fell from $0.22 to $0.16 because it received three chunks instead of five.

Limits

Reranking can only promote a candidate that vector search has found. When voyageai/voyage-4-lite retrieved 50 candidates instead of 30, the result improved by one question. A chunk that matches the topic is not always the right one either: in the blog’s six-chunk example, an unanswered forum question rose to second place.

cohere/rerank-v3.5 reads only the first 4,096 tokens of a document and gives no warning of the truncation. When the embedding model call omitted the input_type parameter, results fell by 7–12 percentage points.

Rerankers are used through OpenRouter’s /rerank endpoint; large language models do not work with it. Voyage and Qwen charge per token, Cohere per search. The example code in the documentation’s RAG guide differs from the test: it uses cohere/rerank-v3.5, does not set the input_type parameter and recommends 10–20 candidates.

Report an error in this story