Legal technology company Filevine described in a blog post on 9 November how its AI tools find the most relevant passages in documents for a language model to use. The author, Jakub Žovák, wrote that many AI applications succeed or fail precisely because of retrieval.

The starting point is RAG, or retrieval-augmented generation. An embedding model converts documents into vectors, or sequences of numbers, so that texts with similar meanings have vectors close to each other. The system fetches the texts closest to the question from a vector database, and a stronger model re-scores the candidates.

Filevine regards hybrid retrieval as the industry’s established approach. Hybrid retrieval runs an exact keyword search alongside vector search. Reciprocal rank fusion (RRF) merges the results, boosting documents that rank highly in both searches.

Hybrid retrieval is a good baseline but falls short of what the legal domain requires, the company said. The blog contains no measurements, so the superiority of its own methods remains the company’s claim.

Metadata as part of the vector

Legal documents carry metadata such as court level and citation count. Many AI systems can use metadata as a hard filter. Filevine says it supports soft filtering, which can express a preference: a document ideally from an appellate court or higher.

The simplest method is to re-rank similarity search results by metadata. Zilliz’s vector database, for example, supports this. The blog says the method fails if the search returns only trial court documents. In that case, no desired results remain.

Filevine says it can embed metadata directly into vectors with specialised models and weight it at query time. In an example in a blog footnote, text similarity and the document’s metadata value each count for half. The company cites the paper Utilizing Metadata for Better Retrieval-Augmented Generation as background to the method.

Synonyms and token-level comparison

Traditional keyword search does not recognise synonyms or homonyms and stumbles over legal terms, the blog says. As a fix, Filevine says it can use learned sparse retrieval. In this approach, a model weights words by their importance in context and adds related words missing from the text to the search. Ready-made models such as SPLADE are general-purpose, but the company has its own models fine-tuned for legal work.

A standard embedding model condenses a text chunk into a single vector by averaging the per-token vectors, losing some information. The ColBERT method keeps a separate vector for each token. For every token in the question, it finds the best-matching token in the document and sums the similarities of these matches. This typically makes retrieval more expressive, the blog says. The method also shows which part of the question matched which passage.

Multi-vector retrieval has mostly been used for re-ranking because its latency is higher than that of single-vector retrieval. Filevine says Google’s Muvera index removes the latency problem. The company says it can use multi-vector representation for both retrieval and re-ranking.

The blog ends by urging readers to try LOIS, the company’s legal AI. The free trial requires no credit card, according to the company’s website.