Filevine splits contract review across parallel AI agents
In the latest version of Filevine’s LOIS for Word, several AI agents review a contract at the same time and the system merges their conflicting redlines. In the company’s own measurements, users rated 87 per cent of redlines acceptable as written, up from 64 per cent in the previous version.
Mariam Sulakian of the legal technology company Filevine described in a company blog post on 12 November how contract review in LOIS for Word has been rebuilt three times in three years. The program reviews a contract against a company’s playbook and suggests edits.
The first version, Review Classic, turned the playbook into rules, and for each rule the model decided whether the contract met it. Semantic search picked out the most relevant passages for each rule. A review usually took less than 15 seconds, but the search could miss an exception elsewhere in the contract.
In Agent Review, a single AI agent decided for itself what to read and where to make edits. Edits made in the wrong place fell by more than 60 per cent in production, and the share of application errors dropped from 17 per cent to 9 per cent, the company said. But the agent checked issues one at a time, and there was no reliable way to track whether it had checked every requirement.
A coordinating agent keeps track of stored findings
In Review 2.0, a coordinating agent hands parallel agents playbook requirements, the counterparty’s edits and issues the agents have found themselves. One agent can check confidentiality while another checks limitation of liability. All agents read the same stored version of the contract.
For each review item, the system stores the findings, the quotations supporting them and the redlines. The coordinating agent decides which findings to keep or merge, and gives a reason for every finding it drops. The server rejects a submission that leaves out a finding or includes it twice.
When the coordinating agent’s context window fills up, the system builds it a new context from the stored data. Sulakian wrote that the model may still reach a different conclusion on a new call. An interrupted review resumes from where it was saved.
The system merges conflicting redlines
Parallel agents may propose conflicting edits to the same clause. In the blog’s example, one agent would shorten a ten-year obligation in a non-disclosure agreement to seven years, while another would apply the agreement’s general two-year term to the same clause. The system detects such overlapping redlines, and an agent merges them into a single edit. If the merge fails, the system makes none of the conflicting edits and leaves their rationale as comments. Other edits can still go ahead.
The results come from the company’s own tests
On Filevine’s benchmark of 50 contracts, Review 2.0 found 91 per cent of issues (recall), and 89 per cent of its findings were correct (precision). Users rated 87 per cent of redlines acceptable as written, up from 64 per cent in Agent Review. Of the accepted edits, 96 per cent landed in the correct place at the first attempt.
In a small test with six conflicting edits, parallel merging took 5.5 seconds on average and sequential merging 30 seconds. All figures are the company’s own measurements.
Customers also use LOIS to review contracts that arrive by email and in Slack, so the first pass is done before the user opens Word, the company said. Filevine’s website offers a free trial of LOIS with no credit card required.