Research dataAi document review
Redgrave's 88 percent against 64 percent doesn't show generative AI beating managed review
In June 2026, a Redgrave LLP working paper reported that Relativity aiR for Review reached 88 percent recall of responsive documents. A managed review using Relativity Active Learning reached 64 percent. The corpus was 45,004 documents from the Mallinckrodt collection in the UCSF Opioid Industry Documents Archive. Earlier this month, Gordon Cormack and Maura Grossman of the University of Waterloo and Jeremy Pickens of Elevate posted a preprint that reanalyses the same data, which Redgrave shared with them. They argue the two figures measure different tasks. Scored against the same human-coding gate, they say, aiR is "comparable to, but not superior to" the managed review. EDRM republished ComplexDiscovery's account of the exchange.
What are the two papers actually disagreeing about?
Redgrave says aiR found 88 percent of responsive material and managed review 64 percent. The Waterloo and Elevate authors say those percentages describe workflows with different human roles, so the gap overstates aiR's lead.
Redgrave set a demanding standard. A document counted as responsive only if it showed compliance with, violation of, or reckless disregard of federal rules on drug-company marketing, including controlled-substance requirements. On the AI side, Ray Mangum refined an aiR prompt over repeated iterations, then ran it across the corpus in one pass.
The reanalysis turns on one design difference. In one arm, contract reviewers had the final say on what was responsive. In the other, they did not.
Why does it matter whether a human has the final say?
Recall depends on where the human gate sits. If reviewers can overrule the software in one arm but not the other, the two percentages come from different processes and are not a like-for-like score.
This note draws on the republished account, not the preprint. That account doesn't make clear to me which arm carried the gate, so read the paper before repeating either number. If the gate sat only in the managed-review arm, the 64 percent figure includes reviewer decisions that the 88 percent figure never faced.
Weigh the authors accordingly. Redgrave designed and ran the experiment it reported. The reanalysis is a preprint built on data Redgrave supplied. ComplexDiscovery's editor also discloses that he is chief marketing officer of HaystackID, a review provider. None of that discredits the argument, but it is not independent replication.
Does a recall percentage tell me what a review will cost?
No. Recall reports how much responsive material a workflow found. It leaves out prompt-tuning effort, reviewer time and validation work, which is where budgets actually move.
The aiR result followed a long prompt-iteration session by a Redgrave lawyer before the single pass over the corpus. That effort is a real cost and sits outside the headline figure. A high recall number against a strict responsiveness standard is useful, but it needs precision and effort reported beside it.
What should I ask my own vendor?
Ask who defined responsiveness, who coded ground truth, and whether every arm passed through the same human gate before recall was calculated. A vendor who cannot answer has a marketing number, not a benchmark.
One angle the coverage leaves open: both figures are recall on responsive documents. Neither tells you whether a system answers plain-language questions accurately, or whether each answer links to the exhibit it relies on. If you are buying retrieval-and-citation tooling, ask for those measures separately.
Two further things to watch. Redgrave has reserved a manual review of the full corpus for future work. And a protocol clause written around "88 percent" would tie you to a contested number, so keep it out of ESI negotiations until the dispute is settled.
Frequently asked questions
Does the reanalysis show aiR is worse than managed review?
No. The authors report the two as comparable, not that aiR trails. Their claim is that the original gap came from comparing different tasks.
Has either paper been peer reviewed?
No. Redgrave's paper describes itself as a working paper that has not been peer reviewed, and the reanalysis is a preprint posted to arXiv.
What is still outstanding?
Redgrave says it has reserved a manual review of the full corpus for future work. That analysis could test both sets of figures against a common ground truth.
Sources: EDRM, republishing Rob Robinson's ComplexDiscovery article. I did not review the arXiv preprint or the Redgrave working paper directly.
The original report
Eighty-eight percent against 64 percent was the comparison that traveled. A new research paper built on the same 45,004 documents argues the two figures never measured the same task. Measured on a common task, its authors say, the generative AI tool’s lead over the managed review it was tested against largely vanishes.
EDRM
Industry headlines from other publications. Each links to the original reporting on the publisher's own site.