
Quick Answer
Agree on search terms in measured rounds: run each proposed term, share the hit report, code a random sample of hits, calculate precision per term, and revise before signing anything.
A list built this way carries its own evidence. When opposing counsel objects, you answer with samples instead of adjectives. When keywords stop improving, hybrid concept search or AI-assisted review can take over under the same validation checks.
Key Points
- A 2024 r/ediscovery thread on defending Relativity's aiR noted that precision and recall can be calculated for any review method , search terms included.
- Anith Mathai's 2019 guide shows a term returning 500 hits across 10,000 files but only 10 on one computer's 100 files , proof that testing small samples first pays.
- Microsoft 365 for IT Pros warns that a search preview is only a representative sample of what a full search finds, so hit reports need full-search counts.
A search-term negotiation runs on hit reports and coded samples, not on adjectives.
This piece covers one practical question: how to reach a search-term list with opposing counsel that still stands when someone attacks it six months later.
The answer is a method, not a meeting. Send every revised term list as an editable Word redline, not a PDF: Jordan Rothman of The Rothman Law Firm wrote in Above the Law in 2026 that receiving PDFs of documents that clearly need negotiation is "one of my biggest pet peeves." Each round leaves a paper trail. If the other side's revised list arrives locked, ask for the editable file at once: Rothman reports that one opposing lawyer admitted sending a PDF on purpose to make review and revision harder.
I have watched a great deal of advice on this subject stop at the words meet and confer, as though the phrase were an incantation and the documents would sort themselves out of sheer courtesy, like guests at a Long Island garden party filing politely toward the canapés. They do not. Hit counts arrive, tempers rise, and somebody drafts a motion.
Below, in order, you will find:
- How each round works, with the exact documents that change hands.
- What the evidence says about why tested lists outlast blind ones, including what practitioners report about validation thresholds.
- How to answer opposing counsel when they reject your terms or question how you got your numbers, the challenge litigators raise most often about the defensibility of an ESI collection.
- When keywords hit their ceiling and hybrid concept search or AI-assisted coding should take over, and which of those is easier to defend if the other side objects.
Small firms with one sprawling, messy collection will find the method especially useful, because the sampling that protects them in a dispute is the same sampling that tells them where their review budget is leaking.
Every term you concede unmeasured sets what your team pays to read, and review, argues John Tredennick of Merlin Search Technologies, is where "all the money (and the time) goes." Most firms skip the measuring.
I confess a weakness for the meet and confer as theater. Two lawyers, a conference line humming like a Bronx refrigerator, and a keyword list passed back and forth with the solemnity of a papal bull, although nobody on the call has seen a single document it retrieves. The list is then settled on adjectives such as overbroad, unduly burdensome and reasonable, which is a bit like pricing a used Packard by the poetry in its brochure.
The approach fails for a mechanical reason. A hit count measures volume, not relevance. Even the counts can mislead, because a search preview in Microsoft Purview shows only a representative sample of what the full search will find. A term list negotiated on those numbers is a cost commitment signed without an invoice.
Before the next call, put one plain question to opposing counsel: which of your proposed terms have you actually run, and against what data?
What is the Hit-Report Ladder, and how does each rung work?
It is a sequence of short rounds in which both sides run proposed terms against the real collection, sample the hits, measure precision, and revise before anyone signs anything.
Every rung asks the same three things of a term list, in this order:
- Run it. Draft the terms and run them against the actual collection to produce a hit report: total hits, unique hits per term, and the document families each term drags along.
- Sample it. Pull a random set of hits for each term, have an attorney code them responsive or not, and calculate precision, meaning the share of sampled hits that are genuinely responsive.
- Revise it. Keep the terms that earn their hits, narrow the ones that haul in noise, drop the ones that return nothing worth reading, then send the new report across the table and climb again.
The arithmetic behind the sampling step is not exotic. A 2024 r/ediscovery thread on defending Relativity's aiR against TAR 2.0 put it bluntly: precision and recall can be calculated for any review method, whether aiR, TAR 2.0, managed review, random selection, or search terms. Recall, briefly, is the share of all responsive documents a search actually finds. For search terms, the recall half of that check means coding a small random sample of the documents the terms did not hit and counting the responsive ones left behind. They merely tend to escape it.
Conventional wisdom holds that the meet and confer ends when both sides initial a keyword list. I think that is precisely backwards. A list negotiated blind carries roughly the evidentiary heft of a tip sheet sold outside Belmont Park, gorgeous in its confidence and useless by the fifth race. The Sedona Conference titled its guidance A Nutshell on Negotiating E-Discovery Search Protocols, and the noun it chose is telling. A protocol is a method. A list is merely its byproduct.
| Rung step | What you send opposing counsel | What it lets you argue |
|---|---|---|
| Run | Hit report with total and unique hits per term | Which terms dominate volume and cost |
| Sample | Coded random sample and precision per term | Which terms return responsive documents and which return noise |
| Revise | Redlined term list plus the next hit report | Why each change was made, with the numbers that justified it |
Our own product demonstration rests on the same instinct: bring a messy collection and a hard question, and watch the work happen on your evidence rather than reading a case study about someone else's. Terms behave the same way. They reveal their character only on the real data.
A comparison of 3 sources shows the pattern clearly: validation math, protocol thinking and testing on the actual collection all point to one conclusion. Measure first. Bargain second.
When the two sides' numbers refuse to agree on the very first run, the trouble is usually mechanical rather than adversarial, and a search equivalence test for mismatched hit counts will usually locate the culprit before anyone starts drafting a motion, which is where the blind list tends to end up.
Why does a term list tested in rounds hold up better than one agreed blind?
Search for "post" in a document collection and you will also pull up imposter, outpost and postponed. Nobody at the negotiating table sees that coming until somebody reads the hits.
Anith Mathai used that example in a 2019 guide to eDiscovery searches, written for small law firms on behalf of a cloud review service. His other traps are just as ordinary. A custodian named William may appear as Will, Bill, Willie or Willy. A keyword in an email footer can generate "hundreds or even thousands of 'false positives'." A hit count shows none of this. Reading a handful of hits shows all of it.
His fix was to work small. In his example, a term returns 500 hits across 10,000 files but only 10 on one computer's 100 files, few enough to review quickly. The test can then be rerun "five or six times" until relevant files outnumber the noise. "It ends up being much more efficient refining your searches on smaller sample sizes before applying them on entire cases," he wrote.
Reading the hits catches the obvious errors. Measuring them makes the result defensible. In a 2024 r/ediscovery thread on validating AI document review, practitioners said the math is the same whatever produced the calls. "Precision and recall can be calculated for any methods," one wrote, listing search terms beside technology-assisted review (TAR) and generative AI. Draw a random sample, have a subject-matter expert code it, then compare those calls with the method's calls. Another practitioner needed four words: "Defensibility is in the process."
A measured round also gives the dispute a legal footing. Federal Rule of Civil Procedure 26(b)(1) asks, among other factors, "whether the burden or expense of the proposed discovery outweighs its likely benefit." When most of a term's sampled hits are footers, that burden is on paper. Our own figures put manual review at roughly $19K per gigabyte. We sell AI-assisted review as the cheaper route, so weigh that number knowing our stake.
How good is good enough? The thread offers practitioner experience, and no source states a rule. The chart shows the scores commenters reported. One commenter also wanted the ESI protocol to require newer review methods to "use the same TAR metrics" as older ones. Nobody gave a threshold for search terms, so the two sides must set any target for a term list themselves.
Then come the exceptions, and a raw hit count hides them well. Mathai warned that excluded encrypted or corrupt files "could turn out to be useful." Office365itpros, whose authors track content search in a widely used cloud email suite, notes that some mailbox items are only partially indexed and that a search preview is "only a representative sample of what can be found by a full search." The errors run in both directions:
| Defect | What it does to the hit report | Source |
|---|---|---|
| Short terms such as "post" | Inflates hits with unrelated words | Mathai |
| Keyword repeated in footers | Can add hundreds or thousands of false positives | Mathai |
| Same file in different formats | Counts duplicates, since deduplication "isn't foolproof" | Mathai |
| Missing name variants (Bill for William) | Leaves relevant documents out | Mathai |
| Encrypted, corrupt or unreadable files | Excluded from search | Mathai |
| Scanned images without OCR | Invisible to keyword search | Mathai |
| Partially indexed mailbox items | Only part of the content is searchable | Office365itpros |
| Preview results | Can understate what a full search finds | Office365itpros |
Mathai argued that showing "proficiency with eDiscovery and openness" lowers the chance that courts will entertain objections from opposing counsel. Disclosing these gaps shows both.
The deepest blind spot sits inside precision itself. Precision measures how much of what the terms caught is relevant. It says nothing about relevant documents the terms never touched, and that is exactly where a missing nickname or a scan never run through OCR ends up. TAR 2.0 validation already samples unreviewed documents "to confirm nothing significant was missed," practitioners in the thread explain. No source here prescribes that step for search terms, so applying it to the documents your terms missed is our reading of the method. In this evidence, it is still the only measurement that speaks to recall.
Side by side, the sources split the work cleanly. Rounds of sampled hits make a term list cleaner. A sample of the misses shows whether the list is complete. Files that no search can read sit outside both piles, so the hit report should name them.
Before the next exchange of terms
- Run each proposed term on a small slice of the collection and read the hits before the list goes out. Fix short terms, footer matches and missing name variants, then rerun.
- Next to each hit count, report precision from a random sample coded by someone who knows the case.
- Add an exceptions line to every hit report: encrypted, corrupt or unreadable files, scans without OCR, partially indexed items, and whether counts come from a preview or a full search.
- Before calling the list final, agree with opposing counsel on a random sample of documents the terms missed, and on what finding there would reopen the list.
- Ask opposing counsel whether recall and precision targets for the term list belong in the ESI protocol, and which numbers both sides could defend.
How we checked this
We drew on a 2024 practitioner thread on r/ediscovery, a 2019 guide to eDiscovery searches by Anith Mathai, technical writing on email content search from Office365itpros, and the text of Federal Rule of Civil Procedure 26. The manual review cost of roughly $19K per gigabyte is our own figure. We sell AI-assisted review, so we have a stake in how review is measured. Mathai's guide promoted a cloud eDiscovery service for small firms. The thread's commenters are anonymous and we could not confirm their credentials. Their benchmarks describe TAR and generative AI review from personal experience and cover search terms only in principle, and none of them cited case law. Still unknown: what precision and recall courts will accept for a search-term list, and how large a sample of unhit documents is enough.
- r/ediscovery, practitioner thread on validating AI review, September 18, 2024.
- Anith Mathai, five tips for better eDiscovery searches, LinkedIn, March 25, 2019.
- Office365itpros, articles on content searches, 2018 to 2025.
- Legal Information Institute, Federal Rule of Civil Procedure 26, retrieved November 20, 2026.
- Relevant e-Discovery, internal figures on review costs, 2026.
How do you respond when opposing counsel rejects your terms or attacks your process?
Answer with the hit report and the coded sample, not adjectives. Once precision per term sits on the table, the quarrel moves from opinion to cost and evidence.
Two challenges arrive at nearly every meet and confer, and each has its own reply:
- "Your terms are too narrow." Run the term they want, report how many unique documents it adds that no existing term reaches, then sample those additions and report their precision.
- "We don't trust your numbers." Show how the numbers were produced: which collection, which tool, which date, which sample, and who coded it.
The first reply turns a demand into a price tag. Here the cost side matters enormously. In our own pricing comparison, AI-assisted review runs cents per document, against dollars per document and roughly $19K/GB for manual review. A broad term that pulls in mostly noise is therefore not a philosophical problem. It is an invoice. Precision tells both sides how much of that invoice buys anything.
Consider the difference in practice. Before: "Plaintiff's proposed terms are overbroad and unduly burdensome." Opposing counsel has read that sentence a thousand times and will brandish it back like a dowager's lorgnette. After: "The proposed term adds a block of unique documents that no other term reaches; in the coded random sample, the hits were mostly newsletters and calendar invitations; the sample log is attached." The second version invites a counteroffer. The first invites a motion.
The second challenge, the attack on method, is where careless teams get hurt. Microsoft 365 for IT Pros has warned that the items shown in a search preview are only a representative sample of what a full search can find. A hit report built from preview counts is thus a guess wearing a tie. The same source noted that Microsoft changed Purview eDiscovery from May 26, 2025, altering how content searches, case handling, review and export work. Tools shift beneath you. Record the version and the date for every run, or the numbers become unrepeatable at the worst possible moment.
Questions about method often reach back past search to collection itself, and the gaps that one-click cloud collection tools miss are a favorite opening for that line of attack.
Our platform was designed to be defensible on its face: immutable originals with content hashing, append-only audit trails, documented chain of custody, and a fail-closed privilege gate on production. For a term negotiation, that means every hit report and sample traces to an unaltered source. Frankly, that is the whole game. A number you cannot reproduce is a number you cannot defend.
None of this settles whether keyword terms should carry the entire load, which is the next question any small firm with a sprawling collection will reasonably ask.
Is hybrid keyword and concept search better than keyword search terms alone for litigation document review?
Often, yes. When sampled precision stops improving between rounds, keywords have plateaued, and an AI-assisted review checked by the same sampling math should carry the remaining load.
Three signs tell you the terms have reached their ceiling:
- Precision stalls. Successive revisions no longer move the share of responsive documents in the sample.
- Narrowing costs you known documents. Every tightening that cuts noise also drops documents you already know are responsive, which means recall is falling.
- The evidence speaks in code. Responsive material uses nicknames, chat shorthand or project names that no one could have guessed in a drafting session.
Once those signs appear, I stop polishing words. Further rounds become a kind of Talmudic disputation over the word "settle*", learned and exhausting and, for the client's budget, nothing short of ruinous. The better move is to let keywords do the coarse culling and let concept search or AI classification rank what remains. The hybrid search approach to cutting attorney review hours follows the same logic.
The validation does not change. The 2024 r/ediscovery discussion that framed precision and recall as universal metrics also carried a striking expectation: one commenter predicted that aiR reviews would "easily" exceed 80% recall and 80% precision. That was a practitioner's forecast, not a guarantee. Treat it as a hypothesis your own sample must confirm. The point for negotiation is sharper. The same random-sample check you used on terms now audits the classifier.
Which brings up the follow-on question litigators keep asking: is AI issue coding or technology-assisted review easier to defend if opposing counsel objects? To be perfectly frank, the label matters less than the record. A method survives challenge when both sides can inspect a validation sample and see how each call was made.
That is why our platform is built so AI assists, and never replaces, attorney verification. Each answer links back to the exact source document, so an attorney can check a coding decision before it ever appears in a validation report. The objection then has nowhere vague to land.
| Approach | Where it is strong | How you validate it |
|---|---|---|
| Negotiated keywords alone | Transparent, easy to explain at a meet and confer | Sampled precision per term; recall check on a random sample of non-hits |
| Keywords plus concept search | Catches synonyms and phrasing the term list missed | Same precision and recall sampling across the combined result |
| Keywords plus AI-assisted coding | Ranks large sets and reads context, not just strings | Random-sample validation of recall and precision, with attorney-verified calls |
The negotiation, in short order, shifts from which words to accept toward which sample size and which recall target both sides will live with, and that is a conversation worth opening early.
What should you bring to your next search-term negotiation?
Bring a hit report, a coded random sample, and precision per term. Those three documents turn a quarrel over adjectives into a negotiation over evidence that a court can follow.
I expect the bargaining currency to change within the next couple of years. Sampled precision, and eventually recall, will carry more weight than raw hit counts, because the same validation math already used to audit AI review applies to keywords too. The Sedona Conference framed the subject as negotiating search protocols, and the profession is slowly catching up to its own vocabulary.
One honest trade-off deserves mention. Sampling costs attorney hours up front, and on a tiny matter a blind list may simply be cheaper. On anything larger, needless to say, those early hours buy the right to say no with numbers.
So send the first hit report before the first meet and confer, and let opposing counsel answer a sample rather than a slogan.
Summarize This Article With AI
Open this article in your preferred AI engine for an instant summary.
Frequently Asked Questions
What else do litigators ask about negotiating search terms?
How do I agree on search terms with opposing counsel?
Exchange measured rounds rather than finished lists. Run each proposed term against the collection, share the hit report, code a random sample of the hits, and revise on the precision you find. Repeat until both sides accept the numbers.
What is a hit report?
A hit report is a table showing, for each search term, how many documents it retrieves and how many of those no other term reaches. It shows volume. It does not show relevance, which is why it needs a sample beside it.
What is precision in a search-term negotiation?
Precision is the share of a term's sampled hits that an attorney codes as responsive. A term with low precision mostly buys review cost; a term with high precision mostly buys evidence.
Can I use search preview counts in my hit report?
I would not. Microsoft 365 for IT Pros notes that preview items are only a representative sample of what the full search returns. Run the complete search, then record the tool, the version and the date.
What if opposing counsel refuses to share their hit counts?
Share yours anyway, with the samples attached, and put the request for theirs in writing. A party arguing burden without numbers is in a weaker spot than one who showed its work.
What does AI-assisted review cost compared with manual review?
Relevant e-Discovery prices AI-assisted review in cents per document, a gap against manual review that a firm can see on its own matter. For a quote on a specific collection, contact the team at relevantediscovery.com/contact-us.
Written by
Michael
Kansky
Michael Kansky is a serial software entrepreneur who has spent more than two decades building and bootstrapping profitable SaaS and services companies.
Connect on LinkedIn

