The Collection-Completeness Ratio: Proving You Got It All

Every mapped source gets a row, and every row gets a mark: collected, or a written reason why not.

Defensible collection rests on chain of custody, hash verification and intact metadata. Practitioners treat these as baseline skills, and a completeness challenge tests all three at once.

The challenge usually arrives as a letter. It says the collection is short. It names a custodian, or a phone, or a chat app. Then it asks you to prove a negative.

Most firms answer with a narrative declaration. Diligent efforts. Reasonable steps. Good faith. Opposing counsel reads it, sets it down, and asks for a deposition.

The volume makes the prose look thin. On a 2024 episode of the LawNext podcast, a former O'Melveny & Myers litigation technology lead described one matter where the team collected 800,000 text message bubbles, dumped into Excel. It took roughly two weeks just to get the spreadsheet. No adjective covers that much data.

Forensic habits have been moving into civil work for years. A 2016 practitioner thread already said tools such as EnCase and FTK were becoming more prevalent in civil cases. The standard rose. The way firms describe completeness did not.

I think the answer is a number. A logic note on Lesley's Digital Garden calls a system complete "if we can prove all things right," and a collection should meet that test one source at a time.

What is the Collection-Completeness Ratio, and how do you build its denominator?

The Collection-Completeness Ratio is sources collected divided by sources mapped. Thirty over thirty means every mapped source came in. Anything less names a gap you must explain.

Build the denominator before anyone touches a device:

1. List every custodian by name.
2. List each source type that custodian uses: mailbox, phone, collaboration app, social account.
3. Write every custodian and source pair as its own row. Count the rows.

The row count is the denominator. Nothing else is.

One 2024 practitioner thread did this math out loud. A collection quote covered 10 custodians with 3 sources each. That made 30 sources, not 10 people. The difference matters. The common assumption is that the denominator is the custodian count. It is not. A custodian is a person. A source is a container of data, and each container has its own collection method.

The containers have changed. eDiscovery Today's 2024 State of the Industry Report found that 40.2% of respondents discovered mobile devices in all or most of their cases, and 37.9% did the same for collaboration apps. In my view, that settles the default. The phone row and the chat row go on the map for every custodian. You strike them later, in writing, if they truly do not apply.

Hard sources are the ones that fall off a map. A founder interviewed on a 2024 legal tech podcast described the traditional phone collection: at least four parties in the room and four to eight hours for each device. A source that eats a day is a source somebody quietly forgets to list. Writing it down first stops that.

The money tells the same story. ComplexDiscovery estimated in 2019 that collection had grown to 12% of eDiscovery task spend, up from about 8% in 2012. Collection is no longer a rounding error. A missed source now costs real money twice: once to go back, and once to explain why you did not go the first time.

Combining 5 sources points to one rule. Map by container, not by person. Cloud tools carry their own blind spots here, and the things one-click cloud collection tools miss belong on the map as separate rows, not footnotes.

Each row needs an address. At Relevant e-Discovery, every answer the platform gives links back to the exact exhibit it came from, so an attorney can verify it in one click. A source map should work the same way. Every row points to the custodian, the container, and the item that proves it was collected.

With the denominator fixed, the numerator comes next. That is where the counting either stays honest or does not.

Why doesn't a perfect ratio prove the collection is complete?

A source counts only when its copy is hashed, logged and held under documented chain of custody. Touching a source is not collecting it.

Once the fraction is on paper, the urge is to push it to 100%. Thirty out of thirty looks clean. But the numerator only counts sources touched. It says nothing about whether each one came in whole, or came in right.

Before and after makes the point.

• Before: "Custodian 4, mailbox, collected." The row is checked. Nobody knows how, by whom, or what was left behind.
• After: "Custodian 4, mailbox, collected by examiner, hash recorded, linked attachments re-associated, custody log attached." The row now proves something.

The first row inflates the ratio. The second row defends it.

Practitioners say the same thing in plainer words. In an r/ediscovery thread on interview advice, one experienced commenter warned that collecting metadata the wrong way "has the potential to sink a case or result in penalties/sanctions." The same commenter called it a bad idea to let clients, or even their IT department, self-collect. A self-collected source still adds one to the numerator. It also hands opposing counsel a target. The risk of forensic imaging done by your IT person is the risk of a row that looks done and is not.

Depth cuts the other way too. In a separate r/ediscovery thread on imaging phones, the original poster, a practitioner with "more than 25 years" in the field, said a full forensic image had been needed only "a couple times." The text data was what mattered. Other commenters pushed back hard. One described learning at deposition that a custodian used WhatsApp or Signal, after only a targeted collection, with no way to collect the phone again.

Both camps are right about something. A targeted pull can be proportional. A targeted pull can also be the row that sinks you. The ratio cannot tell the two apart unless the row says what was taken and why.

So I add a depth note to every collected row. It records the method used, what the method leaves out, and who decided. That turns a yes into an answer.

Depth has a cost downstream. Fuller collections mean more to review. That is where Relevant e-Discovery's figures matter: AI-assisted review runs cents per document against the dollars per document and roughly $19K/GB of manual review, while every original stays immutable, content-hashed and tracked in an append-only audit trail. Collecting deeper stops being a budget fight.

None of this helps if the register sits in a drawer. It has to come out the day opposing counsel says your collection has holes.

How do I respond when opposing counsel challenges the defensibility of my ESI collection?

Answer with the ratio, the register behind each row, and a written reason for every gap. Keep that record where you control it, with no vendor retention.

The letter comes. It says your collection is incomplete. It asks for a declaration. Here is how I would answer it:

1. Lead with the number. State sources collected over sources mapped in the first line of your response.
2. Attach the register. One row per custodian and source, with method, date, hash and custody log.
3. Explain each gap in one sentence. Struck by agreement, not in use, or pending, and who made the call.

Compare the two ways to say it.

• Before: "Counsel undertook diligent, reasonable and good-faith efforts to identify and collect all relevant sources of ESI."
• After: "Sources collected over sources mapped appears on line one of Exhibit A. Each collected row carries its method, hash and custody log. The phone rows struck by agreement at the conference are marked, with the date."

The first sentence invites a deposition. The second invites a question about a few marked rows. That is a much smaller fight.

Small fights are cheaper. In 2025, ComplexDiscovery's pricing survey found that 47.2% of respondents reported expert witness testimony at $350 to $550 an hour, and 26.4% reported rates above $550, the highest share of any top tier in the survey. A completeness dispute argued in prose tends to end with experts on both sides reading each other's methods aloud. A register narrows what they can argue about. The narrower the dispute, the fewer hours anyone bills to settle it.

The register also has to be safe to hand over. It names custodians, devices and accounts. Your working copy, and the data behind it, should sit where privilege cannot leak through a vendor's back door. That concern runs deeper than collection, and the question of whether privileged ESI gets embedded before the screen belongs in the same conversation.

That is the reason Relevant e-Discovery runs processing single-tenant, or inside the firm's own AWS account under the firm's own keys, with no vendor retention and no model training. Nothing fed into it trains anyone's model or waives privilege. The same platform's demo asks a buyer to bring a messy collection and a hard question, then watch it read, code and cite that buyer's own evidence. I would use that hour to test the register itself. Ask it which mapped rows have no documents behind them.

A row with no documents is either an empty source or a missed one. You want to find out which before opposing counsel does, and before the cost of finding out climbs.

How can I reduce the cost of document review in litigation?

Map every source before you collect, then collect against the map. A counted, complete set gets reviewed once, instead of being reopened after a completeness challenge lands.

The number will matter more, not less. Volume does the damage. A 2016 practitioner thread described one matter at about 16 GB from a cell phone, 7 GB of Gmail and a 100 GB hard drive image, then said of the lawyers handed that drive: "Most cannot make head or tail of it."

A register fixes that. It says what each gigabyte is, where it came from and what it cost. In 2024, one commenter split a $15K collection quote into about $500 per source, which is the same arithmetic the ratio runs, only pointed at the invoice instead of the court.

I would start there. Pull the quote. Count the mapped rows. Divide, and put the answer on line one of the declaration before opposing counsel asks for it.