How OCR Errors Silently Drop Documents From Your Search

Degraded scans like this one set the ceiling on what OCR, and therefore keyword search, can find.

Even the best OCR engines miss a meaningful share of difficult pages. In 2025, Mistral's internal benchmarks claimed about 94.9% accuracy on difficult PDFs, against roughly 83.4% for Google Document AI and 89.5% for Azure, and open-source Tesseract then scored 80 to 95% on typed text.

Consider what that means for a scanned production. You can open the page and read every word. The search index cannot. In a sponsored 2026 piece hosted by the American Bar Association, the practice-management vendor 8am put the stakes for AI tools plainly: image-based PDFs are "effectively invisible to AI assistants."

The discipline that measures this is older than most review platforms. A 2021 survey of OCR evaluation tools and metrics, hosted by PRImA Research, catalogs how researchers score OCR output against ground truth, meaning a verified human transcription of the same page, and it points to the OCR-D ground truth transcription guidelines as a shared standard for building those transcriptions. The method exists. Discovery practice has mostly ignored it.

So before the first search runs, I would ask one question of whoever processed the collection: was any of its OCR ever scored against a human transcription of these pages?

Why does OCR accuracy decide what your search can find?

OCR accuracy decides discovery outcomes because AI review now runs cents per document against roughly $19K/GB for manual review, and both methods read only the text OCR supplies.

Before you trust a keyword report on a scanned collection, ask three questions in order:

1. Which documents carry a text layer produced by OCR rather than extracted from the native file?
2. What accuracy did that OCR reach on these pages, not on someone else's test set?
3. Which of your search terms would fail if one character inside them were misread?

The arithmetic behind those questions is unforgiving. LlamaIndex's guide to legal document OCR accuracy shows that at 95% accuracy, OCR produces 50 errors per 1,000 words and 500 errors in a standard 10,000-word contract, a level it labels high risk, with material terms likely affected. At 97% accuracy the same contract still carries 300 errors. At 99% it carries 100. A 100,000-word case file read at 95% holds 5,000 errors, and each one sits somewhere a reviewer may never look.

Search does not round up. The same 8am piece notes that discovery productions frequently contain image-based PDFs and Bates-numbered documents, so for many produced pages the OCR layer is the only text there is.

The common assumption is that 97% is good enough for discovery, because the number sounds like a passing grade. I believe that framing hides the real exposure. The grade is scored per character, while your search terms, your privilege screens and your AI prompts all operate on whole words and phrases, which means a small character loss compounds into a much larger loss of findable meaning. As LlamaIndex puts it, "Errors at the extraction layer carry through every process built on top of it."

That is the reason the processing layer, not the review layer, is where defensibility is won or lost. At Relevant e-Discovery our platform keeps immutable originals with content hashing and append-only audit trails, so the original image stays available to test any text layer against. We pair that defensible spine of Bates, privilege and chain of custody with cited question answering in one tool, which matters here because a cited answer is only as complete as the text beneath it.

Combining 4 sources points to one pattern: cheap review multiplies the value of good text and the cost of bad text in equal measure. The same blind spot explains why AI semantic search still misses the documents that matter, since embeddings built from garbled characters inherit the garbling. Faster review over a damaged layer only reaches the wrong answer sooner.

Which leaves the harder question of what an accuracy percentage actually counts, and what it quietly leaves out of the count.

What does a character error rate actually measure?

Character error rate counts misread letters, not lost meaning, so only checking the answer against the exact exhibit image shows which responsive words search never saw.

Character error rate, or CER, is the share of characters OCR got wrong compared with a correct transcription of the same page. Splunk's explainer, citing the OCR-D evaluation specification, gives the formula as CER = (S + D + I) / N: substitutions plus deletions plus insertions, divided by the total characters in the reference text. In plain terms, a CER of 5% means one out of every 20 characters needs correction.

The same explainer offers a small before and after that I find more instructive than any table.

• Before: the reference text reads "machine learning," 16 characters including the space.
• After: OCR returns "machin lerning," two characters deleted.
• Result: a CER of 12.5%, and neither word would match an exact keyword search.

Two deletions. Two dead search terms.

Here is where the metric starts to work against you. Splunk notes that standard CER charges every substitution the same cost of 1, so confusing O with Q, l with 1, or m with n weighs no more than any other slip, even though those are the exact confusions that break account numbers, names and defined terms. It also warns that normalization choices, such as stripping punctuation or casing, can lower a reported rate enough to make cross-study comparisons invalid. And it says plainly that CER "is blind to meaning."

The implication for discovery is uncomfortable. Before you accept a vendor's CER, ask which pages it was measured on and whether anyone ran your actual search terms against those same pages. The research section below shows what one swapped character does to a single invoice number.

To be clear, I am not arguing against measuring CER. I am arguing that a reported CER belongs in your file beside a written list of the search terms someone actually tested on those pages. Keyword strategy has to account for this, which is part of the case for pairing exact terms with concept retrieval, as we describe in how to reduce attorney review hours with hybrid search in e-discovery.

The practical answer is verification at the level of the source. In our platform every output links back to the exact exhibit it came from, so an attorney can one-click verify before filing and see the scanned page beside the text that was searched. With deployment inside your own AWS account under your own keys, that page-by-page check happens without the scans leaving your control. It is also why we ask buyers to bring a messy collection and a hard question to a Relevant e-Discovery demo: watching the tool read, code and cite your own evidence shows where your scans hold up and where they do not, which a case study about someone else's documents cannot do.

The next problem is that the published accuracy figures most teams rely on were never measured on documents that look like yours.

Why do published OCR accuracy figures overstate your results?

Published accuracy overstates litigation results because benchmarks skip messy legal scans; trust only figures measured on your own collection, processed single-tenant or inside your own AWS account.

A benchmark is a fixed test set used to score OCR engines against one another. The trouble is what goes into the set. A 2026 survey by Fitsum Sileshi Beyene and Christopher L. Dancy of The Pennsylvania State University reviewed OCR papers and benchmark datasets published between 2006 and 2025 and found that leading benchmarks such as OCRBench, OmniDocBench and DocVQA rely heavily on scientific papers, corporate forms and current digital PDFs. Historical and community-produced documents, they found, "rarely appear in reported training data or evaluation benchmarks."

Their sharper finding concerns layout. The authors show that a model can report a satisfactory CER while silently eliminating multi-column structure, through failures they name as column collapse, typographic errors and hallucinated text. As they write, "The design of evaluation techniques does more than just measure machine learning systems; it shapes what such systems are built to see."

I keep returning to that sentence because a litigation collection is its own kind of archive. Faxed contracts, microfilmed records, handwritten intake notes, stamped exhibits with footnotes running into the margin: none of it resembles a clean corporate form. What the benchmark never measured, the engine was never built to read.

One OCR vendor's published accuracy guide makes the same point in its own numbers, and its claims map closely onto the scans that fill discovery sets.

Scan condition	Vendor-claimed effect on OCR	What it means for search
Resolution below 150 DPI	Character accuracy drops below 90%	Terms on low-resolution pages are the first to vanish
Resolution below 100 DPI	Accuracy described as unusable	The page is effectively image-only
Page skewed 5 degrees	Traditional OCR accuracy degrades by 3 to 8%	A crooked scan can turn a clean page into a lossy one
Clean original faxed and re-scanned	99% character accuracy may fall to 88%	Faxed copies of key documents deserve separate testing
Cursive or messy handwriting	60 to 85% character accuracy	Margin notes and intake forms rarely surface in keyword hits
Complex tables, traditional OCR	60 to 80% row-level accuracy	Amounts, dates and account numbers misalign or drop

The pattern matters more than any single row. Each condition is common in litigation. That is where a 2024 law-firm primer from LexWorkplace, a document-management vendor, overreached when it said OCR ensured digital records were "precise replicas of their physical counterparts."

So the defensible conclusion is narrow and firm: measure on your own documents. That is the gap Relevant e-Discovery was built for, a platform operable and affordable for a solo or small firm on a single messy matter, which is the segment enterprise platforms price out and cannot serve without an administrator. Our processing runs single-tenant or inside your own AWS account under your own keys, with no vendor retention and no model training, so re-running OCR on privileged scans does not move them anywhere new. It is the same reasoning behind asking where AI relevance scores come from, and what they hide: a score built elsewhere tells you little about your own pages.

Measurement begins, though, with a question most teams skip entirely: which text layer did the search actually run against?

What should you do before the next search term runs?

Treat every OCR text layer as unverified until tested: isolate image-only files, demand native extracted text where it exists, and sample OCR pages against a human transcription.

The problem is not new. FindLaw's EDRM history of document review notes that imaged collections "continue to be a major component of litigation discovery," and that electronic data "does not always easily co-exist" with scanned paper. What has changed is the price of doing better. In 2025, model-based OCR was priced around $1 per 1,000 pages, which makes re-running only the weak pages a small line item.

I would start with three moves:

1. Isolate non-searchable, image-only PDFs instead of running OCR over everything.
2. For non-redacted documents in a production, request an overlay of the original extracted text, since practitioners describe OCR as prone to unnecessary errors while extracted text is the true original layer.
3. Test a sample of OCR pages against a verified transcription, and check your actual search terms, not only the error rate.

A single misread letter in a few hundred per contract is enough. Untested, that hole sits inside your production, undisclosed, until someone else finds the page.