Benchmark · Directional
PDF text extraction with OCR
On eight small PDFs, Webfold's character error rate is 0.023 on a fuzzy scan. pdftotext scores 0.628. That row uses an interim OCR page rule that was never committed. On two columns every tool scores 0.141. Eight fixtures say nothing about real-world PDFs.
Webfold's PDF route against pdftotext, MarkItDown or Tesseract. Eight PDFs. Scored by character error rate.
What question does this benchmark ask?
Does the PDF route get the text right on scans or rotated pages? What about columns? How long does it take?
How was it run?
- Eight PDFs from the superdocuments test fixtures, 1.3 to 71.6 KB: born-digital, rotated, scanned, fuzzy-scanned, mixed, two-column, borderless-table and ASCII85-encoded.
- Score: character error rate (alphanumeric, lower-cased Levenshtein) against each file's expected text, with HTML tags removed first. Median of 6 calls after one warm-up, on a shared 4-core machine at a load average of 12 to 30.
- Compared: Webfold's route (native parser, then OCR for the pages that need it), Poppler 24.02.0 pdftotext, MarkItDown 0.1.9, and Tesseract 5.3.4 run on page 1 at 300 dpi.
- The Webfold rows are a superdocuments build with the PDF parser changes and an interim OCR page rule, measured through the example programs the command line calls. No commit hash is recorded for that build.
What are the results?
| Webfold | pdftotext | MarkItDown | Tesseract, page 1 | |
|---|---|---|---|---|
| Invoice | 0.000 (9 ms) | 0.000 (20 ms) | 0.000 (1,506 ms) | 0.000 (1,774 ms) |
| Rotated 15 degrees | 0.000 (12 ms) | 0.686 (19 ms) | 0.264 (1,372 ms) | not scored (1,882 ms) |
| Scanned text | 0.000 (640 ms) | no text (20 ms) | no text (1,498 ms) | 0.000 (1,759 ms) |
| Scanned, fuzzy | 0.023 (776 ms) | 0.628 (23 ms) | 0.628 (1,448 ms) | 0.023 (1,915 ms) |
| Mixed native and scanned | 0.119 (1,005 ms) | 0.212 (23 ms) | 0.212 (1,511 ms) | 0.212 (1,855 ms) |
| Two columns | 0.141 (8 ms) | 0.141 (23 ms) | 0.141 (1,487 ms) | 0.141 (3,091 ms) |
| Borderless table | 0.000 (12 ms) | 0.000 (26 ms) | 0.000 (1,386 ms) | 0.000 (1,867 ms) |
| ASCII85 streams | 0.000 (9 ms) | 0.000 (19 ms) | 0.000 (1,418 ms) | 0.000 (1,937 ms) |
Where the text is not simply sitting in the file, Webfold gets it right and the text extractors do not: a fuzzy scan at 0.023 against 0.628, a rotated page at 0.000 against 0.686 for pdftotext, and a mixed page at 0.119 against 0.212.
On two columns every tool scores 0.141. The expected text leaves out one sentence that the PDF contains, so 0.141 is the best a correct read can score.
Pages the native parser reads take 5 to 12 ms in a process of their own, against 19 to 26 ms for pdftotext. Pages that go to OCR take 0.6 to 1.0 seconds.
What does this not show?
Know the limits
- Eight small fixtures made for testing: this says nothing about large real-world PDFs. Character error rate does not score reading order or table structure.
- The Webfold rows are not the merged route. They use an interim rule that OCRs a page with an image and fewer than 100 letters or digits, in a build that was never committed. The merged rule looks at image coverage and text density. Webfold main pins superdocuments 68c216f, the merge of that rule, so its command line has the merged rule. The numbers were not re-measured on that binary.
- The Webfold command line was not rebuilt for the final parser commit, so the figures come from the example programs it calls.
- Tesseract was run on page 1 only, so it cannot read the scanned second page of the mixed file. Its times are not like for like with Webfold's, which renders pages differently.
- Corrected on 2026-11-13: this page earlier showed the 14 October run, which is withdrawn. Its error-rate code counted extra leading text as free, and its two-column figure of 0.027 does not reproduce. Its empty native output on three born-digital files came from ASCII85 streams that the parser it used did not decode. The decode was added before the revision Webfold pins.
- Memory is not compared: the subprocess tools report parent-side memory only.
Which other benchmarks are there?
- HTML to Markdown conversion Measured
- Document conversion Measured
- Main-content extraction quality Measured
- Web archive extraction Measured
- Footprint of a scrape or crawl Directional
- PII detection against Presidio Measured
- Localisation extraction Directional
- PDF extraction against other extractors Directional
- LaTeX conversion with BibTeX Directional
- A whole site to Markdown Directional
- Agent-browser compatibility (Lexbench) Measured
- Headless engine over CDP Directional
- Search-results parsing Directional
- Folder search Directional
Check it on your own pages.
Run the command-line tool on your content.