Benchmark · Directional
Headless engine over CDP
Webfold's engine ran about 8.8 times the concurrent tasks a second of headless Chrome. It used about 24 times less memory. That is one scripted task on one shared VM. Read the ratios. Do not read the milliseconds.
Webfold's engine against headless Chrome on one scripted task. Moli ran too. So did Lightpanda. Scrapling is the rival on a page that needs JavaScript.
What question does this benchmark ask?
How fast is Webfold's CDP engine on a task that runs page scripts? How light? How concurrent? Compared with the engines people drive over CDP.
How was it run?
- Task: open a target, navigate to a page that builds 200 rows from a script and an API call, run six feature probes, read the result, close. Each task asserts its content, so an engine that fails fast shows up as a failure. All four engines finished 363 of 363 tasks.
- One shared 4-core VM and one fixture server. Three repeats, 40 warm sequential tasks and 8 workers by 10 tasks for concurrency. Chrome is chromium-1194 headless; Moli 1.1.19; Lightpanda 1.1.0-nightly.
- Scrapling 0.5 lane: 15 runs per tool on a page that builds 200 rows and fetches /api/data. Scrapling drives Chromium; Webfold uses its JavaScript interpreter over the same CDP session.
What are the results?
| Measurement (run 1) | Webfold | Chrome | Moli | Lightpanda |
|---|---|---|---|---|
| Cold start, ms | 8 | 240 | 7 | 23 |
| First task, ms | 73 | 201 | 149 | 115 |
| Warm median, ms | 70.7 | 164.7 | 137.9 | 113.9 |
| Warm p95, ms | 74.1 | 179.0 | 146.3 | 119.9 |
| Concurrent tasks per second | 106.6 | 12.1 | 32.1 | 65.1 |
| Resident memory, MB | 30 | 724 | 88 | 31 |
| CPU seconds for the run | 0.8 | 12.1 | 6.7 | 1.4 |
Against headless Chrome: about 8.8 times the concurrent tasks per second (8.4 times in the 3 November rerun), 2.3 times faster per warm task, 24 times less memory and 15 times less CPU.
On a page that needs JavaScript, Webfold answered in 3.3 ms where Scrapling's Chromium took 907.6 ms (about 275 times; 238 times at p95). That gap is the browser, not Scrapling's code.
Webfold is not first everywhere. Moli starts 1 ms sooner (7 against 8 ms), Lightpanda uses 1 MB more or less the same memory, and on a plain fetch Scrapling's curl_cffi is faster (9.3 ms against 12.0 ms median).
What does this not show?
Know the limits
- One fixture page on one shared VM. The source's own words: ratios are meaningful, digits are not.
- The rerun on 3 November moved Webfold's numbers by 15 to 100 percent (warm median 76.6 ms, 75.4 tasks per second). Read the ratios and the pass counts, not the milliseconds.
- This says nothing about real sites, rendering fidelity or JavaScript-heavy applications. The interpreter runs the scripts the fixture uses; Chromium runs far more.
- The Scrapling fetch lane is a different job: Webfold was slower there, by 1.3 times (12.0 ms against 9.3 ms median, 14.1 against 10.3 at p95). The parse lane compares a warm Scrapling process with a Webfold process per document.
- Run 2 has no memory or CPU figures, and run 1 survives only as a table in the README.
Which other benchmarks are there?
- HTML to Markdown conversion Measured
- Document conversion Measured
- Main-content extraction quality Measured
- Web archive extraction Measured
- Footprint of a scrape or crawl Directional
- PII detection against Presidio Measured
- Localisation extraction Directional
- PDF text extraction with OCR Directional
- PDF extraction against other extractors Directional
- LaTeX conversion with BibTeX Directional
- A whole site to Markdown Directional
- Agent-browser compatibility (Lexbench) Measured
- Search-results parsing Directional
- Folder search Directional
Check it on your own pages.
Run the command-line tool on your content.