Crawl benchmark: which web crawling API gets an agent the right answer
Read a whole section of a site and return every record in it. 3 providers ran the same 10 tasks with the same model and prompt. Each task allowed 1 call, 10 minutes and $2.00.
Scrapfly finished the most crawl tasks, but its lead over Context.dev (60%) is too small to rule out chance. Scrapfly cost the least per finished task, 2.88¢. With no tool at all, the model alone passed none.
How a crawl task runs
Tasks done every answer correct, checked by script, with the cost of each finished task
None of the gaps between providers is big enough to rule out chance on 10 tasks, since tasks from the same site are counted as one group.
Cost per finished task all API and model spend, failures included, divided by tasks done
Time per finished task median, lower is better
Only 10 tasks. Firecrawl's time includes waiting out its free-plan rate limit.
Every provider the numbers behind the charts
| Provider | Tasks done | 95% interval | Cost per finished task | Median time |
|---|---|---|---|---|
| Scrapfly | 7 of 10 (70%) | 0%–95% | 2.88¢ | 14 s |
| Context.dev | 6 of 10 (60%) | 0%–82% | 7.58¢ | 18 s |
| Firecrawl | 5 of 10 (50%) | 0%–68% | 14.2¢ | 60 s |
Only 10 tasks, from 4 site clusters. Crawl asks for every record in a section, so it tests extraction as well as retrieval.
Scrapfly had the shortest median time per finished task, 14 s. Not tested: Exa, Tavily, Bright Data and Perplexity.
Head to head pairs that both ran crawl
Other tracks
How every task is graded, and when a gap counts, is on the methodology page.