Earlier v1 run: web data API benchmark

Search was real web. Scrape was 50 tasks on our own test site plus 10 PyPI pages, and map and crawl mixed test-site sections with a few real docs sites.

Different tasks and providers from v2, so the two runs can't be compared to show change over time. Same Claude Haiku 5.5 agent and limits for every provider; a script checks every answer.

Tasks done every answer correct, checked by script, with the cost of each finished task

Correct page givencontrol100%0.38¢ per task

Context.dev92%0.72¢ per task

Linkup90%0.60¢ per task

Firecrawl90%1.22¢ per task

TinyFish90%0.43¢ per task

Scrapfly88%0.61¢ per task

Jina87%0.57¢ per task

Browserbase82%1.12¢ per task

Spider20%2.48¢ per task

Model alonecontrol0%

Jina Reader's score is under a 5,000-token rejecting budget. This track mostly tests docs headings and structure, not general extraction.

Cost per finished task all API and model spend, failures included, divided by tasks done

TinyFish0.43¢

Jina0.57¢

Linkup0.60¢

Scrapfly0.61¢

Context.dev0.72¢

Browserbase1.12¢

Firecrawl1.22¢

Spider2.48¢

Time per finished task median, lower is better

Firecrawl3.3 s

Browserbase3.3 s

Linkup3.4 s

TinyFish3.5 s

Context.dev3.7 s

Jina3.8 s

Spider4.6 s

Scrapfly5.0 s

Answer in the provider's response checked by script on everything the provider returned, no model. An upper bound: it can't tell a heading from a menu link with the same words

Context.dev93%0.26¢ per task

Firecrawl93%0.80¢ per task

Jina93%0.01¢ per task

Linkup91%0.12¢ per task

TinyFish91%free

Scrapfly78%0.02¢ per task

Browserbase67%0.65¢ per task

Spider18%0.04¢ per task

Tuned settings each provider's best documented settings, frozen before the run

Context.devdefault 92%92%0.69¢ per task

Scrapflydefault 88%92%0.62¢ per task

Linkupdefault 90%90%1.04¢ per task

Jinadefault 87%87%0.59¢ per task

Firecrawldefault 90%82%2.24¢ per task

How a scrape task runs

60 tasks. 50 pages on our own test site, including JavaScript, PDFs, tabs and missing answers, plus 10 project pages on PyPI.

Spider refused our test site because its filter flags the workers.dev domain; it scraped all 10 PyPI pages correctly. Firecrawl's tuned run hit its free plan's 10 requests a minute, which is why it scores lower than its default.