Context.dev for AI agents: benchmark results
Context.dev finished the most scrape tasks (92%, of 8 providers) and the most map tasks (47%, of 2 providers), but neither lead is big enough to rule out chance. It had the lowest cost per finished task on map (1.84¢), counting API and model spend, failures included.
Against Firecrawl, on the same tasks: scrape 92% vs 90% of tasks done, at 0.72¢ vs 1.22¢ per finished task; search 75% vs 85% of tasks done, at 1.82¢ vs 3.04¢ per finished task; map 47% vs 40% of tasks done, at 1.84¢ vs 3.35¢ per finished task; crawl 60% vs 50% of tasks done, at 7.58¢ vs 14.2¢ per finished task. The scrape, search and map gaps in tasks done are too small to rule out chance. On crawl, neither led, so the benchmark didn't test the two against each other. Firecrawl vs Context.dev, track by track.
Context.dev (content API) was tested on scrape, search, map and crawl. It got the same tasks, the same model (Claude Haiku 5.5), the same prompt and the same limits as every other provider, and a script checked every answer.
Scrape 60 tasks · Read one page and pull out exact fields: prices, versions, install commands.
- Tasks done
- 92%
- 55 of 60
- 95% interval
- 90%–100%
- Cost per finished task
- 0.72¢
- Median time
- 3.7 s
1st of 8 on tasks done. Context.dev finished the most scrape tasks, but its lead over Linkup, Firecrawl and TinyFish (90%) is too small to rule out chance.
Raw output, no model: the API's own response held every answer on 51 of 55 applicable tasks (93%), at 0.26¢ of API spend per finished task.
Tuned settings, frozen before the run: 55 of 60 (92%), against 92% with defaults.
| Provider | Tasks done | 95% interval | Cost per finished task | Median time |
|---|---|---|---|---|
| Context.dev | 55 of 60 (92%) | 90%–100% | 0.72¢ | 3.7 s |
| Linkup | 54 of 60 (90%) | 90%–90% | 0.60¢ | 3.4 s |
| Firecrawl | 54 of 60 (90%) | 90%–90% | 1.22¢ | 3.3 s |
| TinyFish | 54 of 60 (90%) | 80%–92% | 0.43¢ | 3.5 s |
| Scrapfly | 53 of 60 (88%) | 80%–90% | 0.61¢ | 5.0 s |
| Jina | 52 of 60 (87%) | 60%–92% | 0.57¢ | 3.8 s |
| Browserbase | 49 of 60 (82%) | 78%–100% | 1.12¢ | 3.3 s |
| Spider | 12 of 60 (20%) | 4%–100% | 2.48¢ | 4.6 s |
Search 60 tasks · Find a fact on the live web: a package's latest version, a filing figure, a release date.
- Tasks done
- 75%
- 45 of 60
- 95% interval
- 52%–96%
- Cost per finished task
- 1.82¢
- Median time
- 5.2 s
3rd of 5 on tasks done. Context.dev is behind Firecrawl (85%), but the gap of 10 points is too small to rule out chance.
Raw output, no model: the API's own response held every answer on 29 of 53 applicable tasks (55%), at 0.46¢ of API spend per finished task.
| Provider | Tasks done | 95% interval | Cost per finished task | Median time |
|---|---|---|---|---|
| Firecrawl | 51 of 60 (85%) | 69%–100% | 3.04¢ | 11 s |
| Linkup | 50 of 60 (83%) | 65%–97% | 2.55¢ | 5.3 s |
| Context.dev | 45 of 60 (75%) | 52%–96% | 1.82¢ | 5.2 s |
| TinyFish | 44 of 60 (73%) | 47%–96% | 1.31¢ | 5.3 s |
| Browserbase | 23 of 60 (38%) | 21%–59% | 5.97¢ | 12 s |
Map 15 tasks · List every URL in a section of a site nobody has indexed yet.
- Tasks done
- 47%
- 7 of 15
- 95% interval
- 11%–100%
- Cost per finished task
- 1.84¢
- Median time
- 6.1 s
1st of 2 on tasks done. Context.dev finished the most map tasks, but its lead over Firecrawl (40%) is too small to rule out chance.
Raw output, no model: the API's own response held every answer on 3 of 15 applicable tasks (20%), at 1.10¢ of API spend per finished task.
| Provider | Tasks done | 95% interval | Cost per finished task | Median time |
|---|---|---|---|---|
| Context.dev | 7 of 15 (47%) | 11%–100% | 1.84¢ | 6.1 s |
| Firecrawl | 6 of 15 (40%) | 11%–100% | 3.35¢ | 11 s |
Crawl 10 tasks · Read a whole section of a site and return every record in it.
- Tasks done
- 60%
- 6 of 10
- 95% interval
- 0%–82%
- Cost per finished task
- 7.58¢
- Median time
- 18 s
2nd of 3 on tasks done. Context.dev is behind Scrapfly (70%), but the gap of 10 points is too small to rule out chance.
| Provider | Tasks done | 95% interval | Cost per finished task | Median time |
|---|---|---|---|---|
| Scrapfly | 7 of 10 (70%) | 0%–95% | 2.88¢ | 14 s |
| Context.dev | 6 of 10 (60%) | 0%–82% | 7.58¢ | 18 s |
| Firecrawl | 5 of 10 (50%) | 0%–68% | 14.2¢ | 60 s |
Context.dev compared
- Firecrawl vs Context.dev scrape, search, map and crawl, compared with Firecrawl
- Context.dev vs Linkup scrape and search, compared with Linkup
- Context.dev vs TinyFish scrape and search, compared with TinyFish
- Context.dev vs Scrapfly scrape and crawl, compared with Scrapfly
- Context.dev vs Jina scrape, compared with Jina