Context.dev vs TinyFish for AI agents
Context.dev and TinyFish both ran scrape and search, with the same tasks, the same model (Claude Haiku 5.5), the same prompt and the same limits. A task counts only when a script confirms every answer. Beyond that, only Context.dev ran map and only Context.dev ran crawl.
The benchmark tests each provider's gap to its track's leader, with 95% intervals from resampling whole sites. It doesn't test every pair directly, so this page says when a gap is real and when it isn't one anybody can claim.
Scrape 60 tasks · Read one page and pull out exact fields: prices, versions, install commands.
Context.dev led this track, but its gap to TinyFish is too small to rule out chance.
TinyFish cost 0.43¢ per finished task and Context.dev 0.72¢, counting all API and model spend, failures included.
| Provider | Tasks done | 95% interval | Cost per finished task | Median time |
|---|---|---|---|---|
| Context.dev | 55 of 60 (92%) | 90%–100% | 0.72¢ | 3.7 s |
| TinyFish | 54 of 60 (90%) | 80%–92% | 0.43¢ | 3.5 s |
Search 60 tasks · Find a fact on the live web: a package's latest version, a filing figure, a release date.
Neither gap to the track's leader cleared chance, and the benchmark doesn't test these two against each other, so this 2-point gap isn't a result either can claim.
TinyFish cost 1.31¢ per finished task and Context.dev 1.82¢, counting all API and model spend, failures included.
| Provider | Tasks done | 95% interval | Cost per finished task | Median time |
|---|---|---|---|---|
| Context.dev | 45 of 60 (75%) | 52%–96% | 1.82¢ | 5.2 s |
| TinyFish | 44 of 60 (73%) | 47%–96% | 1.31¢ | 5.3 s |
Read these numbers with
- TinyFish here means its Fetch API for scrape and its Search API for search.
- TinyFish wasn't given a tuned run: its plan offered no affordable alternate preset.
More
Every provider on each track: scrape, search. How a gap is tested: methodology.