String vs Context.dev for AI agents
String and Context.dev both ran scrape, search and map, with the same tasks, the same model (Claude Haiku 5.5), the same prompt and the same limits. A task counts only when a script confirms every answer. Beyond that, only Context.dev ran crawl.
The benchmark tests each provider's gap to its track's leader, with 95% intervals from resampling whole sites. It doesn't test every pair directly, so this page says when a gap is real and when it isn't one anybody can claim.
Scrape 40 tasks · Test heading and structure preservation on documentation pages, plus package versions and JS fields.
Context.dev led this track, but its gap to String is too small to rule out chance.
String cost 1.33¢ per finished task and Context.dev 1.36¢, counting all API and model spend, failures included.
| Provider | Tasks done | 95% interval | Cost per finished task | Median time |
|---|---|---|---|---|
| Context.dev | 38 of 40 (95%) | 85%–100% | 1.36¢ | 4.6 s |
| String | 35 of 40 (88%) | 76%–97% | 1.33¢ | 3.9 s |
Search 30 tasks · Find facts, registry versions and documentation headings on the live web.
String finished clearly fewer tasks than Linkup, the leader, and Context.dev did not. The two weren't tested against each other directly, so this page doesn't call a winner between them.
String cost 1.32¢ per finished task and Context.dev 1.39¢, counting all API and model spend, failures included.
| Provider | Tasks done | 95% interval | Cost per finished task | Median time |
|---|---|---|---|---|
| Context.dev | 25 of 30 (83%) | 63%–100% | 1.39¢ | 8.4 s |
| String | 23 of 30 (77%) | 56%–96% | 1.32¢ | 6.8 s |
Map 15 tasks · Discover canonical URLs within a verified documentation scope.
String led this track, but its gap to Context.dev is too small to rule out chance.
Context.dev cost 0.98¢ per finished task and String 0.98¢, counting all API and model spend, failures included.
| Provider | Tasks done | 95% interval | Cost per finished task | Median time |
|---|---|---|---|---|
| String | 14 of 15 (93%) | 80%–100% | 0.98¢ | 8.4 s |
| Context.dev | 12 of 15 (80%) | 60%–100% | 0.98¢ | 6.4 s |
More
Every provider on each track: scrape, search, map. How a gap is tested: methodology.