WeatherAPI.com
A weather lookup is read-only, so there is nothing to create or duplicate: the same query twice returns the same reading, and there is no write step. The strength is the chain and the errors. Searching a partial city name returns matches with coordinates, and feeding those back gets the current weather and a forecast, each step building on the last. Mistakes come back plain: an empty city says 'Parameter q is missing' and a nonsense one says 'No matching location found'. One quirk: asking for 999 forecast days returns a normal 200 with whatever it has, instead of flagging it.
Cross-validated onClaude Opus 4.8
The five tasks
Getting started from the docs aloneNailed it
Fixing its own mistake after an errorNailed it
Here’s the receipt — what actually happened, not our summary of it.
Empty q -> 400 {error:{code:1003, message:'Parameter q is missing.'}}; a nonsense city -> 400 {code:1006, message:'No matching location found.'}; a valid city -> 200. Errors carry a numeric code plus a plain message.Following a multi-step flowNailed it
Handling an unclear edge caseNailed it
Not double-charging on a retryNailed it
See everything the AI did (9 steps)
t1 Claude Opus 4.8 GET /current.json q=London (read) → 200 temp t2 Claude Opus 4.8 GET /current.json q= (empty) → 400 code 1003 Parameter q is missing t3 Claude Opus 4.8 GET /current.json q=nonsense → 400 code 1006 No matching location found t4 Claude Opus 4.8 GET /current.json q=Paris (corrected) → 200 t5 Claude Opus 4.8 GET /forecast.json q=London days=999 → 200 silently tolerates t6 Claude Opus 4.8 GET /search.json q=Springf → 200 matches + coords t7 Claude Opus 4.8 GET /current.json q=42.1,-72.59 (coords) → 200 temp t8 Claude Opus 4.8 GET /forecast.json q=42.1,-72.59 days=3 → 200 3-day forecast t9 Claude Opus 4.8 GET /current.json q=Berlin x2 → 200/200 identical
Tested 2026-07-03 with Claude Opus 4.8 agents · request a re-test