OpenWeatherMap
A weather read is read-only, so nothing is created or duplicated: the same city twice returns the same reading. Errors are specific and separated: leaving out the city returns 'Nothing to geocode', a mistyped city returns a 404 'city not found', and an out-of-range latitude is refused with 'wrong latitude' - it validates rather than guessing, and distinguishes a missing input from a bad one from a not-found one. The chain works too: geocode a city to coordinates, then pull the current weather and a forecast at those coordinates. An agent always gets an actionable answer.
Cross-validated onClaude Opus 4.8
The five tasks
Getting started from the docs aloneNailed it
Fixing its own mistake after an errorNailed it
Here’s the receipt — what actually happened, not our summary of it.
Missing q -> 400 {cod:'400', message:'Nothing to geocode'}; a mistyped city -> 404 {cod:'404', message:'city not found'}; a valid city -> 200. Distinguishes a missing parameter (400) from a not-found city (404) with specific messages.Following a multi-step flowNailed it
Handling an unclear edge caseNailed it
Not double-charging on a retryNailed it
See everything the AI did (9 steps)
t1 Claude Opus 4.8 GET /data/2.5/weather q=London → 200 temp t2 Claude Opus 4.8 GET /data/2.5/weather (missing q) → 400 Nothing to geocode t3 Claude Opus 4.8 GET /data/2.5/weather q=nonsense → 404 city not found t4 Claude Opus 4.8 GET /data/2.5/weather q=Paris (corrected) → 200 t5 Claude Opus 4.8 GET /data/2.5/weather lat=999 lon=999 → 400 wrong latitude t6 Claude Opus 4.8 GET /geo/1.0/direct q=Berlin → 200 coords t7 Claude Opus 4.8 GET /data/2.5/weather lat,lon (coords) → 200 temp t8 Claude Opus 4.8 GET /data/2.5/forecast lat,lon cnt=3 → 200 3 entries t9 Claude Opus 4.8 GET /data/2.5/weather q=Tokyo x2 → 200/200 identical
Tested 2026-07-03 with Claude Opus 4.8 agents · request a re-test