Agent-hostile

Tomorrow.io

A weather read is read-only, so nothing is created or duplicated: the same query twice returns the same reading. The chain is nice - the current-weather response resolves your city to coordinates, and you feed those into the forecast and a coordinate read. The weak spot is bad input: ask for weather at latitude 999 and instead of an error you get a 200 with an empty payload (blank time, no values), so an agent can't tell the location was garbage - it looks like a call that just found nothing. A missing field is caught, but its message leaks internal struct names instead of plain guidance.

Cross-validated onClaude Opus 4.8

The five tasks

Getting started from the docs aloneNailed it

Fixing its own mistake after an errorNailed it

Following a multi-step flowNailed it

Handling an unclear edge caseNailed it

Here’s the receipt — what actually happened, not our summary of it.

Docs don't cover out-of-range coordinates. Probed safely: GET /weather/realtime?location=999,999 -> 200 {data:{time:'', values:{}}, location:{lat:999, lon:999}} - it silently returns a success with an empty payload instead of validating the coordinates, so an agent cannot distinguish invalid input from a legitimate empty result. (Contrast: Geoapify rejects lat 999 with an explicit constraint.)

Not double-charging on a retryNailed it

See everything the AI did (8 steps)

t1  Claude Opus 4.8  GET /weather/realtime location=London  → 200 values + location
t2  Claude Opus 4.8  GET /weather/realtime (missing location)  → 400 code 400001 Location required
t3  Claude Opus 4.8  GET /weather/realtime (bad apikey)  → 401 code 401001 Invalid Auth
t4  Claude Opus 4.8  GET /weather/realtime location=Paris (corrected)  → 200
t5  Claude Opus 4.8  GET /weather/realtime location=999,999  → 200 empty payload (no validation)
t6  Claude Opus 4.8  GET /weather/forecast location=51.507,-0.128  → 200 6 daily
t7  Claude Opus 4.8  GET /weather/realtime location=51.507,-0.128 imperial  → 200 temp_F 77.54
t8  Claude Opus 4.8  GET /weather/realtime location=Berlin x2  → 200/200 identical

Changelog

2026-07-05Reclassified to Agent-hostile — silent-failure rule applied consistently across the board.

Tested 2026-07-03 with Claude Opus 4.8 agents · request a re-test