Agent-hostile

Steel

Steel turns a web page into clean text in one call, and its own error messages are the clearest around — a bad key tells you exactly which key and where to get a new one. Repeat reads are safe. The problem: when the page it fetched is really a 404 or 500, Steel still reports success — the true status is buried where an agent won't look, so it passes an error page along as real content. Also, the flagship browser-session feature won't start on a fresh account until you add funds.

Cross-validated onClaude Opus 4.8

The five tasks

Getting started from the docs aloneNailed it

Fixing its own mistake after an errorHalf-nailed it

Here’s the receipt — what actually happened, not our summary of it.

API-misuse errors are best-in-class: bad key -> 401 naming the 'steel-api-key' header and how to get a valid one, plus a docs link; missing url -> 400 with an example URL; a scheme-less url is tolerantly resolved (200). But target HTTP status is not faithfully exposed: httpbin 404 and 500 both return metadata.statusCode 200; the real status only appears in urlSource ('chrome-error://') and the page text ('HTTP ERROR 404'). An agent trusting statusCode ingests an error page as success.

Following a multi-step flowNailed it

Handling an unclear edge caseNailed it

Not double-charging on a retryNailed it

See everything the AI did (12 steps)

t1  Claude Opus 4.8  POST /v1/scrape (cold-start)  → 200, 25,448 chars, 87 links
t2  Claude Opus 4.8  POST /v1/scrape (bad key)  → 401 with header name + how-to-fix + docs link
t3  Claude Opus 4.8  POST /v1/scrape (missing url)  → 400 must include scheme, with example
t4  Claude Opus 4.8  POST /v1/scrape (url without scheme)  → 200 tolerantly resolved
t5  Claude Opus 4.8  POST /v1/scrape (httpbin 404/500)  → 200, metadata.statusCode 200 (real status in urlSource/content)
t6  Claude Opus 4.8  POST /v1/scrape (links/10/0)  → 200, links extracted, pick L1
t7  Claude Opus 4.8  POST /v1/scrape (L1 from A)  → 200, links extracted, pick L2
t8  Claude Opus 4.8  POST /v1/scrape (L2 from L1)  → 200, html 313 chars
t9  Claude Opus 4.8  POST /v1/scrape (no format)  → 200 default html
t10  Claude Opus 4.8  POST /v1/scrape (format 'banana')  → 400 not a supported value
t11  Claude Opus 4.8  POST /v1/scrape (same url) x2  → 200 x2, byte-identical markdown
t12  Claude Opus 4.8  POST /v1/sessions  → 402 out of credits (well-messaged; sessions meter separately)

Changelog

2026-07-05Reclassified to Agent-hostile — silent-failure rule applied consistently across the board.

Tested 2026-07-02 with Claude Opus 4.8 agents · request a re-test