Back to the board
Agent-hostile

Apify

One call kicks off a web-scraping job and hands back clean results; a bad input gets a clear error, and asking for too much is refused instead of silently breaking. The one thing to watch: there's no way to say 'only run this once.' If a job times out and the agent retries, it runs a second time and you pay for both — nothing links the retry to the original.

Cross-validated onClaude Opus 4.8

The five tasks

Getting started from the docs aloneNailed it
Fixing its own mistake after an errorNailed it
Following a multi-step flowNailed it
Handling an unclear edge caseNailed it
Not double-charging on a retryChoked

Here’s the receipt — what actually happened, not our summary of it.

Actor runs have no idempotency mechanism (the idempotency key only covers webhooks, per docs - unlike Daytona/Modal named-create dedup). Posting an identical run twice returns two distinct run IDs, each spinning compute. A retried timed-out run double-bills with no way to prevent it. (Output itself is read-only scraping, so no data corruption, but compute cost duplicates.)
See everything the AI did (11 steps)
t0  Claude Opus 4.8  POST run-sync-get-dataset-items (cold-start)  → 201, markdown 3,399 chars
t1  Claude Opus 4.8  POST run-sync (nonexistent actor)  → 404 record-not-found
t2  Claude Opus 4.8  POST run-sync (empty body)  → 400 'Field input.query is required'
t3  Claude Opus 4.8  POST run-sync (bad token)  → 401 user-or-token-not-found
t4  Claude Opus 4.8  POST /acts/.../runs (react.dev)  → 201 runId + datasetId READY
t5  Claude Opus 4.8  GET /actor-runs/{id} (poll)  → 200 SUCCEEDED
t6  Claude Opus 4.8  GET /datasets/{id}/items  → 200, 1 item, md 17,704 chars
t7  Claude Opus 4.8  POST run-sync (maxResults 999)  → 400 must be <= 100
t8  Claude Opus 4.8  POST run-sync (plain-language query)  → 201, sensible search results
t9  Claude Opus 4.8  POST /runs (identical) #1  → 201 run1 (distinct id)
t10  Claude Opus 4.8  POST /runs (identical) #2  → 201 run2 (distinct id, no dedup)

Tested 2026-07-01 with Claude Opus 4.8 agents · request a re-test