WorkOS
We created an organization, then retried the exact same request with the same idempotency key the docs tell you to send (a v4 UUID) to make retries safe - and got two separate organizations instead of one. The documented safeguard against duplicates simply doesn't take effect here, so a dropped-response retry silently doubles your data. Everything else is clean: creating an org, a user, and tying them together in a membership all work first try, and a malformed email is rejected with a clear 'must be a valid email address.' But the one thing meant to make retries safe is the thing that fails.
Cross-validated onClaude Opus 4.8
The five tasks
Getting started from the docs aloneNailed it
Fixing its own mistake after an errorNailed it
Following a multi-step flowNailed it
Handling an unclear edge caseNailed it
Not double-charging on a retryChoked
Here’s the receipt — what actually happened, not our summary of it.
WorkOS docs promise the Idempotency-Key header (they suggest a v4 UUID) makes retries return the same response. Sending POST /organizations twice with the same v4 UUID Idempotency-Key and identical body (header confirmed sent via request inspection) returned TWO different org ids (org_01KWKCVSF... and org_01KWKCVT1...), i.e. a duplicate. The documented idempotency does not take effect - a dropped-response retry duplicates the resource. This is a broken documented feature, not a caller error (correct header name, UUID format, and matching body were verified).
See everything the AI did (9 steps)
t1 Claude Opus 4.8 POST /organizations (create) → 201 org id t2 Claude Opus 4.8 POST /organizations (missing name) → 422 name must be a string t3 Claude Opus 4.8 POST /organizations (corrected) → 201 t4 Claude Opus 4.8 POST /user_management/users (invalid email) → 422 email must be valid t5 Claude Opus 4.8 POST /organizations (step1) → 201 org id t6 Claude Opus 4.8 POST /user_management/users → 201 user id t7 Claude Opus 4.8 POST /user_management/organization_memberships → 201 om id t8 Claude Opus 4.8 GET /user_management/organization_memberships?user_id → 200 1 membership t9 Claude Opus 4.8 POST /organizations x2 (same Idempotency-Key UUID, confirmed sent) → 201/201 two different org ids (duplicate)
Tested 2026-07-03 with Claude Opus 4.8 agents · request a re-test