Agent-hostile

OneSignal

Everything on the write side is clean: creating a user, a template, adding an alias and reading the user back by that alias all work first try; a template missing its name is rejected with "'name' can't be blank"; and re-creating a user with the same external_id after a dropped response returns the same user - no duplicate. The one trap: POST /notifications to a non-existent segment (or with no subscribers) returns HTTP 200 with {id:'', errors:['All included players are not subscribed']}. An agent that checks the status code sees success and moves on, never learning the message reached nobody.

Cross-validated onClaude Opus 4.8

The five tasks

Fixing its own mistake after an errorNailed it

Getting started from the docs aloneNailed it

Following a multi-step flowNailed it

Handling an unclear edge caseNailed it

Here’s the receipt — what actually happened, not our summary of it.

Docs don't spell out what happens when you send to an empty/unknown audience. Probed safely: POST /notifications {contents, included_segments:['NonexistentSegmentXYZ']} -> HTTP 200 {id:'', errors:['All included players are not subscribed']}. The unknown segment name is not flagged; the failure comes back as a 200 with an errors array and an empty id, which an agent keying on the status code reads as success.

Not double-charging on a retryNailed it

See everything the AI did (6 steps)

t1  Claude Opus 4.8  POST /apps/{app_id}/users (create, external_id)  → 200 onesignal_id
t2  Claude Opus 4.8  POST /templates (no name)  → 400 'name' can't be blank
t3  Claude Opus 4.8  POST /templates (with name)  → 200 template id
t4  Claude Opus 4.8  POST /users then PATCH identity (add alias) then GET by alias  → 200 / 200 / 200 same onesignal_id
t5  Claude Opus 4.8  POST /notifications included_segments:['NonexistentSegmentXYZ']  → 200 {id:'', errors:['All included players are not subscribed']}
t6  Claude Opus 4.8  POST /apps/{app_id}/users x2 same external_id (dropped-response retry)  → 200/200 same onesignal_id (no duplicate)

Changelog

2026-07-05Reclassified to Agent-hostile — silent-failure rule applied consistently across the board.

Tested 2026-07-03 with Claude Opus 4.8 agents · request a re-test