compliance(idempotency): RATE_LIMITED response MUST NOT be cached as idempotency replay

[bokelley]

Why

Spun off from #2615, which proposed a burst-volume runner to exercise the 60 req/s sustained / 300 req/s burst idempotency-cache rate-limit ceiling. Burst-volume testing has structural non-determinism (CI runner jitter, agent cold-start variance, game-able passing floor) and the cost-benefit doesn't justify the runner infrastructure absent a reported violation.

What is genuinely testable and worth landing: the replay invariant — a RATE_LIMITED response on insert MUST NOT be cached as the canonical response for that idempotency key. Otherwise a client that retries after retry_after gets the cached rate-limit error forever, defeating the point of retry_after.

Shape

Deterministic two-request phase, no burst:

1. Drive enough fresh-key inserts in a tight window to trip rate-limiting on a target mutating op. Record the idempotency key from the first request that received RATE_LIMITED.
2. Sleep retry_after seconds.
3. Re-submit the same idempotency key with the same payload.
4. Assert: response is the real handler's success/error result for that input, not a cached replay of RATE_LIMITED.

The "drive enough to trip" step is the only flaky part. Mitigation: scope it to a single agent-declared cheap mutating op (default log_event) and accept that some implementations may need to declare a lower-cost op in their test-kit manifest. If the trip step skips (rate limit not hit), grade not_applicable for that phase — same gating pattern as webhook-receiver-runner.

Scope (target ops)

The phase MUST cover any mutating op the agent declares it supports, including:

• log_event, sync_audiences, sync_catalogs, sync_creatives, sync_event_sources (existing)
• build_creative, validate_input introduced by RFC: Creative Formats v2 — canonical formats live on products, composition primitives, validate_input #3305 (Creative Formats v2) — these are new mutating ops in 3.1 preview and inherit the idempotency MUSTs

Storyboard runs the replay-invariant assertion against whichever ops the agent claims to support via get_adcp_capabilities.

What we are NOT building

• Burst-volume test-kit at static/compliance/source/test-kits/burst-runner.yaml — dropped; non-deterministic and game-able (see compliance: burst-runner test-kit contract for idempotency insert-rate ceiling #2615 close).
• Direct attestation of the 60/300 req/s thresholds — the spec text in docs/building/implementation/security.mdx covers this normatively; runtime grading is deferred until a real implementation violates the threshold.

Acceptance

• New phase in static/compliance/source/universal/idempotency.yaml exercising the replay invariant
• Phase grades not_applicable when the trip step doesn't elicit a RATE_LIMITED response (with reason documented in runner output)
• Coverage extends to RFC: Creative Formats v2 — canonical formats live on products, composition primitives, validate_input #3305's new mutating ops when they land (build_creative, validate_input)
• Reference runner implementation against a conformant and non-conformant fixture

Related

• compliance: burst-runner test-kit contract for idempotency insert-rate ceiling #2615 (closed — burst-runner scope dropped)
• security: promote idempotency-cache rate-limit guidance from SHOULD to MUST? #2559 idempotency cache rate-limit MUST (3.0)
• RFC: Creative Formats v2 — canonical formats live on products, composition primitives, validate_input #3305 Creative Formats v2 RFC — new mutating ops inherit this invariant
• docs/building/implementation/security.mdx:296 bullet 8 (idempotency rate-limit normative text)