From 91b12567b4b5e5c50546f0059125b64ae7b08fd5 Mon Sep 17 00:00:00 2001 From: Brian O'Kelley Date: Sat, 9 May 2026 09:52:48 -0400 Subject: [PATCH] fix(grader): fail agents without published 9421 webhook-signing JWKS MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes the on-ramp loophole in the webhook-emission universal that let agents self-declare themselves out of the signature phase via `webhook_auth_mode == 'hmac_legacy'`. Operationalizes the "no new HMAC implementers after date X" lever from the RFC 9421 migration plan. Two changes to webhook-emission.yaml: 1. New `signing_keys_published` precheck phase. Runner fetches the agent's brand.json, resolves the `agents[].jwks_uri`, and asserts at least one key carries `adcp_use: "webhook-signing"`. Agents that only ever signed HMAC and never published a 9421 key fail here with a specific error code (`webhook_signing_keys_unpublished` / `webhook_signing_keys_wrong_purpose` / `webhook_signing_keys_all_revoked`) before the signature phase runs. 2. `signature_validity` phase is now required. Dropped `optional: true` and `skip_if: agent.webhook_auth_mode == 'hmac_legacy'`. The runner registers the trigger as a 9421-default buyer (no `authentication` block), so the agent is graded on the signatures it emits in that mode. Buyer-side HMAC registration choices are out of scope for grading the agent. Per-buyer registration is unaffected — buyers can still register HMAC-fallback at `push_notification_config.authentication` in 3.x. This change only addresses the agent-side capability claim. Refs #3360, #4205. --- .changeset/3360-grader-hmac-only-fail.md | 9 ++ .../source/universal/webhook-emission.yaml | 140 +++++++++++++----- 2 files changed, 116 insertions(+), 33 deletions(-) create mode 100644 .changeset/3360-grader-hmac-only-fail.md diff --git a/.changeset/3360-grader-hmac-only-fail.md b/.changeset/3360-grader-hmac-only-fail.md new file mode 100644 index 0000000000..e5b4e7615c --- /dev/null +++ b/.changeset/3360-grader-hmac-only-fail.md @@ -0,0 +1,9 @@ +--- +"adcontextprotocol": minor +--- + +Grader: webhook-emission universal now fails agents that haven't published a 9421 webhook-signing JWKS at their `brand.json` `agents[].jwks_uri`. The `signature_validity` phase is required (no longer `optional` / `skip_if hmac_legacy`), and a new `signing_keys_published` precheck phase asserts the JWKS contains a key with `adcp_use: "webhook-signing"` before the signature phase runs. Closes the on-ramp loophole that previously let agents self-declare themselves out of webhook signing via `webhook_auth_mode == 'hmac_legacy'`. Operationalizes the "no new HMAC implementers after date X" enforcement from the RFC 9421 migration plan (#4205). + +New error codes on `signing_keys_published`: `webhook_signing_keys_unpublished` (no JWKS or empty), `webhook_signing_keys_wrong_purpose` (JWKS present but no key with `adcp_use: "webhook-signing"`), `webhook_signing_keys_all_revoked` (all webhook-signing keys revoked). + +Refs #3360, #4205. diff --git a/static/compliance/source/universal/webhook-emission.yaml b/static/compliance/source/universal/webhook-emission.yaml index 5381d07ca8..260b325ef5 100644 --- a/static/compliance/source/universal/webhook-emission.yaml +++ b/static/compliance/source/universal/webhook-emission.yaml @@ -34,9 +34,20 @@ narrative: | This universal grades not_applicable when the runner does not host a webhook receiver (e.g., lint-only runs against an agent that has no webhook-emitting operations, or runs where the operator has not configured the receiver - contract). It grades not_applicable — never fail — for the signature phase - when the agent advertises the deprecated HMAC fallback as its preferred mode; - HMAC-signed webhooks still grade against the idempotency phases. + contract). + + **9421 is the on-ramp.** Any agent advertising webhook-emitting operations + MUST publish a `webhook-signing` JWKS at its `brand.json` `agents[].jwks_uri` + and MUST emit valid 9421 signatures when triggered with no `authentication` + block on `push_notification_config`. Agents without published signing keys + fail the `signing_keys_published` precheck before the signature phase runs; + agents with keys but invalid signatures fail `signature_validity`. The + deprecated HMAC fallback remains a buyer-side registration option in 3.x + (`push_notification_config.authentication`), but it is not a path that + exempts the agent from publishing 9421 keys — the runner registers as a + 9421-default buyer, and the agent is graded on what it does in that mode. + This closes the on-ramp loophole that previously let agents self-declare + themselves out of the signing phase via `webhook_auth_mode == 'hmac_legacy'`. **Clean seam**: the runner does NOT reimplement signature verification or idempotency dedup. It delegates to `@adcp/client` primitives — @@ -74,11 +85,11 @@ prerequisites: operations are advertised; operations the agent does not support are skipped cleanly, not graded as failures. - Agents that sign webhooks with 9421 (the 3.0 baseline) MUST publish the - corresponding JWKS at the `jwks_uri` on their brand.json `agents[]` entry - so the runner's verifier can fetch it. Agents whose buyers have opted into - the deprecated HMAC fallback pass the idempotency phases but skip the - signature phase as not_applicable. + Agents advertising webhook-emitting operations MUST publish a JWKS at the + `jwks_uri` on their brand.json `agents[]` entry containing a key with + `adcp_use: "webhook-signing"`. The `signing_keys_published` precheck + asserts this directly; agents without published signing keys fail + `webhook_signing_keys_unpublished` before the signature phase runs. test_kit: "test-kits/webhook-receiver-runner.yaml" phases: @@ -244,32 +255,87 @@ phases: idempotency_key_format_changed if a later delivery carries a different-shaped value. + - id: signing_keys_published + title: "Agent publishes a webhook-signing JWKS at brand.json" + narrative: | + Precheck: any agent advertising webhook-emitting operations MUST publish + a JWKS at the `jwks_uri` on its `brand.json` `agents[]` entry containing + at least one key with `adcp_use: "webhook-signing"`. This is the on-ramp + gate — an agent that has only ever signed HMAC and never published a + 9421 signing key fails here, before the signature phase even runs. + + The runner resolves the agent's `brand.json` from its + `get_adcp_capabilities` advertisement (or, for agents not yet member- + registered, from operator configuration), fetches the JWKS at + `agents[].jwks_uri`, and asserts at least one key has + `adcp_use: "webhook-signing"` and a non-revoked status. Absent or + malformed JWKS produces `webhook_signing_keys_unpublished`; JWKS present + but with no webhook-signing key produces + `webhook_signing_keys_wrong_purpose`. + + This phase is observable-only and stateless — no traffic flows. It runs + before the signature phase so an operator debugging "why did signing + fail" gets a specific keys-vs-signing diagnostic instead of a generic + `signature_key_unknown` deep in the verifier checklist. + + steps: + - id: fetch_brand_json + title: "Fetch brand.json and resolve jwks_uri" + narrative: | + Runner fetches the agent's `brand.json` (resolved from + `get_adcp_capabilities` or operator configuration), iterates the + `agents[]` array, and asserts at least one entry exposes a + `jwks_uri` reachable at HTTP 200. + task: fetch_brand_jwks + stateful: false + expected: | + brand.json reachable and parseable; at least one `agents[]` entry + has a `jwks_uri` returning a valid JWKS document. Fails with + `brand_json_unreachable`, `brand_json_malformed`, or + `agents_jwks_uri_missing`. + + - id: assert_webhook_signing_key_present + title: "Assert JWKS contains a webhook-signing key" + narrative: | + Runner inspects the JWKS and asserts at least one key has + `adcp_use: "webhook-signing"`. Keys advertising other purposes + (`request-signing`, `webhook-receipts`, etc.) do not satisfy this + check — purpose-separation is enforced per the `adcp_use` rule + (one cryptoKeyVersion per signing purpose; receivers enforce + purpose at JWK `adcp_use`, not RFC 9421 tag). + task: assert_jwks_purpose + stateful: false + expected: | + At least one JWK in the agent's published JWKS carries + `adcp_use: "webhook-signing"` and is not marked revoked. Fails + with `webhook_signing_keys_unpublished` (no JWKS or empty JWKS), + `webhook_signing_keys_wrong_purpose` (JWKS present but no key + with the webhook-signing purpose), or + `webhook_signing_keys_all_revoked` (all webhook-signing keys + revoked). + - id: signature_validity - title: "Webhook signatures validate under the 9421 profile (when 9421 is in effect)" - optional: true - skip_if: "agent.webhook_auth_mode == 'hmac_legacy'" + title: "Webhook signatures validate under the 9421 profile" narrative: | - When the agent is emitting under the 9421 baseline (i.e., the buyer has - NOT opted into the deprecated HMAC fallback via - `push_notification_config.authentication`), every outbound webhook MUST - verify under the 14-step webhook verifier checklist per + Every outbound webhook MUST verify under the 14-step webhook verifier + checklist per docs/building/implementation/security.mdx#verifier-checklist-for-webhooks. - Verification is delegated to the `@adcp/client` 9421 webhook verifier — - this phase is gated on that verifier being available in the client - library (see test-kits/webhook-receiver-runner.yaml client_primitives - section). Until the client verifier lands, this phase grades as - not_applicable rather than skipping silently. + The runner registers the trigger as a 9421-default buyer (no + `authentication` block on `push_notification_config`); the agent is + graded on the signatures it emits in that mode. - Agents whose buyers registered `push_notification_config.authentication` - (the legacy HMAC mode) skip this phase as not_applicable — they are - graded on the idempotency phases only. The deprecated HMAC scheme is - graded via the legacy webhook-hmac-sha256 test vectors. + Verification is delegated to the `@adcp/client` 9421 webhook verifier + (see test-kits/webhook-receiver-runner.yaml client_primitives section). + Until the client verifier lands in a published release, this phase + grades as `not_applicable` rather than skipping silently — operators + get an explicit "verifier not yet available" signal instead of a + false-pass. Negative conformance — signatures that MUST be rejected with a specific webhook_signature_* error code — is tested via static conformance - vectors at `/compliance/{version}/test-vectors/webhook-signing/` (follow-up), - not by this phase. This phase only grades positive-path agents emitting - conformant signatures on live traffic. + vectors at `/compliance/{version}/test-vectors/webhook-signing/` + (follow-up), not by this phase. This phase only grades positive-path + agents emitting conformant signatures on live traffic. steps: - id: trigger_signed_webhook @@ -323,15 +389,23 @@ phases: grading: pass_criteria: | - Phases idempotency_key_presence and idempotency_key_stability MUST pass when - the agent advertises any webhook-emitting operation and the runner hosts a - webhook receiver. Phase signature_validity MUST pass when 9421 is in effect - (no HMAC fallback); legacy-HMAC agents skip that phase as not_applicable. + Phases idempotency_key_presence, idempotency_key_stability, + signing_keys_published, and signature_validity MUST pass when the agent + advertises any webhook-emitting operation and the runner hosts a webhook + receiver. Agents without a published webhook-signing JWKS fail + signing_keys_published — there is no exemption for "HMAC legacy mode." Agents that advertise no webhook-emitting operations grade the entire universal as not_applicable. Runners without a webhook receiver grade the - entire universal as not_applicable. + entire universal as not_applicable. signature_validity may grade + not_applicable if the runner's `@adcp/client` 9421 webhook verifier is + not yet available; this is an explicit operator signal, not a silent + skip. report_format: | Per-phase pass/fail/not_applicable, with per-step error codes on failure. Retry-stability failures include a diff of the observed keys across deliveries so operators can debug sender-side retry bugs without re-running - the full suite. + the full suite. signing_keys_published failures cite the specific gap — + `webhook_signing_keys_unpublished`, `webhook_signing_keys_wrong_purpose`, + or `webhook_signing_keys_all_revoked` — so operators distinguish "never + set up keys" from "set up keys with the wrong purpose label" without + reading the JWKS by hand.