Skip to content

fix(grader): fail agents without published 9421 webhook-signing JWKS (#3360 part A)#4292

Merged
bokelley merged 1 commit intomainfrom
bokelley/3360-grader-hmac-fail
May 9, 2026
Merged

fix(grader): fail agents without published 9421 webhook-signing JWKS (#3360 part A)#4292
bokelley merged 1 commit intomainfrom
bokelley/3360-grader-hmac-fail

Conversation

@bokelley
Copy link
Copy Markdown
Contributor

@bokelley bokelley commented May 9, 2026

Closes #3360 (Part A — webhook signing on-ramp lever).

What this does

Closes the on-ramp loophole in the webhook-emission universal that let agents self-declare themselves out of webhook signing via webhook_auth_mode == 'hmac_legacy'. Operationalizes the "no new HMAC implementers after date X" lever from the RFC 9421 migration plan (#4205).

Two changes to static/compliance/source/universal/webhook-emission.yaml

1. New signing_keys_published precheck phase

Runner fetches the agent's brand.json, resolves agents[].jwks_uri, and asserts at least one key carries adcp_use: "webhook-signing". Agents that only ever signed HMAC and never published a 9421 key fail here, before the signature phase runs.

Three specific error codes for diagnostic clarity:

  • webhook_signing_keys_unpublished — no JWKS or empty
  • webhook_signing_keys_wrong_purpose — JWKS present but no key with the webhook-signing purpose
  • webhook_signing_keys_all_revoked — all webhook-signing keys revoked

This separates "did you set up keys" from "do you sign correctly," which previously produced signature_key_unknown deep in the verifier checklist with no operator-level diagnostic.

2. signature_validity phase is now required

Dropped optional: true and skip_if: agent.webhook_auth_mode == 'hmac_legacy'.

The runner registers the trigger as a 9421-default buyer (no authentication block on push_notification_config, already the case at line 141). The agent is graded on the signatures it emits in that mode. Buyer-side HMAC registration choices are out of scope for grading the agent.

What this does not change

  • Buyer-side HMAC fallback registration is unaffected. Buyers can still register HMAC at push_notification_config.authentication in 3.x. Schema-level removal is tracked at 4.0: drop required: ["authentication"] from reporting-webhook and artifact_webhook #4288 against the 4.0 milestone.
  • Idempotency phases are unchanged. Agents are still graded on idempotency_key_presence and idempotency_key_stability independently of signing.
  • HMAC-fallback receivers in production are unaffected. This is grader behavior, not protocol behavior.

Net behavior

Agent posture Outcome
Publishes JWKS + signs with 9421 ✅ pass
Publishes JWKS, but specific buyer registered HMAC ✅ pass (runner doesn't register HMAC)
Publishes JWKS but signs incorrectly ❌ fails signature_validity with specific webhook_signature_* code
Never published JWKS, only ever signed HMAC ❌ fails signing_keys_published with webhook_signing_keys_unpublished

Refs

Out of scope (Part B → 3.2.0)

OAuth 2.1 + OIDC metadata grading (RFC 8414 / 9728 / 7591). Will be filed as a separate issue against the 3.2.0 milestone.

Changeset

minor — adds new fail conditions to the grader. Existing agents that already publish 9421 keys are unaffected; HMAC-only agents that previously passed the universal will now fail.

Closes the on-ramp loophole in the webhook-emission universal that let
agents self-declare themselves out of the signature phase via
`webhook_auth_mode == 'hmac_legacy'`. Operationalizes the "no new HMAC
implementers after date X" lever from the RFC 9421 migration plan.

Two changes to webhook-emission.yaml:

1. New `signing_keys_published` precheck phase. Runner fetches the
   agent's brand.json, resolves the `agents[].jwks_uri`, and asserts at
   least one key carries `adcp_use: "webhook-signing"`. Agents that
   only ever signed HMAC and never published a 9421 key fail here with
   a specific error code (`webhook_signing_keys_unpublished` /
   `webhook_signing_keys_wrong_purpose` / `webhook_signing_keys_all_revoked`)
   before the signature phase runs.

2. `signature_validity` phase is now required. Dropped `optional: true`
   and `skip_if: agent.webhook_auth_mode == 'hmac_legacy'`. The runner
   registers the trigger as a 9421-default buyer (no `authentication`
   block), so the agent is graded on the signatures it emits in that
   mode. Buyer-side HMAC registration choices are out of scope for
   grading the agent.

Per-buyer registration is unaffected — buyers can still register
HMAC-fallback at `push_notification_config.authentication` in 3.x. This
change only addresses the agent-side capability claim.

Refs #3360, #4205.
@bokelley bokelley added this to the 3.1.0 milestone May 9, 2026
@bokelley bokelley merged commit c2a8855 into main May 9, 2026
19 checks passed
@bokelley bokelley deleted the bokelley/3360-grader-hmac-fail branch May 9, 2026 15:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Grade tools: add OAuth and request signing setup evaluation

1 participant