bug(v3-ref-seller / storyboard): 7 get_products_brief steps fail PERMISSION_DENIED — fixture seed or allowlist policy regression?

[bokelley]

Surfaced by #693 (which made the storyboard CI gate actually assert on results).

Failures

Seven media_buy_seller/*/get_products_brief steps fail with identical error text:

PERMISSION_DENIED: Buyer agent is not authorized for this seller.
The seller's commercial allowlist did not authorize this credential.
Resolve out-of-band via the seller's onboarding contact;
this is not a request-side error the buyer can correct.

Affected scenarios:

• media_buy_seller/refine_products/get_products_brief
• media_buy_seller/measurement_terms_rejected/get_products_brief
• media_buy_seller/pending_creatives_to_start/get_products_brief
• media_buy_seller/inventory_list_targeting/get_products_brief
• media_buy_seller/inventory_list_no_match/get_products_brief
• media_buy_seller/invalid_transitions/get_products_brief
• media_buy_seller/creative_fate_after_cancellation/get_products_brief

The shared text + shared step name across 7 unrelated scenarios strongly suggests one of:

Hypotheses

1. Storyboard fixture seed missing. The runner is presenting a buyer-agent credential that the v3 ref seller's PgBuyerAgentRegistry doesn't carry. Either a seed/migration step was removed, the storyboard's expected buyer-agent isn't being inserted at CI bootstrap, or the test controller skipped a setup hook.

2. Allowlist policy regression. The v3 ref seller is correctly rejecting an unauthorized credential the storyboard intended to be authorized. This would point at a recent change to commercial-allowlist enforcement in the seller (PR fix(decisioning): Tier 2 codes → spec-conformant PERMISSION_DENIED (#375) #393 / Tier 2 work).

3. Test-controller config drift. The storyboard's commercial-identity Tier 2 gate is expecting a credential shape (e.g. X-AdCP-Authorization header, OAuth client id, signed request) that doesn't match what the seller's auth middleware extracts.

Run

ci run 25732467897 / job 75560889145

Full storyboard report artifact: v3-storyboard-result-1.zip.

How to triage

1. Pull the v3-storyboard-result-1.zip artifact from the linked run.
2. Inspect the request headers + extracted credential the seller saw on a failing get_products_brief call.
3. Cross-reference against the buyer-agent rows the v3 ref seller has in its PgBuyerAgentRegistry after CI bootstrap (`docker compose exec ... psql`).
4. If the credential isn't in the registry — fixture seed gap.
5. If it's in the registry but status != 'active' — allowlist regression.
6. If the credential isn't even being extracted from the request — auth middleware drift.

Why this blocks #693

#693 is correctly diagnosing that the storyboard gate was cosmetic. These 7 failures are exactly the kind of thing the gate is meant to surface. Until they're resolved, the storyboard CI job stays red and #693 either:

• can't merge cleanly, OR
• merges and turns main red on every PR.

Companion issues for the other storyboard failure clusters from the same run:

• bug(v3-ref-seller): storyboard terminal-error codes mismatch on three update / create scenarios #701 — terminal-error code mismatches (3 scenarios)
• bug(storyboard): media_buy_seller/refine_products/get_products_refine sends buying_mode='refine' with brief #702 — get_products_refine step combines buying_mode='refine' with brief
• feat(v3-ref-seller): implement sync_accounts + list_accounts with billing_entity write-only projection guard wired #377 — __spec_conformance__/account_discovery (already open)

Acceptance

• All 7 get_products_brief steps pass the storyboard
• Root cause documented in the fix PR (fixture seed / allowlist policy / middleware drift)
• If it's a fixture issue, the seed step is wired such that it runs deterministically before every storyboard CI invocation

[bokelley]

Manual triage.

Verdict: ready-to-investigate. Shared error text + step name across 7 unrelated scenarios = one root cause.

Most likely hypothesis (rank-ordered):

1. Fixture seed gap (high) — storyboard buyer-agent credential not inserted into PgBuyerAgentRegistry at CI bootstrap. Test this first by checking the seed step against the credential the runner uses.
2. Auth middleware drift (medium) — credential header shape mismatch from Tier 2 work (Tier 2 commercial-identity error codes don't conform to AdCP spec — cross-tenant oracle risk #375/fix(decisioning): Tier 2 codes → spec-conformant PERMISSION_DENIED (#375) #393).
3. Allowlist regression (low) — would require a recent change to commercial-allowlist enforcement; nothing obvious in recent merges.

Investigation steps:

1. Pull v3-storyboard-result-1.zip
2. Inspect request headers + extracted credential on a failing get_products_brief call
3. Cross-reference against rows in PgBuyerAgentRegistry after CI bootstrap

Coupled with #701. Same storyboard run, same gate, likely single investigation pass — separate PRs only if root causes diverge.

[bokelley]

Investigation complete — verdict (E) auth middleware drift

Pulled v3-storyboard-result.json from run 25732467897. All 7 failing steps return the identical wire payload: PERMISSION_DENIED with the registry-miss boilerplate from _resolve_buyer_agent (src/adcp/decisioning/handler.py:441-552).

#701 closed as duplicate — its 3 terminal-error mismatches are the same root cause. The handler logic that would emit the spec-correct codes never runs.

Hypotheses ruled out

• (D) seed gap. seed.py:74 seeds api_key_id="dev-bearer-token-acme-1" on tenant t_acme; CI .github/workflows/ci.yml:654 passes --auth dev-bearer-token-acme-1. Values match.
• (A/B) seller-side / translator. Handler logic never reached.
• (C) cross-repo fixture. Storyboard fixture is unambiguously well-formed.

Why all 10 fail identically

The preceding setup/sync_accounts steps skip via sole-stateful-step exemption. get_products_brief is the first authenticated tool of each scenario; the rejection wire shape is byte-identical because it originates from a single function regardless of which tool was called. get_adcp_capabilities succeeds because it explicitly bypasses auth (handler.py:1279).

Bisect candidates

PRs that post-date the last passing v3-storyboard run AND touch the bearer→ApiKeyCredential path:

• PR fix(server): BearerTokenAuthMiddleware now populates ctx.auth_info for bearer flows #579 (901aa53e) — rewrote bearer-flow adcp.auth_info population (src/adcp/server/auth.py:621-633); added pop("adcp.auth_info", ...) after _extract_auth_info in handler.py:961-965
• PR fix(server): A2A auth middleware populates current_principal contextvars (closes #590) #592 (54309427) / PR feat(auth): per-leg header config + agent-card bearerAuth scheme #595 (52f45efb) — A2A middleware + per-leg header config
• PR feat(server): default MCP streamable-http to stateful with idle eviction #636 (3173a545) — default MCP streamable-http to stateful with idle eviction; session-task boundary matches the failure mode at auth.py:563-574

Reproduction recipe

1. Boot v3 ref seller locally: cd examples/v3_reference_seller && python -m src.app
2. curl -H "Authorization: Bearer dev-bearer-token-acme-1" http://acme.localhost:3001/mcp — call get_products with a brief
3. Add temporary logging at examples/v3_reference_seller/src/app.py:117-125 (in _build_context_factory.build) to verify whether it runs on data-tool calls AND whether api_key_id + the original adcp.auth_info are both present at that point

If build runs but the credential is missing → tenant resolution failure (SubdomainTenantMiddleware/current_tenant() returning None under stateful streamable-http session task).

If build doesn't run → seller context_factory is leg-asymmetric with the framework's bearer middleware after one of the bisect candidates.

Relevant files

• examples/v3_reference_seller/src/app.py (lines 83-128 _build_context_factory, 338-339 serve() wiring)
• examples/v3_reference_seller/src/buyer_registry.py (lines 72-87 tenant scoping, 89-109 resolve_by_credential)
• src/adcp/decisioning/handler.py (lines 441-552 _resolve_buyer_agent, 961-965 auth_info pop, 1057-1063 registry call site, 1279 capabilities bypass)
• src/adcp/server/auth.py (lines 554-639 auth_context_factory)

Investigation report saved at /tmp/storyboard-investigation.md in the debugger agent's working dir.