feat: declarative policy engine, denial explanation, and dry-run mode#67
Merged
Conversation
…#42, #48, #43) Implements three related policy DX features in a single changeset: - **DeclarativePolicyEngine** (`policy_dsl.py`): loads access-control rules from YAML or TOML files; top-down first-match-wins evaluation; supports safety_class, sensitivity, roles, attributes, and min_justification match conditions. - **Denial explanation engine**: adds `explain()` to the `PolicyEngine` protocol and `DefaultPolicyEngine`. Returns `DenialExplanation` with all `FailedCondition` objects (no short-circuit), remediation hints, and a human-readable narrative. `Kernel.explain_denial()` convenience method wraps this without requiring a token. - **Dry-run invocation**: `kernel.invoke(..., dry_run=True)` verifies the token and resolves the execution plan without calling the driver. Returns `DryRunResult` with resolved driver_id, operation, response_mode, and estimated_cost tier. Also adds `policy` optional extra (`pip install weaver-kernel[policy]`), example policy files in YAML and TOML, mypy ignore_missing_imports overrides for optional deps, and 50+ new tests covering all three features. https://claude.ai/code/session_01AK9ywPwQGzUKVxi7dYBT8d
There was a problem hiding this comment.
Pull request overview
This PR adds three developer-experience features around policy evaluation and invocation planning: a declarative policy engine (YAML/TOML), structured denial explanations, and a dry-run mode for Kernel.invoke().
Changes:
- Introduces
DeclarativePolicyEngine(policy_dsl.py) with rule parsing/evaluation and anexplain()API. - Adds structured denial explanation types (
DenialExplanation,FailedCondition) and implementsexplain()forDefaultPolicyEngineplusKernel.explain_denial(). - Adds
dry_run=Truesupport toKernel.invoke()returningDryRunResult, plus packaging/docs/examples updates.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 19 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_policy.py | Adds tests for DefaultPolicyEngine.explain() and DeclarativePolicyEngine parsing/evaluation/explanation. |
| tests/test_kernel.py | Adds integration tests for dry-run invocation and Kernel.explain_denial(). |
| src/agent_kernel/policy.py | Extends the PolicyEngine protocol with explain() and implements explanation logic in DefaultPolicyEngine. |
| src/agent_kernel/policy_dsl.py | New declarative policy engine with YAML/TOML loaders, evaluation, and explanation. |
| src/agent_kernel/models.py | Adds new public result models: FailedCondition, DenialExplanation, DryRunResult. |
| src/agent_kernel/kernel.py | Adds dry_run overloads/implementation and explain_denial() helper. |
| src/agent_kernel/errors.py | Adds PolicyConfigError for malformed/unreadable policy configs. |
| src/agent_kernel/drivers/mcp.py | Tweaks optional McpError import handling. |
| src/agent_kernel/init.py | Re-exports new policy engines/types and bumps __version__. |
| pyproject.toml | Adds policy optional extra deps and mypy overrides for optional imports. |
| examples/policies/default.yaml | Adds an example YAML declarative policy file. |
| examples/policies/default.toml | Adds an example TOML declarative policy file. |
| CHANGELOG.md | Documents the new engines, dry-run mode, and new public exports. |
…ackaging Apply the 19 Copilot inline review findings on PR #67, grouped: Packaging / optional deps (#1, #2, #12) - Defer `yaml` and `tomllib`/`tomli` imports into the `DeclarativePolicyEngine.from_yaml` / `from_toml` loaders so `import agent_kernel` works without the `policy` extra installed. Missing parser → `PolicyConfigError` with an install hint. Policy DSL parsing (#3, #4) - Validate types of `roles` (list[str]), `attributes` (dict[str, str]), `min_justification` (int — bool rejected), and `constraints` (mapping) in `_parse_rule()`; raise `PolicyConfigError` with precise messages instead of silently producing misbehaving rules or crashing at evaluation time. Policy DSL explain() (#5) - Correctly report explicit deny rules that fully match (previously fell through to a misleading `no_matching_rule` fallback and dropped the rule's `reason`). Skip partial-match deny rules so the explanation focuses on the actionable allow rule rather than suggesting changes that would only trigger the deny. Example policy files (#6, #7, #8, #9, #10, #11) - Rename `default_action` → `default` (the parser reads `default`, the previous key was silently ignored). - Express PII-with-tenant as an allow rule paired with default-deny; the prior `deny-pii-no-tenant` was inverted under first-match-wins. - Move `allow-secrets-service` before `deny-secrets-non-service`; the deny was previously unreachable. - Tighten `allow-read-*` / `allow-write-*` to `sensitivity: [NONE]` so PII reads route through the dedicated allow-pii rule. Kernel dry-run (#13, #14, #17) - Resolve `DryRunResult.operation` the same way drivers do (`args.get("operation", capability_id)`) so it matches what a driver would actually receive — instead of `capability.impl.operation`, which can diverge. - Mirror the Firewall's admin-only gate for `raw` mode: non-admin principals see their requested `raw` downgraded to `summary` in `DryRunResult`, matching real-invoke behaviour. Prevents probing for raw availability via dry-run. Docs / annotations (#15, #16, #18) - `Kernel.explain_denial()` docstring no longer contradicts itself ("never raises" vs. `CapabilityNotFound`). - `drivers/mcp.py` adds an explicit `_McpError: type[Exception] | None` annotation so mypy --strict is happy across the try/except branches. - `DryRunResult.budget_remaining` docstring no longer references the unimplemented `BudgetManager`; documented as reserved for a future cross-invocation budget mechanism. Protocol softening (#19) - Split `explain()` out of `PolicyEngine` into a new `ExplainingPolicyEngine` protocol so downstream engines that implement only `evaluate()` keep satisfying `PolicyEngine`. `Kernel.explain_denial()` uses `getattr` and raises a clear `AgentKernelError` when the configured engine cannot explain. Both built-in engines satisfy the richer protocol. Tests - Add tests for: explicit-deny fully-matched explanation, partial-match deny skipping, every `_parse_rule` validation error, install-hint paths for `from_yaml` / `from_toml`, dry-run operation resolution, dry-run raw-mode downgrade for non-admin, raw preserved for admin, and explain_denial against an engine without `explain()`. Docs - `docs/agent-context/invariants.md` adds a "Dry-run response-mode parity" trap entry so future contributors keep dry-run in sync with the Firewall's admin gate and the driver's operation resolution. CHANGELOG - Documents all of the above under [Unreleased]. `make ci` equivalents: ruff format/check, mypy --strict, 306 passing tests at 95% coverage, all three example scripts complete. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…omparative test Addresses the audit findings from the second review pass on PR #67: Finding A — `docs/architecture.md` now describes `PolicyEngine` / `ExplainingPolicyEngine` protocols, both built-in engines (with full DSL semantics for `DeclarativePolicyEngine` including the `[policy]` optional extra story), and `Kernel.invoke(dry_run=True)` (admin gate + operation resolution rule). Closes the canonical Components & API reference gap called out by AGENTS.md's Documentation map. Finding B — `docs/capabilities.md` adds three sections: - "Dry-run mode" — usage example, three parity rules with the real-invoke path (token verification, operation resolution, raw-mode admin gate), no-side-effects guarantee. - "Declarative policies" — loader signatures, match conditions, optional- extras behaviour, pointer to `examples/policies/default.yaml`. - "Denial explanations" — `explain_denial` usage, `ExplainingPolicyEngine` contract, what happens for custom engines that don't implement explain. Closes #43's "Affected Files" entry (`docs/capabilities.md`). Finding C — `tests/test_policy.py` adds `test_declarative_replicates_default_policy_decisions`, a comparative test that walks a curated scenario matrix (READ × NONE / PII / PCI / SECRETS, WRITE / DESTRUCTIVE with required role/justification combinations) through both `DeclarativePolicyEngine` and `DefaultPolicyEngine` and asserts the same allow/deny outcomes. Documents out-of-scope features (rate limiting, max_rows ceiling, allowed_fields enforcement) explicitly in the docstring. Closes #42's "comparative test" acceptance criterion. Finding F (nit, trivial) — widens `_McpError`'s type annotation in `drivers/mcp.py` from `type[Exception] | None` to `type[BaseException] | None`, matching the conventional wider bound for exception-type variables. Deferred to follow-up issue (Findings D, E): module-size refactor of `policy_dsl.py` (now 503 lines; AGENTS.md guideline is ≤300) and explicit dry-run tests against `HTTPDriver` / `MCPDriver`. Both are non-blocking; the dry-run short-circuit is structurally driver-agnostic and the module size predates this PR. `make ci` equivalents: ruff format/check, mypy --strict, 307 passing tests at 95% coverage. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #42, #48, #43.
Implements three related policy DX features.
DeclarativePolicyEnginelets teams load access-control rules from YAML or TOML files instead of writing Python.explain()returns a structured diagnosis of why a request would be denied.dry_run=Trueoninvoke()lets callers validate tokens and inspect the resolved execution plan without executing the driver.What changed
src/agent_kernel/policy_dsl.pyDeclarativePolicyEngineclass (~160 LOC) withPolicyMatchandPolicyRuledataclasses;from_dict/from_yaml/from_tomlconstructorssrc/agent_kernel/policy.pyexplain()toPolicyEngineProtocol; implementedDefaultPolicyEngine.explain()collecting allFailedConditionobjects without short-circuitingsrc/agent_kernel/kernel.pyinvoke()overloads fordry_run: Literal[True/False];DryRunResultreturn path; newexplain_denial()methodsrc/agent_kernel/models.pyFailedCondition,DenialExplanation,DryRunResultsrc/agent_kernel/errors.pyPolicyConfigErrorfor malformed declarative policy filessrc/agent_kernel/__init__.pypyproject.tomlpolicyoptional extra;types-PyYAMLin dev;ignore_missing_importsoverrides for optional deps (tomli,yaml,httpx,mcp.*)tests/test_policy.pyDefaultPolicyEngine.explain()andDeclarativePolicyEngine(evaluation, explain, YAML/TOML round-trips, config validation)tests/test_kernel.pyexplain_denial()examples/policies/default.yamlexamples/policies/default.tomlCHANGELOG.mdDesign decisions
explain()on the Protocol directly. No opt-in adapter or mixin — everyPolicyEngineimplementation is required to support it. Breaking change accepted in owner repo; called out in checklist.explain(). Unlikeevaluate()which raises on the first denial,explain()collects all failing conditions so callers get a complete picture in one call.dry_runvia@overload.dry_run: Literal[True] → DryRunResultanddry_run: Literal[False] → Frameare expressed as typed overloads so mypy catches misuse without runtime cost.explain_denial()is sync. No I/O is involved — policy evaluation is pure computation — so the convenience method onKernelis sync, consistent withrequest_capabilities()andget_token().DeclarativePolicyEngine. Rules are evaluated top-down; the first match determines the outcome. Predictable and easy to reason about from a policy file.attributes: {"key": "*"}means key must be present. Wildcard value matches any non-null attribute value, enabling "require tenant" rules without hardcoding values.ignore_missing_importsoverrides for optional deps.tomli,yaml,httpx, andmcp.*are optional; rather than scattering# type: ignorecomments, they are suppressed once inpyproject.toml. Pre-existingmcp.py:28unusedtype: ignorealso cleaned up.Testing
Documentation
CHANGELOG.md: Added entries under[Unreleased]for all three features.examples/policies/: Addeddefault.yamlanddefault.tomlshowing a realistic multi-rule policy covering READ/WRITE/DESTRUCTIVE safety classes and PII/SECRETS sensitivity.__init__.pyupdated to mentionDeclarativePolicyEngine.AI agent instruction files reviewed
AGENTS.md— no change needed; new types follow existing naming andslots=Truedataclass conventions; module size delta accepted as owner-repo delta (noted in checklist).docs/agent-context/invariants.md— no change needed; new features add surface area but do not alter existing security invariants..github/copilot-instructions.md— no change needed..claude/CLAUDE.md— no change needed.Checklist
make cipasses (fmt, lint, mypy strict, pytest, examples)capability,principal,grant,Framethroughoutexplain()is a breaking addition toPolicyEngineProtocol — external implementers must add the methodCHANGELOG.mdupdated