loader: add a new ContainerPolicy payload to Vtl2Config region in IGVM#3572
Open
mayank-microsoft wants to merge 21 commits into
Open
loader: add a new ContainerPolicy payload to Vtl2Config region in IGVM#3572mayank-microsoft wants to merge 21 commits into
mayank-microsoft wants to merge 21 commits into
Conversation
Adds a new optional measured VTL2 paravisor config page,
`ContainerPolicy`, that carries product-specific policy enforcement
state attestable at boot. The first real product is CWCOW
(Confidential Windows Container Optimized Workload).
Design highlights
- `ContainerPolicy` is a single `mesh_protobuf::Protobuf`-derived enum
used as both the on-wire format and the manifest schema. Each variant
identifies a product on the wire via its `#[mesh(N)]` tag.
- Manifest JSON deserializes directly into the wire enum (gated behind
a `manifest` feature on `loader_defs` so the runtime crate stays
no_std). `serde::Serialize` is intentionally not derived to prevent
asymmetric round-trips with field-level `#[serde(deserialize_with)]`
adapters (e.g. CWCOW's `custom_uefi_json` path reader).
- The measured page location is stored in a new
`container_policy_location: u64` field on
`ParavisorMeasuredVtl2Config`, packed as (low 52 bits = page index,
high 12 bits = page count). `page_count == 0` signals absent so old
IGVMs (whose 4 KiB zero-padded page reads back as zero) continue to
decode cleanly.
- The page payload is length-prefixed (4-byte LE `u32`) before the
mesh body so the runtime tolerates the IGVM importer's page-aligned
zero padding.
Files
- `loader_defs/src/paravisor.rs`: `ContainerPolicy`, `CwcowPolicy`,
framing helpers, region constants, pack/unpack helpers on
`ParavisorMeasuredVtl2Config`, manifest feature.
- `loader/src/paravisor.rs`: encode + import sequence + mock-importer
tests across x64/arm64.
- `igvmfilegen_config/src/lib.rs`: `Image::Openhcl.container_policy`
field wiring directly to the wire enum.
- `igvmfilegen/src/main.rs`: identity-forward (no translation).
- `underhill_core/.../vtl2_config/{mod.rs,container_policy.rs}`:
pointer-based runtime read + bounds checks + decode.
- `flowey/...`: `X64CvmCwcow` recipe wired through pipelines, recipes,
artifact mappings, and CLI tool-out names.
- `vm/loader/manifests/openhcl-x64-cvm-cwcow-{dev,release}.json`:
exemplar CWCOW manifests.
- `Guide/src/dev_guide/contrib/container_policy.md`: dev guide for
adding new products.
Tests
54 new unit tests, all passing:
- loader_defs (25): pack/unpack edge cases incl. debug-assert overflow,
struct layout + size + field offsets, legacy back-compat, mesh
round-trip, framing tolerance, serde positive + negative.
- loader (10): mock-importer integration on both x64 and arm64,
config-before-policy sequencing, exclusive acceptance, full
reserved-size declaration, oversize rejection, non-overlap.
- igvmfilegen_config (9): absent/null/full-cwcow/debug-mode manifest
cases incl. unknown-variant + unknown-field rejection.
- underhill_core (10): known decode, trailing zeros, garbage/
truncated/oversize rejection, multi-page payload, no-panic random
inputs, future field-add forward compat.
No new `unsafe`. cargo check --workspace, cargo doc --no-deps, and
cargo xtask fmt --fix all clean.
Refactors the ContainerPolicy layout to live in-place on the same
measured config region as ParavisorMeasuredVtl2Config. The struct
carries a `container_policy_size: u32` field; the build sizes the
region's page count to fit `sizeof(struct) + policy_size` (1..=MAX
pages); the runtime reads exactly that many trailing bytes and
mesh-decodes them.
Replaces the pointer-based "separate page" design with a single
self-contained region. CWCOW's `custom_uefi_json` field also moves
from a build-time file path to a base64-encoded inline string, and
the redundant `debug_mode` flag is removed.
Region layout
0..8 magic
8 vtom_offset_bit
9..16 padding
16..20 container_policy_size: u32 (0 ⇒ absent)
20..24 reserved
24..24+N mesh_protobuf-encoded ContainerPolicy
24+N..end zero padding to next page boundary
The struct grows from 16 to 24 bytes. Pre-feature IGVMs (16-byte
struct on a zero-padded page) still decode cleanly because bytes
[16..20] are zero ⇒ size = 0 ⇒ absent. Non-CWCOW recipes import
exactly one zero-padded page — byte-for-byte identical to legacy
builds, no measurement drift.
Build / runtime API
- New `measured_vtl2_config_pages_for_policy(size)` picks the page
count from the policy size (clamped to MIN_PAGES..=MAX_PAGES).
- `PARAVISOR_MEASURED_VTL2_CONFIG_REGION_PAGE_COUNT` reserves
MAX_PAGES (= 4) of GPA space; the IGVM file only imports the
pages actually used.
- `build_measured_vtl2_config_region(config, policy_bytes)` returns
the zero-padded byte image AND the exact page count, allowing a
single `import_pages` call per region.
- The runtime reads the struct's `container_policy_size` field and,
if non-zero, reads exactly that many bytes from
`CONTAINER_POLICY_INLINE_OFFSET` and mesh-decodes them.
Removed
- The pointer-based `container_policy_location: u64` field and its
pack/unpack helpers.
- Length-prefix framing on the policy bytes (the struct field IS
the framing now).
- `PARAVISOR_MEASURED_VTL2_CONFIG_CONTAINER_POLICY_*` constants and
region-budget arithmetic for a separate policy page.
- `loader::import_container_policy_page` / `pack_container_policy_location`
loader helpers and the second `import_pages` call.
- `read_container_policy_from_mapping` runtime helper.
- CWCOW's `debug_mode` field (#[mesh(6)] is now permanently
reserved; partial relaxation is expressed by flipping individual
`require_*` flags).
- The build-side `std::fs::read` for `custom_uefi_json`; `loader_defs`
is `#![no_std]` again. The `manifest` feature now enables
`dep:base64` (alloc-only) instead of `serde/std`.
Sample manifests `openhcl-x64-cvm-cwcow-{dev,release}.json` carry a
real base64-encoded UEFI JSON inline (SecureBootEnable, SetupMode,
BootConfigurationDataHash).
Tests (45 total, all passing)
- loader_defs (19): struct = 24 bytes, field offsets including
container_policy_size, pages_for_policy_grows_with_size,
pre_feature_zeroed_page absent, mesh round-trip, decode
rejections, base64 happy/empty/invalid, serde positive +
negative.
- loader (10): build_region_absent uses MIN_PAGES,
build_region_present records exact size, large policy spans
multiple pages, mock-importer integration on x64+arm64 with a
single variable-page import_pages call, oversize rejection,
page-count-scales-with-policy regression.
- igvmfilegen_config (9): manifest serde happy + negative paths
including a partial-relaxation manifest.
- underhill_core (7): runtime reads exact bytes from struct field;
garbage / truncated / empty rejected; no-panic random table;
future-field mesh forward compat.
cargo check --workspace, cargo doc --no-deps, and cargo xtask fmt
--fix all clean.
…pter Replaces the one-way `deserialize_with` adapter on `CwcowPolicy.custom_uefi_json` with a symmetric `#[serde(with = "custom_uefi_json_serde")]` adapter. The helper module provides matching `serialize` (bytes → base64 string) and `deserialize` (base64 string → bytes) functions, so JSON round-trips are byte-identical by construction. With the symmetry guarantee in place, `ContainerPolicy` and `CwcowPolicy` now both derive `serde::Serialize` and `serde::Deserialize`. This: - Lets build tooling emit a manifest representation from a wire-enum value (useful for `igvmfilegen --dump-manifest`-style workflows). - Replaces the previous "never derive Serialize" hard rule with a cargo-check-enforced contract: any future field whose adapter is asymmetric will fail the `json_round_trip_is_byte_identical` test. The previous "Hard rule" doc block is gone. The dev guide now teaches the symmetric-adapter pattern as the canonical extension point for non-trivial field encodings. `igvmfilegen_config::Image::Openhcl.container_policy` switches from `skip_serializing` back to `skip_serializing_if = "Option::is_none"`, so absent policies are omitted (preserving the pre-feature manifest shape) while present policies survive a round trip. Tests 21 in loader_defs (up from 19): adds - `json_round_trip_is_byte_identical` — full + default round trips. - `serialize_emits_custom_uefi_json_as_base64_string` — verifies the serialize side emits a base64 string (not a JSON array of bytes). Total: 47 tests pass (21 + 10 + 9 + 7). cargo check --workspace, cargo doc --no-deps, and cargo xtask fmt --fix all clean.
The container_policy tests' `MockImporter` only exercises `import_pages`; the `ImageLoad` trait still requires `create_parameter_area*` and `import_parameter`. Replace the three `unimplemented!()` panics with `anyhow::bail!()` so any future test that accidentally reaches these paths gets a clean error result instead of unwinding. No behavioural change for existing tests (none of them hit these methods). 10 loader tests still pass.
`debug_mode` was removed before any committed branch shipped to external IGVMs, so the never-reuse-a-tag rule doesn't apply yet. Drop the "tag 6 is permanently reserved" comment and renumber `custom_uefi_json` from `#[mesh(7)]` to `#[mesh(6)]` so the CwcowPolicy field tags are contiguous (1..=6) again. All 47 tests still pass.
Code review caught the dev guide's example code block showing the pre-renumber tag. Sync it to the actual implementation.
Correct the acronym expansion across the dev guide, the wire-format doc comment in loader_defs, and the two flowey recipe doc comments. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…overflow Reverts the dynamic 1..=4-page sizing of the measured VTL2 config region to a static 1-page region (matching pre-feature builds), while still appending the optional ContainerPolicy mesh-encoded body in-place after ParavisorMeasuredVtl2Config on the same page. If the encoded body would not fit, encode_container_policy_bytes now panics with a message that tells the developer to bump PARAVISOR_MEASURED_VTL2_CONFIG_SIZE_PAGES (currently 1) and accept the attestation-measurement change. Choosing panic over a recoverable error makes the size-knob change a conscious, deliberate act, not a silent build-time decision. Runtime parsing of container_policy_size is unchanged; the defence-in-depth bound (container_policy_max_size_bytes) automatically tracks the new static size. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Doubles the static measured VTL2 config region from 1 to 2 pages so the ContainerPolicy budget grows from 4072 to 8168 bytes. The absent-policy measurement changes accordingly for every IGVM (with or without a configured policy) - this is the deliberate attestation-contract change the previous panic-on-overflow design forces us to make consciously. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…S bump A rubber-duck review of the PR comments turned up several doc-comments and Markdown paragraphs that still described the old dynamic-page-count model, or claimed that absent-policy builds were byte-for-byte identical to pre-feature (1-page) builds. After bumping PARAVISOR_MEASURED_VTL2_CONFIG_SIZE_PAGES to 2 those claims are false: the measured region is now always 2 pages and any future bump re-measures every IGVM, with or without a configured policy. Touched only prose — no code or test logic changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove the serde(default) on CwcowPolicy.custom_uefi_json so the field is mandatory in manifest JSON. Add a per-product validation pass in encode_container_policy_bytes that panics at IGVM build time if a CWCOW policy carries an empty custom_uefi_json, matching the rest of the build-time strictness contract for container policies. Update the dev guide to call out both contracts. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The test only re-exercised the already-covered well-formed round-trip path (known_cwcow_variant_round_trips / default_cwcow_round_trips already do this) and the backward-compat property it gestures at is owned by mesh_protobuf, not container_policy. Remove the redundant case. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ents Follow-up cleanup after rubber-duck review of the CWCOW container policy changes. Removes ~10 redundant tests across loader_defs, loader, igvmfilegen_config, and underhill_core whose coverage is already provided by other tests in the same module, and trims verbose doc and inline comments that we had added in this branch to be more concise. Also fixes a pre-existing multi-space indentation bug in the validate_container_policy_for_build panic message that crept in from an earlier heredoc-style edit. No behavioural change. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The functions encode/decode a ContainerPolicy protobuf body and are agnostic to paging. The _page suffix dated to when the measured config region was a single page; the region is now multi-page and the policy body may span pages within it. Rename to encode_container_policy / decode_container_policy for accuracy. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The build_measured_vtl2_config_region tests already cover the region shape, magic, container_policy_size, and policy bytes. The MockImporter-based tests were re-verifying the same invariants by stubbing ImageLoad, which added no coverage beyond what the pure-data tests provide. Drop the MockImporter, ImportCall, the integration helpers, and the three tests that depended on them. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…tion The behavior is self-evident from the manifest schema and the serde derives on ContainerPolicy; the comment added no information. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds an optional measured ContainerPolicy payload to the fixed VTL2 measured config region, wires it through IGVM generation, and introduces an end-to-end CWCOW (Confidential Windows Container on Windows) recipe + manifests with accompanying runtime parsing and contributor documentation.
Changes:
- Extend the measured VTL2 config region (now 2 pages) and append an inline mesh-encoded
ContainerPolicybody with explicit size framing. - Enable manifest-driven
ContainerPolicyconfiguration inigvmfilegen, and encode it into the measured config region during image build. - Add runtime decoding support in Underhill plus a new Flowey recipe and Guide documentation for onboarding new container products.
Reviewed changes
Copilot reviewed 16 out of 17 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| vm/loader/src/paravisor.rs | Builds a fixed-size measured config region image (struct + optional policy bytes) and passes it to import_pages; adds unit tests for encoding/region layout. |
| vm/loader/loader_defs/src/paravisor.rs | Extends ParavisorMeasuredVtl2Config, defines ContainerPolicy wire types + encode/decode helpers, and exports sizing/offset constants. |
| vm/loader/loader_defs/Cargo.toml | Adds feature-gated serde/base64 support for manifest-shaped policy types; adds serde_json for tests. |
| vm/loader/igvmfilegen/src/main.rs | Threads optional ContainerPolicy from manifest config into the OpenHCL loader calls. |
| vm/loader/igvmfilegen_config/src/lib.rs | Extends manifest schema with image.openhcl.container_policy and adds JSON round-trip tests. |
| vm/loader/igvmfilegen_config/Cargo.toml | Enables loader_defs manifest feature so ContainerPolicy can deserialize from JSON. |
| vm/loader/manifests/openhcl-x64-cvm-cwcow-release.json | New release manifest enabling CWCOW container policy for SNP/TDX guest configs. |
| vm/loader/manifests/openhcl-x64-cvm-cwcow-dev.json | New dev manifest enabling CWCOW container policy (and debug) for SNP/TDX/VBS configs. |
| openhcl/underhill_core/src/loader/vtl2_config/mod.rs | Reads container_policy_size, bounds-checks, reads bytes from the measured region, and decodes into MeasuredVtl2Info. |
| openhcl/underhill_core/src/loader/vtl2_config/container_policy.rs | New runtime helper module to decode the measured policy bytes with tests. |
| Guide/src/SUMMARY.md | Adds the new ContainerPolicy contributor page to the Guide navigation. |
| Guide/src/dev_guide/contrib/container_policy.md | New onboarding/format documentation for adding new container products/policies. |
| flowey/flowey_lib_hvlite/src/build_openhcl_igvm_from_recipe.rs | Adds the X64CvmCwcow IGVM recipe wiring to the new manifests. |
| flowey/flowey_lib_hvlite/src/artifact_openhcl_igvm_from_recipe.rs | Adds filename ↔ recipe mappings for the new CWCOW recipe. |
| flowey/flowey_lib_hvlite/src/_jobs/local_build_igvm.rs | Adds local build output naming for the new recipe. |
| flowey/flowey_hvlite/src/pipelines/build_igvm.rs | Adds CLI exposure/plumbing for X64CvmCwcow. |
| Cargo.lock | Records dependency graph changes (notably base64/serde feature usage via loader_defs). |
The previous `&Vec<u8>` signature triggers `clippy::ptr_arg`. `#[serde(with = ...)]` expands to a call passing `&self.field`, so for a `Vec<u8>` field serde produces `&Vec<u8>`, but `&Vec<u8>` auto-coerces to `&[u8]` at the call site. Taking `&[u8]` directly is strictly more flexible (works for any slice source) and eliminates the lint without needing `#[allow]`/`#[expect]`. The deserialize side is unaffected. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Update Guide/src/dev_guide/contrib/container_policy.md to match the code on this branch: Wrong: - `custom_uefi_json_serde::serialize` now takes `&[u8]`, not `&Vec<u8>` (see prior commit). - Padding goes to the end of the fixed measured-config region, not "to the page boundary". - The encode panic refers to the whole measured-config-region budget, not a "per-page" budget. Stale: - Drop reference to `igvmfilegen --dump-manifest`; describe the `json_round_trip_is_byte_identical` test instead. Missing: - Document the public helper APIs (`encode_container_policy` and `decode_container_policy`) and the loader's private `encode_container_policy_bytes` wrapper. - Add a "Current wire schema" section enumerating every product's mesh tag and every field's tag and type. - Document runtime decode behavior: size 0 -> None; nonzero size is validated against `CONTAINER_POLICY_MAX_SIZE_BYTES` before decode. Polish: - "framed bytes on measured page" -> "mesh_protobuf-encoded bytes in measured config region" in the architecture diagram. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CWCOW's encoded CwcowPolicy (including the typical ~150-byte custom_uefi_json blob) fits comfortably in 4072 bytes (one page minus the 24-byte ParavisorMeasuredVtl2Config header), so the earlier bump to 2 pages was not actually needed. Going back to 1 page reduces the measured surface area and reverts the unnecessary attestation- measurement change that the bump implied. Tests reference the constant directly so they adapt automatically. The encode_container_policy_bytes_panics_on_oversize test still exercises the new tighter budget. Update Guide/src/dev_guide/contrib/container_policy.md to match: - "(currently 2)" -> "(currently 1)" - "(currently two 4 KiB pages)" -> "(currently one 4 KiB page)" - "i.e. 8168 bytes today" -> "i.e. 4072 bytes today" - "(e.g. from 2 to 3)" -> "(e.g. from 1 to 2)" Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… guide The section largely restated facts already covered in "Region layout" (fixed `SIZE_PAGES`-page region, padding, measurement impact of bumping `SIZE_PAGES`). Drop it to reduce duplication. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Comment on lines
+462
to
+465
| Some( | ||
| container_policy::read_container_policy(&buf) | ||
| .context("container policy decode failed")?, | ||
| ) |
Contributor
Author
There was a problem hiding this comment.
These changes will be taken up in follow up PRs.
mayank-microsoft
changed the title
mayank-microsoft
changed the title
Contributor
Author
|
@chris-oo @sunilmut I have taken an approach to append the new policy after vtl2 config structure ends. Right now, we don't need more than the existing page, tomorrow we can increase the config region pages constant value and the solution should scale. We hard exit if the policy a user is trying to add goes beyond 1 page today. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an optional measured ContainerPolicy payload to the paravisor's VTL2 config region, plus its first product CWCOW (Confidential Windows Container on Windows). The policy ismesh-encoded into the measured config region at IGVM-build time and strongly-typed-decoded at boot — giving CWCOW a tamper-evident, attestable place to carry product invariants(read-only VMGS, secure-boot requirements, custom UEFI JSON, etc.) without inventing a new framing layer.
What changes
Lab validation (SNP)
Built x64-cvm-cwcow IGVM and booted it on a real SNP VM. uhdiag-dev inspect /vm/measured_vtl2_info surfaced the full decoded CwcowPolicy (vmgs_read_only: true,custom_uefi_json: 151 B, all require_* flags as configured) — confirming the manifest → mesh → measured-region → runtime-decode round trip works end-to-end. Theinspect-surfacing patch itself was not committed; only IGVM-build and runtime decode are shipped.
Adding a new product
See Guide/src/dev_guide/contrib/container_policy.md — two edits in loader_defs/src/paravisor.rs (define body struct, add #[mesh(N)] enum variant). No new dispatch trait, noproduct_id field; the mesh tag is the product identifier and the compiler enforces the strongly-typed body.