diff --git a/architectureDocs/deployment/derot-key-issuance.md b/architectureDocs/deployment/derot-key-issuance.md new file mode 100644 index 00000000..1f44a0f1 --- /dev/null +++ b/architectureDocs/deployment/derot-key-issuance.md @@ -0,0 +1,56 @@ +# Key Issuance from DeRoT (Onchain KMS) + +How a CVM obtains its application keys from the Decentralized Root of Trust on Base. + +DeRoT uses an **Onchain KMS** where a DstackKms contract on Base holds the KMS root CA, and a +per-application DstackApp contract holds the compose-hash whitelist. The KMS will only issue keys +to CVMs whose attested code is whitelisted by the app owner. + +## Flow + +```mermaid +sequenceDiagram + autonumber + participant CVM as CVM
(TDX hardware) + participant KMS as Onchain KMS
(dstack-kms) + participant DstackApp as DstackApp contract
(Base) + participant DstackKms as DstackKms contract
(Base) + participant Intel as Intel DCAP
(quote verification) + + Note over CVM: Boot — OS + containers start.
dstack measures OS, compose, images into RTMRs. + + CVM->>KMS: Key request + TDX attestation quote
(contains RTMR3 with compose-hash) + KMS->>Intel: Verify TDX quote signature + Intel-->>KMS: Quote valid / TCB status + KMS->>DstackKms: Is OS image hash whitelisted? + DstackKms-->>KMS: Yes (MRTD/RTMR0-2 check passes) + KMS->>DstackApp: Is compose-hash whitelisted? + DstackApp-->>KMS: Yes (owner approved this version) + KMS-->>CVM: App keys (derived from KMS root secret) + + Note over CVM: CVM now holds its secrets.
TLS cert generated inside TEE.
Application starts serving requests. +``` + +## Components + +| Component | Role | +|-----------|------| +| **TDX hardware** | Hardware root of trust; measures everything into RTMR registers at boot | +| **dstack OS** | Hashes OS + kernel + boot params into MRTD/RTMR0-2; hashes docker-compose (with pinned image digests) into RTMR3 | +| **Onchain KMS** | dstack-kms service; verifies attestation quote; checks on-chain whitelists before issuing keys | +| **DstackKms (Base)** | Contract holding whitelisted OS image hashes and KMS aggregated MR | +| **DstackApp (Base)** | Per-application contract; owner controls which compose-hashes are authorized; `0x2f83172A49584C017F2B256F0FB2Dca14126Ba9C` | + +## Key Points + +- **No whitelist → no keys.** If the compose-hash is not in DstackApp, the KMS refuses key issuance, and the CVM cannot start serving. +- **Image digests must be pinned.** `docker-compose` must use `@sha256:` digests; mutable tags like `:latest` produce a non-deterministic compose-hash that won't match any whitelist entry. +- **Key derivation is deterministic.** The same attested code always gets the same derived keys, so data encrypted by one CVM instance is readable by a later instance running the same code. +- **Owner controls governance.** The DstackApp `owner` (wallet, multisig, timelock, or DAO) is the sole entity that can whitelist new compose-hashes. + +## References + +- [Phala: Cloud vs Onchain KMS](https://docs.phala.com/phala-cloud/key-management/cloud-vs-onchain-kms) +- [Phala: DeRoT design](https://docs.phala.com/dstack/design-documents/decentralized-root-of-trust) +- [requirements.md FR-2, FR-2.6](planning/requirements.md) +- [PLAN.md — Onchain KMS on Base](planning/PLAN.md) diff --git a/architectureDocs/deployment/trust-stack-verification.md b/architectureDocs/deployment/trust-stack-verification.md new file mode 100644 index 00000000..78bc0b47 --- /dev/null +++ b/architectureDocs/deployment/trust-stack-verification.md @@ -0,0 +1,156 @@ +# Platform Verification with dstack-verifier + +How to verify that a running CVM is executing on genuine Intel TDX hardware with a trusted OS — +before trusting it with API requests. + +[dstack-verifier](https://github.com/Dstack-TEE/dstack/tree/main/verifier) is the tool Phala +provides for this. It automates cryptographic validation of the TDX quote, OS image, and event log +integrity. Governance checks (compose-hash and OS-image whitelists on Base) are separate and +performed with `cast` against the DstackApp / DstackKms contracts. + +Reference: [Phala Cloud — Verify the Platform](https://docs.phala.com/phala-cloud/attestation/verify-the-platform) + +## What dstack-verifier Checks Automatically + +| Check | What it does | +|-------|-------------| +| **TDX quote signature** | Verifies the quote against Intel root certs via `dcap-qvl` | +| **TCB SVN** | Confirms the hardware is running the latest security patches; debug mode is off | +| **OS image hash** | Downloads the OS image from `download.dstack.org` and compares against the hash in the quote (MRTD/RTMR0-2) | +| **RTMR3 event log replay** | Replays the event log and asserts the result matches the quoted RTMR3 | +| **KMS identity** | Extracts the KMS ID from the `key-provider` event in RTMR3 | + +When all pass, `is_valid: true`. A real TDX deployment also produces `os_image_hash_verified: true` +(the simulator uses a synthetic OS image hash not published on `download.dstack.org`, so that field +is always `false` in local testing — see CI notes below). + +## What dstack-verifier Does NOT Check + +These require separate governance verification (see below): + +- Whether the **compose-hash is whitelisted** in the DstackApp contract +- Whether the **OS image hash is whitelisted** in the DstackKms contract +- Whether the **KMS aggregated MR** is whitelisted +- TLS certificate binding / CAA DNS records + +## Verification Workflow + +```mermaid +sequenceDiagram + autonumber + participant Op as Operator + participant CVM as CVM
(/attestation) + participant DV as dstack-verifier
(localhost:8080) + participant Intel as Intel DCAP + participant Phala as download.dstack.org + participant Base as Base chain
(DstackApp / DstackKms) + + Note over Op,CVM: Step 1 — Collect attestation data + + Op->>CVM: GET /attestation + CVM-->>Op: { quote, event_log, vm_config } + + Note over Op,DV: Step 2 — Platform verification (automated by dstack-verifier) + + Op->>DV: POST /verify { quote, event_log, vm_config, attestation: null } + DV->>Intel: Verify TDX quote signature (dcap-qvl) + Intel-->>DV: Quote valid; TCB SVN + DV->>Phala: Download OS image (os_image_hash from vm_config) + Phala-->>DV: OS image bytes + DV->>DV: Compute MRTD/RTMR0-2 from OS image;
replay event log → assert RTMR3 matches;
extract KMS ID from key-provider event + DV-->>Op: { is_valid, quote_verified, os_image_hash_verified,
details: { kms_id, ... } } + + Note over Op,Base: Step 3 — Governance verification (manual) + + Op->>Base: DstackApp.allowedComposeHashes(composeHash)? + Base-->>Op: true / false + Op->>Base: DstackKms.allowedOsImages(osImageHash)? + Base-->>Op: true / false +``` + +## Running dstack-verifier + +**Start as an HTTP server:** + +```bash +# Docker +docker run -p 8080:8080 dstacktee/dstack-verifier:latest + +# Or build from source (used in CI) +cargo build --manifest-path=path/to/dstack/verifier/Cargo.toml --bin dstack-verifier +./dstack-verifier -c dstack-verifier.toml +``` + +**Collect attestation data and verify:** + +```bash +# 1. Get attestation data from the CVM +curl https://api.litprotocol.com/attestation -o attestation.json + +# 2. Strip 0x prefix from quote, set attestation: null (required by dstack-verifier) +python3 -c ' +import sys, json +d = json.load(open("attestation.json")) +q = d["quote"] +q = q[2:] if q.startswith("0x") else q +print(json.dumps({"quote": q, "event_log": d["event_log"], "vm_config": d["vm_config"], "attestation": None})) +' > verify-request.json + +# 3. POST to dstack-verifier +curl -s -X POST localhost:8080/verify \ + -H 'Content-Type: application/json' \ + -d @verify-request.json | jq +``` + +**Expected output (real TDX hardware):** + +```json +{ + "is_valid": true, + "quote_verified": true, + "os_image_hash_verified": true, + "details": { + "kms_id": "...", + "kms_name": "phala-mainnet" + } +} +``` + +## Governance Verification + +After `is_valid: true`, verify the application and OS image are authorized on-chain: + +```bash +# Is the current compose-hash whitelisted in DstackApp? +cast call "allowedComposeHashes(bytes32)" --rpc-url + +# Is the OS image hash whitelisted in DstackKms? +cast call "allowedOsImages(bytes32)" --rpc-url +``` + +DstackApp address for this deployment: `0x2f83172A49584C017F2B256F0FB2Dca14126Ba9C` + +The compose-hash can be extracted from the RTMR3 event log (the `app-compose` event) or computed +locally: `sha384(docker-compose.yml contents)` with all image references pinned to `@sha256:` +digests. + +## CI Usage Notes + +The phala-simulator CI job (`phala-simulator.yml`) runs dstack-verifier in **oneshot mode** +(`--verify FILE`) against the dstack simulator. Because the simulator's `attestation.bin` contains +a synthetic OS image hash that has never been published on `download.dstack.org`, the result is: + +- `quote_verified: true` — the quote signature pipeline is correct ✓ +- `os_image_hash_verified: false` — expected; the hash is synthetic, not a real published image +- `is_valid: false` — expected; a real deployment against actual Phala Cloud returns `true` + +The CI assertion is `quote_verified=true`, which is the meaningful signal: it confirms that +`get_quote()` → dstack-verifier is wired end-to-end correctly. + +## References + +- [Phala Cloud: Verify the Platform](https://docs.phala.com/phala-cloud/attestation/verify-the-platform) +- [Phala Cloud: Chain of Trust](https://docs.phala.com/phala-cloud/attestation/chain-of-trust) +- [dstack-verifier source](https://github.com/Dstack-TEE/dstack/tree/main/verifier) +- [derot-key-issuance.md](derot-key-issuance.md) +- [phala-simulator.yml](../../.github/workflows/phala-simulator.yml) diff --git a/architectureDocs/deployment/vm-code-upgrade.md b/architectureDocs/deployment/vm-code-upgrade.md new file mode 100644 index 00000000..ad7f5b83 --- /dev/null +++ b/architectureDocs/deployment/vm-code-upgrade.md @@ -0,0 +1,86 @@ +# Upgrading VM Code + +How to safely roll out a new version of the CVM application — from code change through +on-chain governance approval to live traffic. + +Because the Onchain KMS only issues keys to whitelisted code, every upgrade requires two +independent actions: **a governance tx** (whitelist the new compose-hash) and **a deployment** (push +the new image and restart the CVM). Neither alone is sufficient. + +## Flow + +```mermaid +flowchart TB + subgraph Dev["Developer"] + A["Edit code"] --> B["Build & push Docker image\n(tagged with SHA256 digest)"] + B --> C["Compute new compose-hash\nfrom docker-compose + image digests"] + end + + subgraph Governance["On-chain Governance (Base)"] + D["Owner submits tx:\nDstackApp.addAllowedComposeHash(newHash)"] + E{Multisig / timelock\napproval?} + D --> E + E -->|approved| F["New compose-hash\nwhitelisted on-chain"] + E -->|rejected| X["Upgrade blocked"] + end + + subgraph CI["CI/CD (GitHub Actions / just deploy)"] + G["Substitute @sha256: digest\nin docker-compose"] --> H["phala deploy\n(or DeRoT deploy)"] + H --> I["New CVM starts with new image"] + end + + subgraph Boot["New CVM Boot"] + I --> J["dstack measures compose into RTMR3"] + J --> K["CVM requests keys from Onchain KMS"] + K --> L{compose-hash\nwhitelisted?} + L -->|yes| M["Keys issued\nCVM starts serving"] + L -->|no| N["Key request denied\nCVM cannot start"] + end + + C --> D + C --> G + F -.->|"whitelist in place\nbefore deploy"| L + + style X fill:#fee2e2,stroke:#dc2626 + style N fill:#fee2e2,stroke:#dc2626 + style M fill:#dcfce7,stroke:#16a34a +``` + +## Steps + +1. **Build & push** — CI builds the new Docker image and pushes it to the registry. The registry + returns a `@sha256:` digest for the image. + +2. **Compute compose-hash** — dstack hashes the full `docker-compose.yml` (with pinned image + digests) to produce the compose-hash that will appear in RTMR3 at boot. + +3. **Governance approval** — The DstackApp owner (wallet, multisig, or timelock) submits an + on-chain transaction to whitelist the new compose-hash. A timelock or multisig introduces a + mandatory delay, giving stakeholders time to review before the new code can receive keys. + +4. **Deploy** — After the whitelist tx is confirmed, CI redeploys (`phala deploy` or DeRoT + equivalent) with the new image digest substituted into docker-compose. + +5. **Boot & key issuance** — The new CVM boots, dstack measures the compose into RTMR3, and the + CVM requests keys from the Onchain KMS. The KMS verifies the attestation and checks the + DstackApp whitelist (step 3). If the compose-hash matches, keys are issued and the CVM starts. + +## Ordering Requirement + +The governance tx **must be confirmed on-chain before the new CVM boots** and requests keys. +If the CVM boots before the whitelist tx is finalized, the KMS will reject the key request and +the CVM cannot serve traffic. The diagram shows the whitelist flowing into the key issuance check +to reflect this dependency. + +## Rollback + +To roll back, redeploy the previous image (whose compose-hash is already whitelisted). No +governance action is required if the old compose-hash was not removed from the whitelist. +To block a rolled-back version, the owner removes its compose-hash from DstackApp. + +## References + +- [derot-key-issuance.md](derot-key-issuance.md) — how keys are issued after deploy +- [deployment.md](deployment.md) — deploy workflow (`just deploy`, GitHub Actions) +- [requirements.md DR-1, FR-2.6](planning/requirements.md) +- [Phala: Cloud vs Onchain KMS](https://docs.phala.com/phala-cloud/key-management/cloud-vs-onchain-kms)