Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions architectureDocs/deployment/derot-key-issuance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Key Issuance from DeRoT (Onchain KMS)

How a CVM obtains its application keys from the Decentralized Root of Trust on Base.

DeRoT uses an **Onchain KMS** where a DstackKms contract on Base holds the KMS root CA, and a
per-application DstackApp contract holds the compose-hash whitelist. The KMS will only issue keys
to CVMs whose attested code is whitelisted by the app owner.

## Flow

```mermaid
sequenceDiagram
autonumber
participant CVM as CVM<br/>(TDX hardware)
participant KMS as Onchain KMS<br/>(dstack-kms)
participant DstackApp as DstackApp contract<br/>(Base)
participant DstackKms as DstackKms contract<br/>(Base)
participant Intel as Intel DCAP<br/>(quote verification)

Note over CVM: Boot — OS + containers start.<br/>dstack measures OS, compose, images into RTMRs.

CVM->>KMS: Key request + TDX attestation quote<br/>(contains RTMR3 with compose-hash)
KMS->>Intel: Verify TDX quote signature
Intel-->>KMS: Quote valid / TCB status
KMS->>DstackKms: Is OS image hash whitelisted?
DstackKms-->>KMS: Yes (MRTD/RTMR0-2 check passes)
KMS->>DstackApp: Is compose-hash whitelisted?
DstackApp-->>KMS: Yes (owner approved this version)
KMS-->>CVM: App keys (derived from KMS root secret)

Note over CVM: CVM now holds its secrets.<br/>TLS cert generated inside TEE.<br/>Application starts serving requests.
```

## Components

| Component | Role |
|-----------|------|
| **TDX hardware** | Hardware root of trust; measures everything into RTMR registers at boot |
| **dstack OS** | Hashes OS + kernel + boot params into MRTD/RTMR0-2; hashes docker-compose (with pinned image digests) into RTMR3 |
| **Onchain KMS** | dstack-kms service; verifies attestation quote; checks on-chain whitelists before issuing keys |
| **DstackKms (Base)** | Contract holding whitelisted OS image hashes and KMS aggregated MR |
| **DstackApp (Base)** | Per-application contract; owner controls which compose-hashes are authorized; `0x2f83172A49584C017F2B256F0FB2Dca14126Ba9C` |

## Key Points

- **No whitelist → no keys.** If the compose-hash is not in DstackApp, the KMS refuses key issuance, and the CVM cannot start serving.
- **Image digests must be pinned.** `docker-compose` must use `@sha256:` digests; mutable tags like `:latest` produce a non-deterministic compose-hash that won't match any whitelist entry.
- **Key derivation is deterministic.** The same attested code always gets the same derived keys, so data encrypted by one CVM instance is readable by a later instance running the same code.
- **Owner controls governance.** The DstackApp `owner` (wallet, multisig, timelock, or DAO) is the sole entity that can whitelist new compose-hashes.

## References

- [Phala: Cloud vs Onchain KMS](https://docs.phala.com/phala-cloud/key-management/cloud-vs-onchain-kms)
- [Phala: DeRoT design](https://docs.phala.com/dstack/design-documents/decentralized-root-of-trust)
- [requirements.md FR-2, FR-2.6](planning/requirements.md)
- [PLAN.md — Onchain KMS on Base](planning/PLAN.md)
156 changes: 156 additions & 0 deletions architectureDocs/deployment/trust-stack-verification.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
# Platform Verification with dstack-verifier

How to verify that a running CVM is executing on genuine Intel TDX hardware with a trusted OS —
before trusting it with API requests.

[dstack-verifier](https://github.com/Dstack-TEE/dstack/tree/main/verifier) is the tool Phala
provides for this. It automates cryptographic validation of the TDX quote, OS image, and event log
integrity. Governance checks (compose-hash and OS-image whitelists on Base) are separate and
performed with `cast` against the DstackApp / DstackKms contracts.

Reference: [Phala Cloud — Verify the Platform](https://docs.phala.com/phala-cloud/attestation/verify-the-platform)

## What dstack-verifier Checks Automatically

| Check | What it does |
|-------|-------------|
| **TDX quote signature** | Verifies the quote against Intel root certs via `dcap-qvl` |
| **TCB SVN** | Confirms the hardware is running the latest security patches; debug mode is off |
| **OS image hash** | Downloads the OS image from `download.dstack.org` and compares against the hash in the quote (MRTD/RTMR0-2) |
| **RTMR3 event log replay** | Replays the event log and asserts the result matches the quoted RTMR3 |
| **KMS identity** | Extracts the KMS ID from the `key-provider` event in RTMR3 |

When all pass, `is_valid: true`. A real TDX deployment also produces `os_image_hash_verified: true`
(the simulator uses a synthetic OS image hash not published on `download.dstack.org`, so that field
is always `false` in local testing — see CI notes below).

## What dstack-verifier Does NOT Check

These require separate governance verification (see below):

- Whether the **compose-hash is whitelisted** in the DstackApp contract
- Whether the **OS image hash is whitelisted** in the DstackKms contract
- Whether the **KMS aggregated MR** is whitelisted
- TLS certificate binding / CAA DNS records

## Verification Workflow

```mermaid
sequenceDiagram
autonumber
participant Op as Operator
participant CVM as CVM<br/>(/attestation)
participant DV as dstack-verifier<br/>(localhost:8080)
participant Intel as Intel DCAP
participant Phala as download.dstack.org
participant Base as Base chain<br/>(DstackApp / DstackKms)

Note over Op,CVM: Step 1 — Collect attestation data

Op->>CVM: GET /attestation
CVM-->>Op: { quote, event_log, vm_config }

Note over Op,DV: Step 2 — Platform verification (automated by dstack-verifier)

Op->>DV: POST /verify { quote, event_log, vm_config, attestation: null }
DV->>Intel: Verify TDX quote signature (dcap-qvl)
Intel-->>DV: Quote valid; TCB SVN
DV->>Phala: Download OS image (os_image_hash from vm_config)
Phala-->>DV: OS image bytes
DV->>DV: Compute MRTD/RTMR0-2 from OS image;<br/>replay event log → assert RTMR3 matches;<br/>extract KMS ID from key-provider event
DV-->>Op: { is_valid, quote_verified, os_image_hash_verified,<br/>details: { kms_id, ... } }

Note over Op,Base: Step 3 — Governance verification (manual)

Op->>Base: DstackApp.allowedComposeHashes(composeHash)?
Base-->>Op: true / false
Op->>Base: DstackKms.allowedOsImages(osImageHash)?
Base-->>Op: true / false
```

## Running dstack-verifier

**Start as an HTTP server:**

```bash
# Docker
docker run -p 8080:8080 dstacktee/dstack-verifier:latest

# Or build from source (used in CI)
cargo build --manifest-path=path/to/dstack/verifier/Cargo.toml --bin dstack-verifier
./dstack-verifier -c dstack-verifier.toml
```

**Collect attestation data and verify:**

```bash
# 1. Get attestation data from the CVM
curl https://api.litprotocol.com/attestation -o attestation.json

# 2. Strip 0x prefix from quote, set attestation: null (required by dstack-verifier)
python3 -c '
import sys, json
d = json.load(open("attestation.json"))
q = d["quote"]
q = q[2:] if q.startswith("0x") else q
print(json.dumps({"quote": q, "event_log": d["event_log"], "vm_config": d["vm_config"], "attestation": None}))
' > verify-request.json

# 3. POST to dstack-verifier
curl -s -X POST localhost:8080/verify \
-H 'Content-Type: application/json' \
-d @verify-request.json | jq
```

**Expected output (real TDX hardware):**

```json
{
"is_valid": true,
"quote_verified": true,
"os_image_hash_verified": true,
"details": {
"kms_id": "...",
"kms_name": "phala-mainnet"
}
}
```

## Governance Verification

After `is_valid: true`, verify the application and OS image are authorized on-chain:

```bash
# Is the current compose-hash whitelisted in DstackApp?
cast call <DSTACK_APP_ADDRESS> "allowedComposeHashes(bytes32)" <COMPOSE_HASH> --rpc-url <BASE_RPC>

# Is the OS image hash whitelisted in DstackKms?
cast call <DSTACK_KMS_ADDRESS> "allowedOsImages(bytes32)" <OS_IMAGE_HASH> --rpc-url <BASE_RPC>
```

DstackApp address for this deployment: `0x2f83172A49584C017F2B256F0FB2Dca14126Ba9C`

The compose-hash can be extracted from the RTMR3 event log (the `app-compose` event) or computed
locally: `sha384(docker-compose.yml contents)` with all image references pinned to `@sha256:`
digests.

## CI Usage Notes

The phala-simulator CI job (`phala-simulator.yml`) runs dstack-verifier in **oneshot mode**
(`--verify FILE`) against the dstack simulator. Because the simulator's `attestation.bin` contains
a synthetic OS image hash that has never been published on `download.dstack.org`, the result is:

- `quote_verified: true` — the quote signature pipeline is correct ✓
- `os_image_hash_verified: false` — expected; the hash is synthetic, not a real published image
- `is_valid: false` — expected; a real deployment against actual Phala Cloud returns `true`

The CI assertion is `quote_verified=true`, which is the meaningful signal: it confirms that
`get_quote()` → dstack-verifier is wired end-to-end correctly.

## References

- [Phala Cloud: Verify the Platform](https://docs.phala.com/phala-cloud/attestation/verify-the-platform)
- [Phala Cloud: Chain of Trust](https://docs.phala.com/phala-cloud/attestation/chain-of-trust)
- [dstack-verifier source](https://github.com/Dstack-TEE/dstack/tree/main/verifier)
- [derot-key-issuance.md](derot-key-issuance.md)
- [phala-simulator.yml](../../.github/workflows/phala-simulator.yml)
86 changes: 86 additions & 0 deletions architectureDocs/deployment/vm-code-upgrade.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# Upgrading VM Code

How to safely roll out a new version of the CVM application — from code change through
on-chain governance approval to live traffic.

Because the Onchain KMS only issues keys to whitelisted code, every upgrade requires two
independent actions: **a governance tx** (whitelist the new compose-hash) and **a deployment** (push
the new image and restart the CVM). Neither alone is sufficient.

## Flow

```mermaid
flowchart TB
subgraph Dev["Developer"]
A["Edit code"] --> B["Build & push Docker image\n(tagged with SHA256 digest)"]
B --> C["Compute new compose-hash\nfrom docker-compose + image digests"]
end

subgraph Governance["On-chain Governance (Base)"]
D["Owner submits tx:\nDstackApp.addAllowedComposeHash(newHash)"]
E{Multisig / timelock\napproval?}
D --> E
E -->|approved| F["New compose-hash\nwhitelisted on-chain"]
E -->|rejected| X["Upgrade blocked"]
end

subgraph CI["CI/CD (GitHub Actions / just deploy)"]
G["Substitute @sha256: digest\nin docker-compose"] --> H["phala deploy\n(or DeRoT deploy)"]
H --> I["New CVM starts with new image"]
end

subgraph Boot["New CVM Boot"]
I --> J["dstack measures compose into RTMR3"]
J --> K["CVM requests keys from Onchain KMS"]
K --> L{compose-hash\nwhitelisted?}
L -->|yes| M["Keys issued\nCVM starts serving"]
L -->|no| N["Key request denied\nCVM cannot start"]
end

C --> D
C --> G
F -.->|"whitelist in place\nbefore deploy"| L

style X fill:#fee2e2,stroke:#dc2626
style N fill:#fee2e2,stroke:#dc2626
style M fill:#dcfce7,stroke:#16a34a
```

## Steps

1. **Build & push** — CI builds the new Docker image and pushes it to the registry. The registry
returns a `@sha256:` digest for the image.

2. **Compute compose-hash** — dstack hashes the full `docker-compose.yml` (with pinned image
digests) to produce the compose-hash that will appear in RTMR3 at boot.

3. **Governance approval** — The DstackApp owner (wallet, multisig, or timelock) submits an
on-chain transaction to whitelist the new compose-hash. A timelock or multisig introduces a
mandatory delay, giving stakeholders time to review before the new code can receive keys.

4. **Deploy** — After the whitelist tx is confirmed, CI redeploys (`phala deploy` or DeRoT
equivalent) with the new image digest substituted into docker-compose.

5. **Boot & key issuance** — The new CVM boots, dstack measures the compose into RTMR3, and the
CVM requests keys from the Onchain KMS. The KMS verifies the attestation and checks the
DstackApp whitelist (step 3). If the compose-hash matches, keys are issued and the CVM starts.

## Ordering Requirement

The governance tx **must be confirmed on-chain before the new CVM boots** and requests keys.
If the CVM boots before the whitelist tx is finalized, the KMS will reject the key request and
the CVM cannot serve traffic. The diagram shows the whitelist flowing into the key issuance check
to reflect this dependency.

## Rollback

To roll back, redeploy the previous image (whose compose-hash is already whitelisted). No
governance action is required if the old compose-hash was not removed from the whitelist.
To block a rolled-back version, the owner removes its compose-hash from DstackApp.

## References

- [derot-key-issuance.md](derot-key-issuance.md) — how keys are issued after deploy
- [deployment.md](deployment.md) — deploy workflow (`just deploy`, GitHub Actions)
- [requirements.md DR-1, FR-2.6](planning/requirements.md)
- [Phala: Cloud vs Onchain KMS](https://docs.phala.com/phala-cloud/key-management/cloud-vs-onchain-kms)