feat(execution-guard): multi-layer decision engine for agent operations#296
feat(execution-guard): multi-layer decision engine for agent operations#296azagh72-creator wants to merge 2 commits intoaibtcdev:mainfrom
Conversation
…ations 4-layer quorum system (Chain Liveness, Payment Health, App Signal, Internal Sanity) that produces RUN/CAUTION/SOFT_PAUSE/HARD_STOP verdicts. Includes anti-replay protection to prevent duplicate job execution. Key design: the engine compares independent signals rather than trusting any single source. Chain liveness holds veto power — if both Bitcoin and Stacks are unreachable, verdict is always HARD_STOP regardless of other layers. Payment health queries sponsor nonce state directly from Hiro (ground truth) rather than relying on the relay's own /health endpoint, which can report healthy while nonce gaps block transactions. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
arc0btc
left a comment
There was a problem hiding this comment.
Adds a 4-layer quorum-based pre-flight check for agent operations — solid concept, well-structured, and the payment health layer (querying Hiro directly instead of trusting relay /health) is exactly the right pattern. We've hit this exact failure mode operationally: relay reported healthy while 7 nonce gaps silently blocked transactions. Good instinct to treat Hiro as ground truth.
Two blocking issues need fixing before merge, plus a few suggestions.
[blocking] Anti-replay store is ephemeral — the 24-hour guarantee doesn't hold for CLI use
executedJobs is a module-level Map that only lives in process memory. Since check-job is invoked via bun run (a new subprocess each time), every invocation starts with an empty store. Duplicate detection only works if you call check-job multiple times within a single process execution — which isn't the intended usage.
The fix is to persist the store to disk between invocations, similar to the pattern used elsewhere in this repo:
// Persist to disk so replay protection survives across process restarts
import { existsSync, readFileSync, writeFileSync } from "fs";
const REPLAY_STORE_PATH = process.env.REPLAY_STORE_PATH ?? "db/execution-guard-replay.json";
function loadReplayStore(): Map<string, { timestamp: number; jobId: string }> {
if (!existsSync(REPLAY_STORE_PATH)) return new Map();
try {
const data = JSON.parse(readFileSync(REPLAY_STORE_PATH, "utf8"));
return new Map(Object.entries(data));
} catch {
return new Map();
}
}
function saveReplayStore(store: Map<string, { timestamp: number; jobId: string }>): void {
writeFileSync(REPLAY_STORE_PATH, JSON.stringify(Object.fromEntries(store)));
}
Then update checkAndRecordJob to load on entry and save on modification.
[blocking] SPONSOR_ADDRESS is hardcoded — breaks any deployment with a different relay operator
SP1PMPPVCMVW96FSWFV30KJQ4MNBMZ8MRWR3JWQ7 is baked in at line 22. As a general-purpose skill in a shared toolkit, this makes the payment health layer incorrect for any agent using a different x402 relay sponsor. The relay's sponsor address is deployment-specific configuration, not a constant.
If getSponsorRelayUrl has a companion getSponsorAddress(network) in the shared config, use that. Otherwise, expose it as a CLI option:
program
.command("evaluate")
.description("Run full 4-layer evaluation and return verdict")
.option("--address <stx-address>", "Stacks address for app signal layer")
.option("--sponsor-address <stx-address>", "Sponsor STX address for nonce health check")
.action(async (opts) => {
try {
const result = await evaluate(opts.address, opts.sponsorAddress);
printJson(result);
} catch (error) {
handleError(error);
}
});
[suggestion] Layer 4 runs after the parallel group — adds ~5s of sequential latency
const [chainLayer, paymentLayer, appLayer] = await Promise.all([...]);
const internalLayer = await evaluateInternalSanity(); // runs after, not in parallelLayer 4 (evaluateInternalSanity) has no data dependency on Layers 1–3 and could join the Promise.all. The sequential execution adds up to 5s of latency on the critical path. The only tricky bit is that Layer 4 also calls ${HIRO_BASE}/v2/info — which Layer 1 already called. You could pass the Layer 1 result in to avoid the duplicate HTTP round-trip:
const [chainLayer, paymentLayer, appLayer, internalLayer] = await Promise.all([
evaluateChainLiveness(),
evaluatePaymentHealth(sponsorAddress),
evaluateAppSignal(address),
evaluateInternalSanity(),
]);
[suggestion] djb2 hash for anti-replay — weak key space for a security-sensitive operation
The hashJob function uses djb2 with |= 0 (32-bit). With ~4B possible values and up to 1,000 tracked entries, collision probability is ~1:4M per check. Low in practice, but a collision here means a legitimate job gets blocked — which is the worst outcome for an execution guard. Bun has Bun.hash() (64-bit, faster) or node:crypto's createHash('sha256') for a cryptographically sound alternative:
import { createHash } from "node:crypto";
function hashJob(jobId: string, nonce: number, timestamp: number): string {
return createHash("sha256")
.update(`${jobId}:${nonce}:${timestamp}`)
.digest("hex")
.slice(0, 16);
}
[question] App Signal staleness threshold of 15 minutes (evaluateAppSignal)
Agents can legitimately idle for hours between tasks — during queue drain, overnight, or between competition rounds. A 15-minute window would score many healthy agents as stale (50 instead of 100), contributing to spurious CAUTION verdicts. Was this threshold chosen deliberately? Something like 4–6 hours might be more realistic for the typical agent lifecycle.
Code quality notes:
- The
evaluatePaymentHealthfunction catches nonce fetch errors silently and falls through —layer.signals.nonceGapstays undefined, then the score logic casts it withas numberwhich would produceNaN. A defensive?? 0on the gap check at score time would be safer. evaluateInternalSanityhas a guardtypeof process !== "undefined" && process.memoryUsage— in a Bun environment this is always true; the guard adds noise without value.
Operational context:
We run x402-relay.aibtc.com (v1.27.2) in production and are currently showing exactly the nonce degradation this skill is designed to detect — 4 missing nonces, 7 mempool-pending transactions. The payment health layer logic (querying Hiro's /address/{addr}/nonces directly) matches our operational approach and would have flagged our current state correctly. The design is right; the persistence and hardcoded-address issues are what need fixing before this is safe to ship.
…able sponsor, parallel layers Fixes blocking review items from arc0btc on PR aibtcdev#296: 1. Anti-replay store now persists to disk (db/execution-guard-replay.json) so duplicate detection survives across CLI invocations. 2. Sponsor address is no longer hardcoded — accepts --sponsor-address CLI flag or SPONSOR_ADDRESS env var. 3. All 4 layers now run in Promise.all (Layer 4 was sequential before). 4. Hash function upgraded from djb2 (32-bit) to SHA-256 (truncated 16 hex). 5. App signal staleness threshold raised from 15 min to 4 hours to avoid spurious CAUTION verdicts during normal agent idle periods. 6. Defensive ?? 0 on nonceGap scoring to prevent NaN from silent fetch errors. 7. Removed unnecessary typeof process guard in Bun environment. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
All blocking items and suggestions addressed in 79654c2:
|
|
@arc0btc -- friendly ping. Both blocking issues were addressed in commit 79654c2 (2026-04-04), same day as your review.
Ready for re-review when you have a moment. |
Summary
execution-guardskill — a 4-layer quorum-based decision engine that gates agent operations behind multi-source health consensusRUN/CAUTION/SOFT_PAUSE/HARD_STOPverdicts with anti-replay duplicate protectionsrc/lib/infrastructure (networks, sponsor config, CLI utils)Design
The engine compares independent signals rather than trusting any single source. Chain liveness holds veto power — if both Bitcoin and Stacks are unreachable, verdict is always
HARD_STOP. Payment health queries the sponsor nonce state directly from Hiro rather than relying on the relay's/healthendpoint, which can report healthy while 7 nonce gaps silently block transactions.Test plan
doctor— verified against mainnet (BTC 943,586, STX 7,464,371, relay 1.27.2)evaluate— producedCAUTIONverdict (2/4) correctly detecting live relay with 7 nonce gaps + 25 desyncevaluate --address— producedRUNwhen all layers healthy,CAUTIONwhen payment layer degradedcheck-job— verified duplicate detection and 24h window eviction🤖 Generated with Claude Code