Skip to content

feat: reactive engine expansion — DAG mutation, ephemeral proxy, HITL gates#11

Merged
Emperiusm merged 22 commits intomainfrom
feature/reactive-engine-expansion
Apr 13, 2026
Merged

feat: reactive engine expansion — DAG mutation, ephemeral proxy, HITL gates#11
Emperiusm merged 22 commits intomainfrom
feature/reactive-engine-expansion

Conversation

@Emperiusm
Copy link
Copy Markdown
Owner

Summary

Transforms OpenTools from a static scan pipeline into a reactive, operator-gated execution engine with ephemeral infrastructure support.

Phase A — Dynamic DAG Mutation

  • OutputAnalyzer protocol with Nmap + Nuclei analyzers extracting structured intel (services, vulns, URLs) from tool output
  • KillChainState accumulates attack surface knowledge across all completed tasks
  • MutationStrategy protocol with RedisProbeStrategy auto-pivoting on discovered services
  • Engine integration: mutation layer hooks into _mark_completed synchronously — zero race conditions

Phase B — Ephemeral Proxy Routing

  • CloudNodeProvider ABC with DigitalOcean + Vultr implementations (httpx, async)
  • Vultr provider enforces sshkey_id in creation payload for immediate SSH tunnel auth
  • ephemeral_proxy AsyncContextManager with shielded teardown (guaranteed node destruction even under cancellation)
  • ProxiedShellExecutor routes NETWORK_ISOLATED tasks through auto-provisioned SOCKS5 proxies
  • Orphan sweeper cleans up leaked nodes on startup

Phase C — HITL Approval Gate

  • Execution wrapper model (not a separate TaskType) — requires_approval metadata block on any task
  • Persistence-first: SQLite is always the source of truth, asyncio.Event is just a notification tripwire
  • Write-before-signal: FastAPI routes write decision to DB before calling event.set()
  • Read-after-wake: gate executor reads authoritative status from DB, never trusts in-memory state
  • Database-owned expiry: approval_expires_at UTC timestamp survives server restarts
  • Dedicated approval_gate resource group (9999 slots) — sleeping gates cost zero CPU/RAM
  • FastAPI endpoints: list/approve/reject with user-scoped authorization

Review fixes applied:

  • Command injection guard on strategy-spawned task hosts (_validate_host regex)
  • ApprovalRegistry wired into _active_scans + resource pool configured
  • Gate endpoints scoped by user_id via get_scan ownership check

Test plan

  • 693 CLI scanner tests passing (34 new mutation + 37 new infra + 16 new approval + 6 engine integration)
  • 3 web backend gate route tests passing
  • Zero regressions against existing 676 scanner tests
  • Existing engine tests (31) pass with mutation + gate additions
  • Spec compliance reviewed for Tasks 1, 2, 3, 4, 12 (critical integration points)
  • Code quality review on full branch — C1/C2/I7/I8 fixed before merge

🤖 Generated with Claude Code

Emperiusm and others added 22 commits April 13, 2026 02:36
Per-engagement interactive graph visualization using force-graph (vasturiano),
edge curation UI, MITRE ATT&CK phase coloring, server-side filtering for scale.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…r plans

Three planning documents for the reactive engine expansion:
- Phase A+B implementation plan (DAG mutation + ephemeral proxy routing)
- Phase C implementation plan (HITL approval gate + Vultr provider)
- Design spec for HITL gate (persistence-first, execution wrapper model)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
15 tasks covering backend (subgraph endpoint, curation PATCH, drift DTO),
frontend (force-graph canvas, detail panel, filters, legend, empty state),
and browser verification.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…dels

Introduces the mutation layer data models: DiscoveredService, DiscoveredVuln,
IntelBundle, and KillChainState. KillChainState accumulates attack surface
knowledge across completed tasks via ingest(), deduplicating on canonical keys.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…otations

Include port in KillChainState.ingest() vuln key (host:port:template_id) to
prevent silent data loss when the same template fires on multiple ports. Add
__all__ re-exports to mutation __init__.py. Tighten bare dict annotations to
dict[str, Any]. Rename test_immutable_model to test_round_trip_serialization.
Add two new tests covering multi-port and no-port vuln collision scenarios.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…yzers

Implements the OutputAnalyzer runtime-checkable protocol, NmapAnalyzer
(XML, open-ports-only), NucleiAnalyzer (JSON-lines), and AnalyzerRegistry
with register_builtins(); 33 tests, all passing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Introduces a runtime-checkable MutationStrategy Protocol and a concrete
RedisProbeStrategy that spawns redis-cli INFO tasks whenever nmap/masscan
discovers Redis services. Self-tracking via _spawned_keys makes evaluate()
idempotent without engine callbacks. 25 tests, all passing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…→ inject

Wire OutputAnalyzer registry, MutationStrategy list, and KillChainState
into ScanEngine._mark_completed so discovered intel triggers dynamic task
synthesis (e.g., nmap finds Redis → redis-cli probe injected). Harden
_inject_tasks with dependency validation. Add 6 integration tests covering
spawn, budget, coexistence with reactive edges, and the no-registry guard.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Introduces the ephemeral infrastructure provisioning package with:
- CloudNodeProvider ABC defining create_node/poll_status/destroy_node + wait_until_ready
- EphemeralNode pydantic model, ProvisioningError/ProvisioningTimeout exceptions
- DigitalOceanProvider using httpx.AsyncClient with from_token factory
- Comprehensive tests via httpx.MockTransport (19 tests, all passing)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…guard

VultrProvider implements CloudNodeProvider for Vultr REST API v2:
- sshkey_id sent as array (REQUIRED — absent key breaks SSH tunnel auth)
- 0.0.0.0 IP guard in poll_status: treats premature active+0.0.0.0 as still creating
- power_status == "running" check prevents false-ready on stopped active instances
- list_nodes_by_tag for orphan sweeping
- 18 dedicated Vultr tests in test_infra_provider.py, all passing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds ProxiedShellExecutor that transparently routes NETWORK_ISOLATED
tasks through an ephemeral SOCKS5 proxy node while running all other
tasks as plain subprocesses. Includes port counter for concurrent calls
and full test coverage.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds sweep_orphaned_nodes() that queries the provider for nodes tagged
with PROXY_TAG (leftover from crashed runs) and destroys them one by
one, tolerating per-node failures and providers that lack list support.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…oval fields on ScanTask

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…-before-signal

Adds approval_ticket_id and approval_expires_at to ScanTaskRecord, three gate endpoints (GET /gates, POST /gates/{id}/approve, POST /gates/{id}/reject) with write-before-signal semantics, two new ScanService methods (get_task_by_ticket, update_task_approval_status), and smoke tests confirming 404 routing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Global cross-engagement graph, Bayesian weight calibration with Beta priors,
timeline playback with temporal anchoring, Markdown path export, swim lane
Kill Chain layout, attack vector scoring (path risk + node pivotality).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ping

- Wire ApprovalRegistry into ScanAPI.execute() and store in _active_scans
  so web gate endpoints can signal the in-memory registry (C1)
- Configure AdaptiveResourcePool with approval_gate group_limits=9999 (I4)
- Add _validate_host() guard in RedisProbeStrategy to reject hosts with
  shell metacharacters before f-string interpolation (C2)
- Add user-scoped ownership check (svc.get_scan) in approve_gate and
  reject_gate before get_task_by_ticket (I7+I8)
- Add tests for host validation and command injection prevention

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Scanner section: dynamic mutation, HITL gates, ephemeral proxy routing
- Architecture diagram: mutation/, infra/, approval.py, proxied executor
- API routes: approval gates on scan endpoints
- Roadmap: Phase 3.5 with all implemented features
- Badge updates: test count, line count, PR count

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@Emperiusm Emperiusm merged commit ea809e1 into main Apr 13, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant